LLamaSharp/docs/xmldocs/llama.llamacontext.md

• LLamaContext
• • Properties
  • • VocabCount
    • • Property Value
    • ContextSize
    • • Property Value
    • EmbeddingSize
    • • Property Value
    • Params
    • • Property Value
    • NativeHandle
    • • Property Value
    • Encoding
    • • Property Value
    • EmbeddingLength
    • • Property Value
  • Constructors
  • • LLamaContext(IModelParams, ILLamaLogger)
    • • Caution
      • Parameters
    • LLamaContext(LLamaWeights, IModelParams, ILLamaLogger)
    • • Parameters
      • Exceptions
  • Methods
  • • Clone()
    • • Returns
    • Tokenize(String, Boolean)
    • • Parameters
      • Returns
    • DeTokenize(IEnumerable<Int32>)
    • • Parameters
      • Returns
    • SaveState(String)
    • • Parameters
    • GetStateData()
    • • Caution
      • Returns
    • GetState()
    • • Returns
    • LoadState(String)
    • • Parameters
      • Exceptions
    • LoadState(Byte[])
    • • Parameters
      • Exceptions
    • LoadState(State)
    • • Parameters
      • Exceptions
    • Sample(LLamaTokenDataArray, Nullable`1&, Single, MirostatType, Single, Single, Int32, Single, Single, Single, SafeLLamaGrammarHandle)
    • • Parameters
      • Returns
    • ApplyPenalty(IEnumerable<Int32>, Dictionary<Int32, Single>, Int32, Single, Single, Single, Boolean)
    • • Parameters
      • Returns
    • Eval(Int32[], Int32)
    • • Parameters
      • Returns
      • Exceptions
    • Eval(List<Int32>, Int32)
    • • Parameters
      • Returns
      • Exceptions
    • Eval(ReadOnlyMemory<Int32>, Int32)
    • • Parameters
      • Returns
      • Exceptions
    • Eval(ReadOnlySpan<Int32>, Int32)
    • • Parameters
      • Returns
      • Exceptions
    • GenerateResult(IEnumerable<Int32>)
    • • Parameters
      • Returns
    • TokenToString(Int32)
    • • Parameters
      • Returns
    • Dispose()

LLamaContext

Namespace: LLama

A llama_context, which holds all the context required to interact with a model

public sealed class LLamaContext : System.IDisposable

Inheritance Object → LLamaContext

Implements IDisposable

Properties

VocabCount

Total number of tokens in vocabulary of this model

public int VocabCount { get; }

Property Value

Int32

ContextSize

Total number of tokens in the context

public int ContextSize { get; }

Property Value

Int32

EmbeddingSize

Dimension of embedding vectors

public int EmbeddingSize { get; }

Property Value

Int32

Params

The model params set for this model.

public IModelParams Params { get; set; }

Property Value

IModelParams

NativeHandle

The native handle, which is used to be passed to the native APIs

public SafeLLamaContextHandle NativeHandle { get; }

Property Value

SafeLLamaContextHandle

Remarks:

Be careful how you use this!

Encoding

The encoding set for this model to deal with text input.

public Encoding Encoding { get; }

Property Value

Encoding

EmbeddingLength

The embedding length of the model, also known as n_embed

public int EmbeddingLength { get; }

Property Value

Int32

Constructors

LLamaContext(IModelParams, ILLamaLogger)

Caution

Use the LLamaWeights.CreateContext instead

---

public LLamaContext(IModelParams params, ILLamaLogger logger)

Parameters

params IModelParams

Model params.

logger ILLamaLogger

The logger.

LLamaContext(LLamaWeights, IModelParams, ILLamaLogger)

Create a new LLamaContext for the given LLamaWeights

public LLamaContext(LLamaWeights model, IModelParams params, ILLamaLogger logger)

Parameters

model LLamaWeights

params IModelParams

logger ILLamaLogger

Exceptions

ObjectDisposedException

Methods

Clone()

Create a copy of the current state of this context

public LLamaContext Clone()

Returns

LLamaContext

Tokenize(String, Boolean)

Tokenize a string.

public Int32[] Tokenize(string text, bool addBos)

Parameters

text String

addBos Boolean

Whether to add a bos to the text.

Returns

Int32[]

DeTokenize(IEnumerable<Int32>)

Detokenize the tokens to text.

public string DeTokenize(IEnumerable<int> tokens)

Parameters

tokens IEnumerable<Int32>

Returns

String

SaveState(String)

Save the state to specified path.

public void SaveState(string filename)

Parameters

filename String

GetStateData()

Caution

Use GetState instead, this supports larger states (over 2GB)

---

Get the state data as a byte array.

public Byte[] GetStateData()

Returns

Byte[]

GetState()

Get the state data as an opaque handle

public State GetState()

Returns

State

LoadState(String)

Load the state from specified path.

public void LoadState(string filename)

Parameters

filename String

Exceptions

RuntimeError

LoadState(Byte[])

Load the state from memory.

public void LoadState(Byte[] stateData)

Parameters

stateData Byte[]

Exceptions

RuntimeError

LoadState(State)

Load the state from memory.

public void LoadState(State state)

Parameters

state State

Exceptions

RuntimeError

Sample(LLamaTokenDataArray, Nullable`1&, Single, MirostatType, Single, Single, Int32, Single, Single, Single, SafeLLamaGrammarHandle)

Perform the sampling. Please don't use it unless you fully know what it does.

public int Sample(LLamaTokenDataArray candidates, Nullable`1& mirostat_mu, float temperature, MirostatType mirostat, float mirostatTau, float mirostatEta, int topK, float topP, float tfsZ, float typicalP, SafeLLamaGrammarHandle grammar)

Parameters

candidates LLamaTokenDataArray

mirostat_mu Nullable`1&

temperature Single

mirostat MirostatType

mirostatTau Single

mirostatEta Single

topK Int32

topP Single

tfsZ Single

typicalP Single

grammar SafeLLamaGrammarHandle

Returns

Int32

ApplyPenalty(IEnumerable<Int32>, Dictionary<Int32, Single>, Int32, Single, Single, Single, Boolean)

Apply the penalty for the tokens. Please don't use it unless you fully know what it does.

public LLamaTokenDataArray ApplyPenalty(IEnumerable<int> lastTokens, Dictionary<int, float> logitBias, int repeatLastTokensCount, float repeatPenalty, float alphaFrequency, float alphaPresence, bool penalizeNL)

Parameters

lastTokens IEnumerable<Int32>

logitBias Dictionary<Int32, Single>

repeatLastTokensCount Int32

repeatPenalty Single

alphaFrequency Single

alphaPresence Single

penalizeNL Boolean

Returns

LLamaTokenDataArray

Eval(Int32[], Int32)

public int Eval(Int32[] tokens, int pastTokensCount)

Parameters

tokens Int32[]

pastTokensCount Int32

Returns

Int32

The updated pastTokensCount.

Exceptions

RuntimeError

Eval(List<Int32>, Int32)

public int Eval(List<int> tokens, int pastTokensCount)

Parameters

tokens List<Int32>

pastTokensCount Int32

Returns

Int32

The updated pastTokensCount.

Exceptions

RuntimeError

Eval(ReadOnlyMemory<Int32>, Int32)

public int Eval(ReadOnlyMemory<int> tokens, int pastTokensCount)

Parameters

tokens ReadOnlyMemory<Int32>

pastTokensCount Int32

Returns

Int32

The updated pastTokensCount.

Exceptions

RuntimeError

Eval(ReadOnlySpan<Int32>, Int32)

public int Eval(ReadOnlySpan<int> tokens, int pastTokensCount)

Parameters

tokens ReadOnlySpan<Int32>

pastTokensCount Int32

Returns

Int32

The updated pastTokensCount.

Exceptions

RuntimeError

GenerateResult(IEnumerable<Int32>)

internal IEnumerable<string> GenerateResult(IEnumerable<int> ids)

Parameters

ids IEnumerable<Int32>

Returns

IEnumerable<String>

TokenToString(Int32)

Convert a token into a string

public string TokenToString(int token)

Parameters

token Int32

Returns

String

Dispose()

public void Dispose()