LLamaSharp/docs/xmldocs/llama.llamamodel.md

• LLamaModel
• • Properties
  • • ContextSize
    • • Property Value
    • Params
    • • Property Value
    • NativeHandle
    • • Property Value
    • Encoding
    • • Property Value
  • Constructors
  • • LLamaModel(ModelParams, String, ILLamaLogger)
    • • Parameters
  • Methods
  • • Tokenize(String, Boolean)
    • • Parameters
      • Returns
    • DeTokenize(IEnumerable<Int32>)
    • • Parameters
      • Returns
    • SaveState(String)
    • • Parameters
    • GetStateData()
    • • Returns
    • LoadState(String)
    • • Parameters
      • Exceptions
    • LoadState(Byte[])
    • • Parameters
      • Exceptions
    • Sample(LLamaTokenDataArray, Single, MiroStateType, Single, Single, Int32, Single, Single, Single)
    • • Parameters
      • Returns
    • ApplyPenalty(IEnumerable<Int32>, Dictionary<Int32, Single>, Int32, Single, Single, Single, Boolean)
    • • Parameters
      • Returns
    • Eval(Int32[], Int32)
    • • Parameters
      • Returns
      • Exceptions
    • GenerateResult(IEnumerable<Int32>)
    • • Parameters
      • Returns
    • Dispose()

LLamaModel

Namespace: LLama

The abstraction of a LLama model, which holds the context in the native library.

public class LLamaModel : System.IDisposable

Inheritance Object → LLamaModel

Implements IDisposable

Properties

ContextSize

The context size.

public int ContextSize { get; }

Property Value

Int32

Params

The model params set for this model.

public ModelParams Params { get; set; }

Property Value

ModelParams

NativeHandle

The native handle, which is used to be passed to the native APIs. Please avoid using it
unless you know what is the usage of the Native API.

public SafeLLamaContextHandle NativeHandle { get; }

Property Value

SafeLLamaContextHandle

Encoding

The encoding set for this model to deal with text input.

public Encoding Encoding { get; }

Property Value

Encoding

Constructors

LLamaModel(ModelParams, String, ILLamaLogger)

public LLamaModel(ModelParams Params, string encoding, ILLamaLogger logger)

Parameters

Params ModelParams

Model params.

encoding String

Encoding to deal with text input.

logger ILLamaLogger

The logger.

Methods

Tokenize(String, Boolean)

Tokenize a string.

public IEnumerable<int> Tokenize(string text, bool addBos)

Parameters

text String

addBos Boolean

Whether to add a bos to the text.

Returns

IEnumerable<Int32>

DeTokenize(IEnumerable<Int32>)

Detokenize the tokens to text.

public string DeTokenize(IEnumerable<int> tokens)

Parameters

tokens IEnumerable<Int32>

Returns

String

SaveState(String)

Save the state to specified path.

public void SaveState(string filename)

Parameters

filename String

GetStateData()

Get the state data as a byte array.

public Byte[] GetStateData()

Returns

Byte[]

LoadState(String)

Load the state from specified path.

public void LoadState(string filename)

Parameters

filename String

Exceptions

RuntimeError

LoadState(Byte[])

Load the state from memory.

public void LoadState(Byte[] stateData)

Parameters

stateData Byte[]

Exceptions

RuntimeError

Sample(LLamaTokenDataArray, Single, MiroStateType, Single, Single, Int32, Single, Single, Single)

Perform the sampling. Please don't use it unless you fully know what it does.

public int Sample(LLamaTokenDataArray candidates, float temperature, MiroStateType mirostat, float mirostatTau, float mirostatEta, int topK, float topP, float tfsZ, float typicalP)

Parameters

candidates LLamaTokenDataArray

temperature Single

mirostat MiroStateType

mirostatTau Single

mirostatEta Single

topK Int32

topP Single

tfsZ Single

typicalP Single

Returns

Int32

ApplyPenalty(IEnumerable<Int32>, Dictionary<Int32, Single>, Int32, Single, Single, Single, Boolean)

Apply the penalty for the tokens. Please don't use it unless you fully know what it does.

public LLamaTokenDataArray ApplyPenalty(IEnumerable<int> lastTokens, Dictionary<int, float> logitBias, int repeatLastTokensCount, float repeatPenalty, float alphaFrequency, float alphaPresence, bool penalizeNL)

Parameters

lastTokens IEnumerable<Int32>

logitBias Dictionary<Int32, Single>

repeatLastTokensCount Int32

repeatPenalty Single

alphaFrequency Single

alphaPresence Single

penalizeNL Boolean

Returns

LLamaTokenDataArray

Eval(Int32[], Int32)

public int Eval(Int32[] tokens, int pastTokensCount)

Parameters

tokens Int32[]

pastTokensCount Int32

Returns

Int32

The updated pastTokensCount.

Exceptions

RuntimeError

GenerateResult(IEnumerable<Int32>)

internal IEnumerable<string> GenerateResult(IEnumerable<int> ids)

Parameters

ids IEnumerable<Int32>

Returns

IEnumerable<String>

Dispose()

public void Dispose()