LLamaSharp/docs/xmldocs/llama.sampling.basesamplingpipeline.md

• BaseSamplingPipeline
• • Properties
  • • Grammar
    • • Property Value
  • Methods
  • • Sample(SafeLLamaContextHandle, Span<Single>, ReadOnlySpan<LLamaToken>)
    • • Parameters
      • Returns
    • Accept(SafeLLamaContextHandle, LLamaToken)
    • • Parameters
    • ProcessLogits(SafeLLamaContextHandle, Span<Single>, ReadOnlySpan<LLamaToken>)
    • • Parameters
    • ProcessTokenDataArray(SafeLLamaContextHandle, LLamaTokenDataArray, ReadOnlySpan<LLamaToken>)
    • • Parameters
      • Returns
    • Reset()
    • Clone()
    • • Returns
    • Dispose()

BaseSamplingPipeline

Namespace: LLama.Sampling

Base class for implementing custom sampling pipelines. This provides a helpful framework for implementing ISamplingPipeline.

public abstract class BaseSamplingPipeline : ISamplingPipeline, System.IDisposable

Inheritance Object → BaseSamplingPipeline

Implements ISamplingPipeline, IDisposable

Properties

Grammar

Grammar to constrain valid tokens

public SafeLLamaGrammarHandle Grammar { get; set; }

Property Value

SafeLLamaGrammarHandle

Methods

Sample(SafeLLamaContextHandle, Span<Single>, ReadOnlySpan<LLamaToken>)

public LLamaToken Sample(SafeLLamaContextHandle ctx, Span<float> logits, ReadOnlySpan<LLamaToken> lastTokens)

Parameters

ctx SafeLLamaContextHandle

logits Span<Single>

lastTokens ReadOnlySpan<LLamaToken>

Returns

LLamaToken

Accept(SafeLLamaContextHandle, LLamaToken)

public void Accept(SafeLLamaContextHandle ctx, LLamaToken token)

Parameters

ctx SafeLLamaContextHandle

token LLamaToken

ProcessLogits(SafeLLamaContextHandle, Span<Single>, ReadOnlySpan<LLamaToken>)

Process the raw logit values

protected abstract void ProcessLogits(SafeLLamaContextHandle ctx, Span<float> logits, ReadOnlySpan<LLamaToken> lastTokens)

Parameters

ctx SafeLLamaContextHandle

The context being sampled from

logits Span<Single>

The logits produced by the model

lastTokens ReadOnlySpan<LLamaToken>

A list of tokens recently returned by the model

ProcessTokenDataArray(SafeLLamaContextHandle, LLamaTokenDataArray, ReadOnlySpan<LLamaToken>)

Process the LLamaTokenDataArray and select a single token

protected abstract LLamaToken ProcessTokenDataArray(SafeLLamaContextHandle ctx, LLamaTokenDataArray candidates, ReadOnlySpan<LLamaToken> lastTokens)

Parameters

ctx SafeLLamaContextHandle

The context being sampled from

candidates LLamaTokenDataArray

The LLamaTokenDataArray data produced by the model

lastTokens ReadOnlySpan<LLamaToken>

A list of tokens recently returned by the model

Returns

LLamaToken

Reset()

public void Reset()

Clone()

public abstract ISamplingPipeline Clone()

Returns

ISamplingPipeline

Dispose()

public void Dispose()