5 Commits (ec8f83236545a1989df2f75da4e1d8d0345b0407)

Author SHA1 Message Date
  Martin Evans f0b0bbcbb7
Mutable Logits (#586) 1 year ago
  Martin Evans 91a7967869
`ReadOnlySpan<float>` in ISamplingPipeline (#538) 1 year ago
  Martin Evans b0acecf080 Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix). 1 year ago
  Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 1 year ago
  Martin Evans 835958398c - Removed the object wrappers and configurable pipeline, they can be better written in code. 1 year ago