6 Commits (43677c511c68f69d264e16f3ae6bf7e7f5fc4d5f)

Author SHA1 Message Date
  Martin Evans f0b0bbcbb7
Mutable Logits (#586) 1 year ago
  Martin Evans 91a7967869
`ReadOnlySpan<float>` in ISamplingPipeline (#538) 1 year ago
  Martin Evans b0acecf080 Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix). 1 year ago
  Martin Evans 2eb52b1630 made casts to/from int explicit, fixed places affected 1 year ago
  Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 1 year ago
  Martin Evans 835958398c - Removed the object wrappers and configurable pipeline, they can be better written in code. 1 year ago