5 Commits (91a7967869ec1d3441e9b819bc164d178d772b96)

Author SHA1 Message Date
  Martin Evans 91a7967869
`ReadOnlySpan<float>` in ISamplingPipeline (#538) 1 year ago
  Martin Evans b0acecf080 Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix). 1 year ago
  Martin Evans 2eb52b1630 made casts to/from int explicit, fixed places affected 1 year ago
  Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 1 year ago
  Martin Evans 835958398c - Removed the object wrappers and configurable pipeline, they can be better written in code. 1 year ago