11 Commits (968e1e464a17b9b310c3b02c5754083829b9eed2)

Author SHA1 Message Date
  Martin Evans 92b9bbe779 Added methods to `SafeLLamaContextHandle` for KV cache manipulation 1 year ago
  Martin Evans 96c26c25f5
Merge pull request #445 from martindevans/stateless_executor_llama_decode 1 year ago
  Martin Evans 9fe878ae1f - Fixed example 1 year ago
  Martin Evans a2e29d393c Swapped `StatelessExecutor` to use `llama_decode`! 1 year ago
  Martin Evans 5b6e82a594 Improved the BatchedDecoding demo: 1 year ago
  Martin Evans 99969e538e - Removed some unused `eval` methods. 1 year ago
  Martin Evans 36a9335588 Removed `LLamaBatchSafeHandle` (using unmanaged memory, created by llama.cpp) and replaced it with a fully managed `LLamaBatch`. Modified the `BatchedDecoding` example to use new managed batch. 1 year ago
  Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 1 year ago
  Martin Evans b868b056f7 Added metadata overrides to `IModelParams` 1 year ago
  Martin Evans 479779e908 Some minor cleanup on example code: 2 years ago
  xbotter 521e36903c
🔀 Remove unused code and update examples 2 years ago