12 Commits (829f32b27dcf99bb535ab7c47ab132e1d68ddff3)

Author SHA1 Message Date
  Martin Evans a9e6f21ab8 - Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring! 2 years ago
  Martin Evans ae8ef17a4a - Added various convenience overloads to `LLamaContext.Eval` 2 years ago
  Martin Evans 479ff57853 Renamed `EmbeddingCount` to `EmbeddingSize` 2 years ago
  Martin Evans d0a7a8fcd6 - Cleaned up disposal in LLamaContext 2 years ago
  Martin Evans f3511e390f WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving. 2 years ago
  Martin Evans 2b2d3af26b Moved `Eval` out of `Utils` and into `SafeLLamaContextHandle` 2 years ago
  Martin Evans 0e5e00e300 Moved `TokenToString` from Utils into `SafeLLamaContextHandle` (thin wrappers around the same method in `SafeLlamaModelHandle`) 2 years ago
  Martin Evans 2d811b2603 - Moved `GetLogits` into `SafeLLamaContextHandle` 2 years ago
  Martin Evans cd3cf2b77d - Moved tokenization from `Utils.Tokenize` into `SafeLLamaContextHandle.Tokenize`, one less thing in `Utils`. 2 years ago
  Martin Evans f16aa58e12 Updated to use the new loading system in llama (llama_state). This new system has split model weights and contexts into two separate things, allowing one set of weights to be shared between many contexts. 2 years ago
  Yaohui Liu 0958bbac2c
feat: add get-embedding api to LLamaModel. 2 years ago
  Yaohui Liu 5a79edeb51
feat: add the framework and basic usages. 2 years ago