22 Commits (experimental_cpp)

Author SHA1 Message Date
  Martin Evans a2e29d393c Swapped `StatelessExecutor` to use `llama_decode`! 1 year ago
  Martin Evans 99969e538e - Removed some unused `eval` methods. 1 year ago
  Martin Evans 835958398c - Removed the object wrappers and configurable pipeline, they can be better written in code. 1 year ago
  Martin Evans 3afc007499 - Added "protected" logits, instead of the awkward save/load mechanism 1 year ago
  Martin Evans 09bc688b3c Skipped slow test again 2 years ago
  Martin Evans cdf20d3c7a Added timing to stateless test 2 years ago
  Martin Evans 7e3cde4c13 Moved helper methods into `LLamaBatchSafeHandle` 2 years ago
  Martin Evans ccb8afae46 Cleaned up stateless executor as preparation for changing it to use the new batched decoding system. 2 years ago
  Martin Evans d8434ea9d6
Merge pull request #185 from martindevans/wip_major_api_change 2 years ago
  Martin Evans 1f8c94e386 Added in the `special` parameter to the tokenizer (introduced in https://github.com/ggerganov/llama.cpp/pull/3538) 2 years ago
  Martin Evans efb0664df0 - Added new binaries 2 years ago
  Martin Evans 45118520fa - Improved coverage of `GBNFGrammarParser` up to 96% 2 years ago
  Martin Evans 2a38808bca - Added threads to context params, replaced all thread args with `uint?` 2 years ago
  Martin Evans 0d40338692 Fixed out-of-context handling in stateless executor 2 years ago
  Martin Evans 3f80190f85 Minimal changes required to remove non-async inference. 2 years ago
  Martin Evans 0c98ae1955 Passing ctx to `llama_token_nl(_ctx)` 2 years ago
  Martin Evans a9e6f21ab8 - Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring! 2 years ago
  Martin Evans 48bc0a6f8a Doe the same for the second test, hopefully fixing CI 2 years ago
  Martin Evans 6f2ab8e039 Not asserting the answer, just that it didn't change 2 years ago
  Martin Evans e7b217f462 Fixed out of context logic 2 years ago
  Martin Evans 4738c26299 - Reduced context size of test, to speed it up 2 years ago
  Martin Evans 4d0c044b9f Added tests for the StatelessExecutor, one is currently failing 2 years ago