Rinne
4e83e48ad1
Merge pull request #122 from martindevans/gguf
Add GGUF support
2 years ago
Martin Evans
a70c7170dd
- Created a higher level `Grammar` class which is immutable and contains a list of grammar rules. This is the main "entry point" to the grammar system.
- Made all the mechanics of grammar parsing (GBNFGrammarParser, ParseState) internal. Just call `Grammar.Parse("whatever")`.
- Added a `GrammarRule` class which validates elements on construction (this allows constructing grammar without parsing GBNF).
- It should be impossible for a `GrammarRule` to represent an invalid rule.
2 years ago
SignalRT
fb007e5921
Changes to compile in VS Mac + change model to llama2
This commit includes changes to compile en VS Mac + changest to use llama2 not codellama.
It includes MacOS binaries in memory and metal
2 years ago
Mihai
920efaca44
Add original test
2 years ago
Mihai
2ae1891c13
Bug fixes after running tests.
SymbolIds is now SortedDictionary (although I'm not sure it really needs to be) because the test was failing due to expected value being in another order. The C++ data structure if SymbolIds is std::map<std::string, uint32_t> so the items are ordered by key.
2 years ago
Mihai
0bd495276b
Add initial tests + fix bugs. Still WIP since the test is failing.
2 years ago
Martin Evans
ba49ea2991
Removed hardcoded paths from projects, modified Runtime.targets to exclude missing binaries
2 years ago
Martin Evans
2022b82947
Added binaries generated by this action: https://github.com/SciSharp/LLamaSharp/actions/runs/6002797872/job/16279896150
Based on this version: 6b73ef1201
2 years ago
Martin Evans
95dc12dd76
Switched to codellama-7b.gguf in tests (probably temporarily)
2 years ago
Martin Evans
0c98ae1955
Passing ctx to `llama_token_nl(_ctx)`
2 years ago
Martin Evans
2056078aef
Initial changes required for GGUF support
2 years ago
Martin Evans
5a6c6de0dc
Merge pull request #115 from martindevans/model_params_record
ModelsParams record class
2 years ago
Martin Evans
ebacdb666d
- Moved the lower level state get/set methods onto SafeLLamaContextHandle
- Used those methods to add a `Clone` method to SafeLLamaContextHandle
- Simplified `LLamaContext` by using the new methods
- Sealed `LLamaContext` and `LLamaEmbedder`
2 years ago
Martin Evans
77aa5fa0d0
Added `JsonConverter` attribute, so System.Text.Json serialization is seamless
2 years ago
Martin Evans
634bd3feed
Loosened requirements even more!
2 years ago
Martin Evans
9587699a60
loosened requirements even further for CI
2 years ago
Martin Evans
bc70358b3e
expanded epsilon enough to allow CI to pass
2 years ago
Martin Evans
08501db155
cleaned up basic test a bit
2 years ago
Martin Evans
498ed109c1
Fixed dot product based test
2 years ago
Martin Evans
35e48691ea
Added a known correct value to the unit tests
2 years ago
Martin Evans
dff308e31b
Added a test for the LLamaEmbedder
2 years ago
Martin Evans
93f24f8a51
Switched to properly typed `Encoding` property
2 years ago
Martin Evans
29df14cd9c
Converted ModelParams into a `record` class. This has several advantages:
- Equality, hashing etc all implemented automatically
- Default values are defined in just one place (the properties) instead of the constructor as well
- Added test to ensure that serialization works properly
2 years ago
Martin Evans
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
- Deleted `NativeInfo` (internal class, not used anywhere)
2 years ago
Martin Evans
9fc17f3136
Fixed unit tests
2 years ago
Martin Evans
759ae26f36
Merge branch 'master' into grammar_basics
2 years ago
Martin Evans
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2 years ago
Martin Evans
48bc0a6f8a
Doe the same for the second test, hopefully fixing CI
2 years ago
Martin Evans
6f2ab8e039
Not asserting the answer, just that it didn't change
2 years ago
Martin Evans
e7b217f462
Fixed out of context logic
2 years ago
Martin Evans
4738c26299
- Reduced context size of test, to speed it up
- Removed some unnecessary `ToArray` calls
- Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from
2 years ago
Martin Evans
4d0c044b9f
Added tests for the StatelessExecutor, one is currently failing
2 years ago
Martin Evans
1db7292b05
Fixed conflicts caused by merging of multi context PR
2 years ago
Martin Evans
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
- Integrated grammar into sampling
- Added a test for the grammar sampling
2 years ago
Martin Evans
76d991f376
Removed embedding test, moved to another PR
2 years ago
Martin Evans
aeb7943710
Removed Console.WriteLine
2 years ago
Martin Evans
6473f8d5e5
Temporarily added a `Console.WriteLine` into the test, to print the embedding vector for "cat" in CI
2 years ago
Martin Evans
1b35be2e0c
Added some additional basic tests
2 years ago
Martin Evans
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2 years ago
Martin Evans
fe3bd11dfa
Merge branch 'master' into master
2 years ago
Martin Evans
d6fc83e981
- Added a folder with a 7B Llama2 model, automatically downloaded from huggingface. This can be used for unit tests.
- Caching that folder, so CI only has to download it once.
2 years ago
SignalRT
528a7aafcb
Merge test changes from @martindevans
The poc of the test is working know. Finally the problem error seems to be related with the process stopping.
Once I changed the context with today llama.cpp binaries is working OK
2 years ago
Tim Miller
bbd2650cf0
Include runtime targets file
2 years ago
Yaohui Liu
56c56b9c51
refactor: drop LLamaModelV1.
2 years ago
Yaohui Liu
856d6549de
build: add linux support.
2 years ago
Yaohui Liu
8978410eab
docs: add examples for running LLamaSharp.
2 years ago
Yaohui Liu
5a79edeb51
feat: add the framework and basic usages.
2 years ago