Martin Evans
d47b6afe4d
Normalizing embeddings in `LLamaEmbedder`. As is done in llama.cpp: 2891c8aa9a/examples/embedding/embedding.cpp (L92)
1 year ago
Martin Evans
744758f110
Using `AddRange` in `LLamaEmbedder`
1 year ago
Martin Evans
c9c8cd0d62
- Swapped embeddings generator to use `llama_decode`
- Modified `GetEmbeddings` method to be async
1 year ago
Martin Evans
9b995510d6
Removed all setters in `IModelParams` and `IContextParams`, allowing implementations to be immutable.
1 year ago
Martin Evans
99969e538e
- Removed some unused `eval` methods.
- Added a `DecodeAsync` overload which runs the work in a task
- Replaced some `NativeHandle` usage in `BatchedDecoding` with higher level equivalents.
- Made the `LLamaBatch` grow when token capacity is exceeded, removing the need to manage token capacity externally.
1 year ago
Martin Evans
c002642268
- Removed some `unsafe` where it wasn't necessary
- Wrapped some native functions which take (pointer, length) in function which take a `span` instead.
1 year ago
Martin Evans
f860f88c36
Code cleanup driven by R# suggestions:
- Made `NativeApi` into a `static class` (it's not intended to be instantiated)
- Moved `LLamaTokenType` enum out into a separate file
- Made `LLamaSeqId` and `LLamaPos` into `record struct`, convenient to have equality etc
1 year ago
xbotter
211ce12bf5
LLamaEmbedder exposes the Context
1 year ago
xbotter
13a312b4ec
update sk to 1.0.0-rc3 & km to 0.18
1 year ago
Martin Evans
7ec318aab5
Added logging to embedder too
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
Martin Evans
669ae47ef7
- Split parameters into two interfaces
- params contains a list of loras, instead of just one
2 years ago
Rinne
4e83e48ad1
Merge pull request #122 from martindevans/gguf
Add GGUF support
2 years ago
Tim Miller
326c802be7
Have weights generate context
2 years ago
Tim Miller
9a1d6f99f2
Add Semantic Kernel support
2 years ago
sa_ddam213
a5d742b72c
Fix Tokenize of new line, Remove space inserts
2 years ago
Martin Evans
ebacdb666d
- Moved the lower level state get/set methods onto SafeLLamaContextHandle
- Used those methods to add a `Clone` method to SafeLLamaContextHandle
- Simplified `LLamaContext` by using the new methods
- Sealed `LLamaContext` and `LLamaEmbedder`
2 years ago
Martin Evans
058c4e84b1
Rewritten LLamaEmbedder to use `LLamaContext` instead of the lower level handles
2 years ago
Martin Evans
6473f8d5e5
Temporarily added a `Console.WriteLine` into the test, to print the embedding vector for "cat" in CI
2 years ago
Martin Evans
1b35be2e0c
Added some additional basic tests
2 years ago
Martin Evans
cd3cf2b77d
- Moved tokenization from `Utils.Tokenize` into `SafeLLamaContextHandle.Tokenize`, one less thing in `Utils`.
- Also refactored it to return an `int[]` instead of an `IEnumerable<int>`, solving the "multiple enumeration" problems at the source!
2 years ago
sa_ddam213
2a04e31b7d
ModelParams abstraction
2 years ago
Martin Evans
7cf1f8ac28
Fixed multiple cases where an `IEnumerable<T>` was enumerated multiple times.
2 years ago
Yaohui Liu
6c400e64c2
docs: publiash documentation 0.4.
2 years ago
Yaohui Liu
b20b6f209e
docs: add some xml comments.
2 years ago
Yaohui Liu
3bf74ec9b9
feat: add chat session for refactored code.
2 years ago
Yaohui Liu
5679e08718
feat: add ILLamaExecutor.InferAsync.
2 years ago
Yaohui Liu
264fb9a706
refactor: LLamaModel and LLamaExecutor.
2 years ago
Yaohui Liu
afedd3c949
fix: errors when input is not English or too long.
2 years ago
Yaohui Liu
aa2b064d1d
fix: add IDisposable to model classes.
2 years ago
Yaohui Liu
0958bbac2c
feat: add get-embedding api to LLamaModel.
2 years ago