Martin Evans
5b6e82a594
Improved the BatchedDecoding demo:
- using less `NativeHandle`
- Using `StreamingTokenDecoder` instead of obsolete detokenize method
1 year ago
Martin Evans
42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
1 year ago
Martin Evans
f860f88c36
Code cleanup driven by R# suggestions:
- Made `NativeApi` into a `static class` (it's not intended to be instantiated)
- Moved `LLamaTokenType` enum out into a separate file
- Made `LLamaSeqId` and `LLamaPos` into `record struct`, convenient to have equality etc
1 year ago
Martin Evans
2a1e1b6183
Removed unused imports
1 year ago
Martin Evans
a2bae178fa
Added a `Metadata` property to `LLamaWeights`
1 year ago
Martin Evans
3c5547b2b7
Reduced some uses of `NativeApi` in `BatchedDecoding` by adding some helper methods
2 years ago
Martin Evans
f1e5a8f995
- Passing the `ILogger` through to every call of `CreateContext`
- Passing `ILogger` into executors
2 years ago
sa_ddam213
4ec9aed47a
Revert LLamasSharp project changes
2 years ago
sa_ddam213
b4b4000342
Merge branch 'master' into upstream_master
# Conflicts:
# LLama.Web/Common/ModelOptions.cs
# LLama.Web/Services/ConnectionSessionService.cs
# LLama/LLamaStatelessExecutor.cs
# LLama/LLamaWeights.cs
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
sa_ddam213
9b8de007dc
Propagate ILogger
2 years ago
Martin Evans
669ae47ef7
- Split parameters into two interfaces
- params contains a list of loras, instead of just one
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
Martin Evans
93f24f8a51
Switched to properly typed `Encoding` property
2 years ago
Martin Evans
9fc17f3136
Fixed unit tests
2 years ago
Martin Evans
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2 years ago
Martin Evans
d0a7a8fcd6
- Cleaned up disposal in LLamaContext
- sealed some classes not intended to be extended
2 years ago
Martin Evans
20bdc2ec6f
- Apply LoRA in `LLamaWeights.LoadFromFile`
- Sanity checking that weights are not disposed when creating a context from them
- Further simplified `Utils.InitLLamaContextFromModelParams`
2 years ago
Martin Evans
e2fe08a9a2
Added a higher level `LLamaWeights` wrapper around `SafeLlamaModelHandle`
2 years ago