Martin Evans
a8ba9f05b3
March Binary Update ( #565 )
* Updated binaries to llama.cpp `3ab8b3a92ede46df88bc5a2dfca3777de4a2b2b6` (build run: https://github.com/SciSharp/LLamaSharp/actions/runs/8118890586 )
* Added abort callback
* Added properties to get/set thread count on `LLamaContext`
* Fixed LLamaLogLevel numbering
1 year ago
Martin Evans
15a98b36d8
Updated everything to work with llama.cpp ce32060198
1 year ago
Martin Evans
199b4418c3
Fixed JSON serialization for ModelParams
1 year ago
Martin Evans
b868b056f7
Added metadata overrides to `IModelParams`
1 year ago
Martin Evans
439d14a061
Updated binaries:
- build run: https://github.com/SciSharp/LLamaSharp/actions/runs/7196891440
- commit: 9fb13f9584
1 year ago
xbotter
340bbbcf48
Move JSON converter for TensorSplitsCollection
1 year ago
Martin Evans
01c7f1b4da
Update LLama/Common/ModelParams.cs
1 year ago
Martin Evans
e6148c952e
Fixed encoding of `Encoding`
1 year ago
Martin Evans
c74f463d8e
Removed the obsolete constructor in `ModelParams`, it was marked obsolete 3 months ago.
2 years ago
Martin Evans
89fef05362
This commit ( 5fe721bdbe) accidentally removed a load of stuff that it shouldn't. Fixed that.
Originally from these PRs:
- https://github.com/SciSharp/LLamaSharp/pull/263
- https://github.com/SciSharp/LLamaSharp/pull/259
2 years ago
SignalRT
97006a214f
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2 years ago
Martin Evans
31244ae691
Merge branch 'master' into YaRN_scaling_parameters
2 years ago
SignalRT
5fe721bdbe
Revert "Merge branch 'pr/268' into RuntimeDetection"
This reverts commit 091b8d58b3502a99b3bfbec9db457c92cc736beb, reversing
changes made to 9b2ca9cf8e .
2 years ago
Martin Evans
db1bc741b0
Modified `ContextSize` in parameters to be nullable. A null value means autodetect from the model.
2 years ago
Martin Evans
04ee64a6be
Exposed YaRN scaling parameters in IContextParams
2 years ago
Martin Evans
529b06b35b
- Fixed rope frequency/base to use the values in the model by default, instead of always overriding them by default!
2 years ago
Martin Evans
321d0b58c4
Merge pull request #202 from martindevans/multi_gpu
Multi GPU
2 years ago
Martin Evans
f6a472ae86
Setting the default seed to `0xFFFFFFFF` (no seed, randomised)
2 years ago
Martin Evans
f621ec67e8
Fixed serialization
2 years ago
Martin Evans
b4e7f64e76
Added System.Text.Json serialization for `TensorSplitsCollectionConverter`
2 years ago
Martin Evans
281e58f059
Fixed default value
2 years ago
Martin Evans
6a4cd506bd
Added a safe `TensorSplitsCollection` to the params which prevents incorrectly setting the `tensor_splits` collection
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
Martin Evans
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
- Replaced all binaries
2 years ago
Martin Evans
669ae47ef7
- Split parameters into two interfaces
- params contains a list of loras, instead of just one
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
Martin Evans
b47977300a
Removed one more unused parameter
2 years ago
Martin Evans
a1b0349561
Removed `ModelAlias` property (unused)
2 years ago
Martin Evans
2056078aef
Initial changes required for GGUF support
2 years ago
Martin Evans
77aa5fa0d0
Added `JsonConverter` attribute, so System.Text.Json serialization is seamless
2 years ago
Martin Evans
ee772a2921
added `using` statement instead of full qualification
2 years ago
Martin Evans
93f24f8a51
Switched to properly typed `Encoding` property
2 years ago
Martin Evans
29df14cd9c
Converted ModelParams into a `record` class. This has several advantages:
- Equality, hashing etc all implemented automatically
- Default values are defined in just one place (the properties) instead of the constructor as well
- Added test to ensure that serialization works properly
2 years ago
Martin Evans
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
- Deleted `NativeInfo` (internal class, not used anywhere)
2 years ago
Erin Loy
8f0b52eb09
Re-renaming some arguments to allow for easy deserialization from appsettings.json.
2 years ago
Martin Evans
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2 years ago
Erin Loy
592a80840b
renamed some arguments in ModelParams constructor so that classcan be serialized easily
2 years ago
Martin Evans
685eb3b9c2
Replaced `nint` with `float[]?` in Model params, which is much more user friendly!
2 years ago
sa_ddam213
2a04e31b7d
ModelParams abstraction
2 years ago
SignalRT
348f2c7d72
Update llama.cpp binaries to 5f631c2 and align the context to that version
It solves the problem with netstandard2 (is it really netstandard2 a thing right now?)
Change context to solve problems.
5f631c2679
2 years ago
Yaohui Liu
1062fe1a7e
feat: upgrade the native libraries.
2 years ago
Yaohui Liu
3bf74ec9b9
feat: add chat session for refactored code.
2 years ago