Martin Evans
a8ba9f05b3
March Binary Update ( #565 )
* Updated binaries to llama.cpp `3ab8b3a92ede46df88bc5a2dfca3777de4a2b2b6` (build run: https://github.com/SciSharp/LLamaSharp/actions/runs/8118890586 )
* Added abort callback
* Added properties to get/set thread count on `LLamaContext`
* Fixed LLamaLogLevel numbering
1 year ago
Martin Evans
15a98b36d8
Updated everything to work with llama.cpp ce32060198
1 year ago
Martin Evans
42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
1 year ago
Martin Evans
f860f88c36
Code cleanup driven by R# suggestions:
- Made `NativeApi` into a `static class` (it's not intended to be instantiated)
- Moved `LLamaTokenType` enum out into a separate file
- Made `LLamaSeqId` and `LLamaPos` into `record struct`, convenient to have equality etc
1 year ago
Martin Evans
199b4418c3
Fixed JSON serialization for ModelParams
1 year ago
Martin Evans
b868b056f7
Added metadata overrides to `IModelParams`
1 year ago
Martin Evans
439d14a061
Updated binaries:
- build run: https://github.com/SciSharp/LLamaSharp/actions/runs/7196891440
- commit: 9fb13f9584
1 year ago
xbotter
340bbbcf48
Move JSON converter for TensorSplitsCollection
1 year ago
Martin Evans
01c7f1b4da
Update LLama/Common/ModelParams.cs
1 year ago
Martin Evans
e6148c952e
Fixed encoding of `Encoding`
1 year ago
Martin Evans
d87d654a34
Merge pull request #348 from martindevans/new_object_based_sampling_pipeline
Custom Sampling Pipelines
1 year ago
Martin Evans
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
1 year ago
Philipp Bauer
67e6d633fd
Rebuild ChatSession class
- Saves with serialized ChatHistory of session
- Only allows use of ChatHistory.Message (instead of raw text)
for easy post-processing with IHistoryTransform implementation
- Provides History Management methods
- Allows user to regenerate last assistant message
2 years ago
Martin Evans
8540c8d220
Merge pull request #312 from martindevans/removed_obsolete_modelparams_constructor
Removed Obsolete ModelParams Constructor
2 years ago
Martin Evans
c74f463d8e
Removed the obsolete constructor in `ModelParams`, it was marked obsolete 3 months ago.
2 years ago
Martin Evans
48c5039054
Improved test coverage. Discovered some issues:
FixedSizeQueue:
- Enqueue would always stop one short of filling the capacity
- Fill would only _replace_ existing items. It was only used in a place where there were not existing items! Removed the method entirely.
LLamaGrammarElement:
- Converted into a `record` struct, removed all of the (now unnecessary) equality stuff.
2 years ago
Martin Evans
89fef05362
This commit ( 5fe721bdbe) accidentally removed a load of stuff that it shouldn't. Fixed that.
Originally from these PRs:
- https://github.com/SciSharp/LLamaSharp/pull/263
- https://github.com/SciSharp/LLamaSharp/pull/259
2 years ago
Martin Evans
e3468d04f0
Merge pull request #277 from martindevans/feature/min_p
MinP Sampler
2 years ago
Martin Evans
d743516070
- Added support for the MinP sampler
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2 years ago
SignalRT
97006a214f
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2 years ago
Martin Evans
31244ae691
Merge branch 'master' into YaRN_scaling_parameters
2 years ago
SignalRT
5fe721bdbe
Revert "Merge branch 'pr/268' into RuntimeDetection"
This reverts commit 091b8d58b3502a99b3bfbec9db457c92cc736beb, reversing
changes made to 9b2ca9cf8e .
2 years ago
Martin Evans
db1bc741b0
Modified `ContextSize` in parameters to be nullable. A null value means autodetect from the model.
2 years ago
Udayshankar Ravikumar
4071c1f5fc
Updated preprocessor directives
2 years ago
Martin Evans
04ee64a6be
Exposed YaRN scaling parameters in IContextParams
2 years ago
Udayshankar Ravikumar
1dad1ff834
Enhance framework compatibility
2 years ago
Martin Evans
529b06b35b
- Fixed rope frequency/base to use the values in the model by default, instead of always overriding them by default!
2 years ago
Martin Evans
c786fb0ec8
Using `IReadOnlyList` instead of `IEnumerable` in `IInferenceParams`
2 years ago
Martin Evans
321d0b58c4
Merge pull request #202 from martindevans/multi_gpu
Multi GPU
2 years ago
Martin Evans
f6a472ae86
Setting the default seed to `0xFFFFFFFF` (no seed, randomised)
2 years ago
Martin Evans
f621ec67e8
Fixed serialization
2 years ago
Martin Evans
b4e7f64e76
Added System.Text.Json serialization for `TensorSplitsCollectionConverter`
2 years ago
Martin Evans
281e58f059
Fixed default value
2 years ago
Martin Evans
6a4cd506bd
Added a safe `TensorSplitsCollection` to the params which prevents incorrectly setting the `tensor_splits` collection
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
Martin Evans
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
- Replaced all binaries
2 years ago
Martin Evans
669ae47ef7
- Split parameters into two interfaces
- params contains a list of loras, instead of just one
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
redthing1
296ba607de
make InferenceParams a record so we can use with
2 years ago
Martin Evans
b1e9d8240d
Merge pull request #149 from martindevans/removed_unused_inference_params
Removed unused properties of `InferenceParams` & `ModelParams`
2 years ago
Martin Evans
b47977300a
Removed one more unused parameter
2 years ago
Martin Evans
a1b0349561
Removed `ModelAlias` property (unused)
2 years ago
sa_ddam213
09d8f434f2
Extract LLamaLogLevel, Remove Logger class
2 years ago
Martin Evans
77bd090150
Simplified `LLamaInteractExecutor` antiprompt matching by using new extension method
2 years ago
Martin Evans
d79a6556a1
Removed 3 unused properties of `InferenceParams`
2 years ago
Martin Evans
2056078aef
Initial changes required for GGUF support
2 years ago
Martin Evans
a911b77dec
Various minor changes, resolving about 100 ReSharper code quality warnings
2 years ago
Martin Evans
77aa5fa0d0
Added `JsonConverter` attribute, so System.Text.Json serialization is seamless
2 years ago
Martin Evans
ee772a2921
added `using` statement instead of full qualification
2 years ago
Martin Evans
93f24f8a51
Switched to properly typed `Encoding` property
2 years ago