Rinne
4f44e3b198
refactor: init some refactorings for experiment.
1 year ago
Martin Evans
d87d654a34
Merge pull request #348 from martindevans/new_object_based_sampling_pipeline
Custom Sampling Pipelines
1 year ago
Martin Evans
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
1 year ago
Philipp Bauer
67e6d633fd
Rebuild ChatSession class
- Saves with serialized ChatHistory of session
- Only allows use of ChatHistory.Message (instead of raw text)
for easy post-processing with IHistoryTransform implementation
- Provides History Management methods
- Allows user to regenerate last assistant message
2 years ago
Martin Evans
8540c8d220
Merge pull request #312 from martindevans/removed_obsolete_modelparams_constructor
Removed Obsolete ModelParams Constructor
2 years ago
Martin Evans
c74f463d8e
Removed the obsolete constructor in `ModelParams`, it was marked obsolete 3 months ago.
2 years ago
Martin Evans
48c5039054
Improved test coverage. Discovered some issues:
FixedSizeQueue:
- Enqueue would always stop one short of filling the capacity
- Fill would only _replace_ existing items. It was only used in a place where there were not existing items! Removed the method entirely.
LLamaGrammarElement:
- Converted into a `record` struct, removed all of the (now unnecessary) equality stuff.
2 years ago
Martin Evans
89fef05362
This commit ( 5fe721bdbe) accidentally removed a load of stuff that it shouldn't. Fixed that.
Originally from these PRs:
- https://github.com/SciSharp/LLamaSharp/pull/263
- https://github.com/SciSharp/LLamaSharp/pull/259
2 years ago
Martin Evans
e3468d04f0
Merge pull request #277 from martindevans/feature/min_p
MinP Sampler
2 years ago
Martin Evans
d743516070
- Added support for the MinP sampler
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2 years ago
SignalRT
97006a214f
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2 years ago
Martin Evans
31244ae691
Merge branch 'master' into YaRN_scaling_parameters
2 years ago
SignalRT
5fe721bdbe
Revert "Merge branch 'pr/268' into RuntimeDetection"
This reverts commit 091b8d58b3502a99b3bfbec9db457c92cc736beb, reversing
changes made to 9b2ca9cf8e .
2 years ago
Martin Evans
db1bc741b0
Modified `ContextSize` in parameters to be nullable. A null value means autodetect from the model.
2 years ago
Udayshankar Ravikumar
4071c1f5fc
Updated preprocessor directives
2 years ago
Martin Evans
04ee64a6be
Exposed YaRN scaling parameters in IContextParams
2 years ago
Udayshankar Ravikumar
1dad1ff834
Enhance framework compatibility
2 years ago
Martin Evans
529b06b35b
- Fixed rope frequency/base to use the values in the model by default, instead of always overriding them by default!
2 years ago
Martin Evans
c786fb0ec8
Using `IReadOnlyList` instead of `IEnumerable` in `IInferenceParams`
2 years ago
Martin Evans
321d0b58c4
Merge pull request #202 from martindevans/multi_gpu
Multi GPU
2 years ago
Martin Evans
f6a472ae86
Setting the default seed to `0xFFFFFFFF` (no seed, randomised)
2 years ago
Martin Evans
f621ec67e8
Fixed serialization
2 years ago
Martin Evans
b4e7f64e76
Added System.Text.Json serialization for `TensorSplitsCollectionConverter`
2 years ago
Martin Evans
281e58f059
Fixed default value
2 years ago
Martin Evans
6a4cd506bd
Added a safe `TensorSplitsCollection` to the params which prevents incorrectly setting the `tensor_splits` collection
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
Martin Evans
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
- Replaced all binaries
2 years ago
Martin Evans
669ae47ef7
- Split parameters into two interfaces
- params contains a list of loras, instead of just one
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
redthing1
296ba607de
make InferenceParams a record so we can use with
2 years ago
Martin Evans
b1e9d8240d
Merge pull request #149 from martindevans/removed_unused_inference_params
Removed unused properties of `InferenceParams` & `ModelParams`
2 years ago
Martin Evans
b47977300a
Removed one more unused parameter
2 years ago
Martin Evans
a1b0349561
Removed `ModelAlias` property (unused)
2 years ago
sa_ddam213
09d8f434f2
Extract LLamaLogLevel, Remove Logger class
2 years ago
Martin Evans
77bd090150
Simplified `LLamaInteractExecutor` antiprompt matching by using new extension method
2 years ago
Martin Evans
d79a6556a1
Removed 3 unused properties of `InferenceParams`
2 years ago
Martin Evans
2056078aef
Initial changes required for GGUF support
2 years ago
Martin Evans
a911b77dec
Various minor changes, resolving about 100 ReSharper code quality warnings
2 years ago
Martin Evans
77aa5fa0d0
Added `JsonConverter` attribute, so System.Text.Json serialization is seamless
2 years ago
Martin Evans
ee772a2921
added `using` statement instead of full qualification
2 years ago
Martin Evans
93f24f8a51
Switched to properly typed `Encoding` property
2 years ago
Martin Evans
29df14cd9c
Converted ModelParams into a `record` class. This has several advantages:
- Equality, hashing etc all implemented automatically
- Default values are defined in just one place (the properties) instead of the constructor as well
- Added test to ensure that serialization works properly
2 years ago
Martin Evans
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
- Deleted `NativeInfo` (internal class, not used anywhere)
2 years ago
Erin Loy
8f0b52eb09
Re-renaming some arguments to allow for easy deserialization from appsettings.json.
2 years ago
Martin Evans
759ae26f36
Merge branch 'master' into grammar_basics
2 years ago
Martin Evans
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2 years ago
Erin Loy
592a80840b
renamed some arguments in ModelParams constructor so that classcan be serialized easily
2 years ago
Martin Evans
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
- Integrated grammar into sampling
- Added a test for the grammar sampling
2 years ago
Martin Evans
afe559ef1c
Added comments to `Logger` and fixed some nullability warnings
2 years ago
sa_ddam213
a67ea36dd9
Typo and formatting
2 years ago