Rinne
4f44e3b198
refactor: init some refactorings for experiment.
1 year ago
Martin Evans
d87d654a34
Merge pull request #348 from martindevans/new_object_based_sampling_pipeline
Custom Sampling Pipelines
1 year ago
Martin Evans
835958398c
- Removed the object wrappers and configurable pipeline, they can be better written in code.
- Added BaseSamplingPipeline which provides a base impl of `ISamplingPipeline`
- Added `DefaultSamplingPipeline` which mimics normal llama.cpp sampling
1 year ago
Martin Evans
3afc007499
- Added "protected" logits, instead of the awkward save/load mechanism
- Added an example usage to one of the tests
1 year ago
Martin Evans
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
1 year ago
Martin Evans
4fc743c9ba
Merge branch 'master' into master
1 year ago
Philipp Bauer
67e6d633fd
Rebuild ChatSession class
- Saves with serialized ChatHistory of session
- Only allows use of ChatHistory.Message (instead of raw text)
for easy post-processing with IHistoryTransform implementation
- Provides History Management methods
- Allows user to regenerate last assistant message
2 years ago
Philipp Bauer
cb480f04af
Prevent compilation errors due to duplicated assembly info
2 years ago
Martin Evans
33358124db
Initial pass at a new sampling pipeline
1 year ago
Rinne
1f97ad874b
Merge pull request #333 from AsakusaRinne/master
feat: allow customized search path for native library loading.
2 years ago
Rinne
ffc347a3f3
resolve comments.
2 years ago
Rinne
b05c3154f4
feat: allow customized search path for native library loading.
2 years ago
Rinne
934358a7b3
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into fix_chinese
2 years ago
Rinne
217c67b757
fix: chinese encoding error.
2 years ago
Philipp Bauer
debc8285f8
Merge branch 'SciSharp:master' into master
2 years ago
Martin Evans
597188c236
Merge pull request #316 from martindevans/update_binaries_nov
November Binary Update
2 years ago
dependabot[bot]
40729c898f
build(deps): bump Microsoft.Extensions.Logging.Abstractions
Bumps [Microsoft.Extensions.Logging.Abstractions](https://github.com/dotnet/runtime ) from 7.0.1 to 8.0.0.
- [Release notes](https://github.com/dotnet/runtime/releases )
- [Commits](https://github.com/dotnet/runtime/compare/v7.0.1...v8.0.0 )
---
updated-dependencies:
- dependency-name: Microsoft.Extensions.Logging.Abstractions
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
dependabot[bot]
93790582e8
build(deps): bump System.Text.Json from 7.0.3 to 8.0.0
Bumps [System.Text.Json](https://github.com/dotnet/runtime ) from 7.0.3 to 8.0.0.
- [Release notes](https://github.com/dotnet/runtime/releases )
- [Commits](https://github.com/dotnet/runtime/compare/v7.0.3...v8.0.0 )
---
updated-dependencies:
- dependency-name: System.Text.Json
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
Martin Evans
7486e6388a
Updated nuspec files
2 years ago
Martin Evans
a3614f6747
Added `native/` back into path prefix
2 years ago
Martin Evans
77003d763e
Added new symbols from llama.h
2 years ago
Martin Evans
37466956c7
Added new binaries.
- Built by this run: https://github.com/SciSharp/LLamaSharp/actions/runs/6921572568
- commit: `e937066420b79a757bf80e9836eb12b88420a218`
- Rearranged paths
2 years ago
Martin Evans
d11702c4ea
Merge pull request #310 from martindevans/better_tensor_splits_specification
Better TensorSplitsCollection Initialisation
2 years ago
Martin Evans
8540c8d220
Merge pull request #312 from martindevans/removed_obsolete_modelparams_constructor
Removed Obsolete ModelParams Constructor
2 years ago
Martin Evans
c74f463d8e
Removed the obsolete constructor in `ModelParams`, it was marked obsolete 3 months ago.
2 years ago
Martin Evans
48c5039054
Improved test coverage. Discovered some issues:
FixedSizeQueue:
- Enqueue would always stop one short of filling the capacity
- Fill would only _replace_ existing items. It was only used in a place where there were not existing items! Removed the method entirely.
LLamaGrammarElement:
- Converted into a `record` struct, removed all of the (now unnecessary) equality stuff.
2 years ago
Martin Evans
e47431ed80
Modified `TensorSplitsCollection` so it accepts any number of splits, as long as it doesn't exceed the number of supported devices
2 years ago
Martin Evans
c517cc18a2
Merge pull request #304 from martindevans/obsolete_attribute_eval
Added Obsolete markings to all `Eval` overloads
2 years ago
Martin Evans
16ab33ba3c
Added Obsolete markings to all `Eval` overloads
2 years ago
Martin Evans
0e51badb38
Exposed `progress_callback` in `LLamaModelParams` (although not in higher level)
2 years ago
Martin Evans
1970023ef4
Merge pull request #292 from martindevans/dotnet8.0
dotnet8.0
2 years ago
Philipp Bauer
75932afc62
Remove debug output
2 years ago
Philipp Bauer
629430a087
Correctly format followup messages in turn-based (chat) inference
2 years ago
Rinne
36c2e64b36
Merge pull request #293 from martindevans/fixed_base_executor
StreamingTextDecoder in LLamaExecutorBase
2 years ago
Martin Evans
89fef05362
This commit ( 5fe721bdbe) accidentally removed a load of stuff that it shouldn't. Fixed that.
Originally from these PRs:
- https://github.com/SciSharp/LLamaSharp/pull/263
- https://github.com/SciSharp/LLamaSharp/pull/259
2 years ago
Martin Evans
3ce0f7d003
Using `StreamingTextDecoder` in `LLama/LLamaExecutorBase.cs`. This should fix weird text decoding issues with multi token characters.
2 years ago
Martin Evans
e9f5dbba89
Processing AVX512 branch on all dotnet versions
2 years ago
Martin Evans
e850115b5f
Added dotnet8.0 as a build target
2 years ago
sa_ddam213
153e61a81f
Make AntipromptProcessor public
2 years ago
Martin Evans
b44e780b0f
Merge pull request #281 from martindevans/NativeLibraryConfig_improvements
CPU Feature Detection 2
2 years ago
Martin Evans
e3468d04f0
Merge pull request #277 from martindevans/feature/min_p
MinP Sampler
2 years ago
Martin Evans
a9d1f6cb47
- Renamed `NativeLibraryConfig.Default` to `NativeLibraryConfig.Instance`. It's not default any more as soon as you call `WithX`!
- using `Lazy<T>` to initialize it automatically.
- Added in `AVX512` support for all dotnet versions (but not autodetected).
- Added in AVX version auto detection.
2 years ago
Rinne
da6718c387
docs: adjust some descriptions.
2 years ago
Yaohui Liu
d7675f7936
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into cuda_detection
2 years ago
Martin Evans
d743516070
- Added support for the MinP sampler
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2 years ago
Yaohui Liu
4d2c5f1003
build: change nuget configuration for cuda detection.
2 years ago
Yaohui Liu
cb5fb210b1
feat: optimize apis for cuda feature detection.
2 years ago
SignalRT
97006a214f
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2 years ago
Yaohui Liu
bbbfbd20b5
fix: cannot load library under some conditions.
2 years ago
Martin Evans
31244ae691
Merge branch 'master' into YaRN_scaling_parameters
2 years ago