Martin Evans
1e69e265b6
Moved some native methods to do with creating/destroying resources into their respective handles. There is **no** safe way to call most of these methods, everything must be done through through handles.
1 year ago
Martin Evans
4e5e994dda
- directly returning a SafeLlamaModelHandle, instead of an IntPtr which is wrapped in a handle.
- made `llama_backend_init` private. This is automatically called, there is no way it can correctly be used externally.
- made `llama_token_to_piece` safe (Span instead of pointer)
1 year ago
Martin Evans
bac3e43498
Fixed handling of empty spans
1 year ago
Martin Evans
c002642268
- Removed some `unsafe` where it wasn't necessary
- Wrapped some native functions which take (pointer, length) in function which take a `span` instead.
1 year ago
Martin Evans
f860f88c36
Code cleanup driven by R# suggestions:
- Made `NativeApi` into a `static class` (it's not intended to be instantiated)
- Moved `LLamaTokenType` enum out into a separate file
- Made `LLamaSeqId` and `LLamaPos` into `record struct`, convenient to have equality etc
1 year ago
Martin Evans
2cded1b296
Fixed alignment of value fields in `LLamaModelMetadataOverride`
1 year ago
Martin Evans
6be3f62321
Fixed loading of very large metadata values (over 1kb)
1 year ago
Martin Evans
fb606c2488
Fixed incorrect values
1 year ago
Martin Evans
47e4fcef2a
Fixed GetString on netstandard2
1 year ago
Martin Evans
2a1e1b6183
Removed unused imports
1 year ago
Martin Evans
a2bae178fa
Added a `Metadata` property to `LLamaWeights`
1 year ago
Martin Evans
1b13f7c717
Improved support for AVX512:
- Enabled more features in build process (VBMI and VNNI)
- Added runtime checking for this features
- Improved runtime checking to no longer require dotnet8.0
1 year ago
Martin Evans
c298ab828a
Merge pull request #368 from martindevans/context_set_seed
Context Set Seed
1 year ago
Martin Evans
a3177ab140
Merge pull request #369 from martindevans/rename_llama_sample_temperature
Renamed `llama_sample_temperature` to `llama_sample_temp`
1 year ago
Martin Evans
db7ecf5a43
Added a method to create a clone of a grammar instance
1 year ago
Martin Evans
ea523d2e2a
Renamed `llama_sample_temperature` to `llama_sample_temp`, Mirroring the same change made in llama.cpp
1 year ago
Martin Evans
2df3e7617e
Added a method to set the RNG seed on the context
1 year ago
Martin Evans
cedef5e45a
Added the `pure` field to `LLamaModelQuantizeParams` (it's been added to llama.cpp)
1 year ago
Martin Evans
b868b056f7
Added metadata overrides to `IModelParams`
1 year ago
Martin Evans
b22d8b7495
- Added `GroupDisposable` to dispose a collection of items all together
- Renamed `LLamaModelKvOverride` to `LLamaModelMetadataOverride`
1 year ago
Martin Evans
5ad2cd1d3c
Added a comment on the type itself
1 year ago
Martin Evans
b0270b5788
Added comments on GGMLType
1 year ago
Martin Evans
b3e576608b
fixed safe handle
1 year ago
Martin Evans
bab6b65b61
Added a safe handle for LLamaKvCacheView
1 year ago
Martin Evans
439d14a061
Updated binaries:
- build run: https://github.com/SciSharp/LLamaSharp/actions/runs/7196891440
- commit: 9fb13f9584
1 year ago
Martin Evans
835958398c
- Removed the object wrappers and configurable pipeline, they can be better written in code.
- Added BaseSamplingPipeline which provides a base impl of `ISamplingPipeline`
- Added `DefaultSamplingPipeline` which mimics normal llama.cpp sampling
1 year ago
Martin Evans
33358124db
Initial pass at a new sampling pipeline
1 year ago
Rinne
1f97ad874b
Merge pull request #333 from AsakusaRinne/master
feat: allow customized search path for native library loading.
2 years ago
Rinne
ffc347a3f3
resolve comments.
2 years ago
Rinne
b05c3154f4
feat: allow customized search path for native library loading.
2 years ago
Rinne
934358a7b3
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into fix_chinese
2 years ago
Rinne
217c67b757
fix: chinese encoding error.
2 years ago
Martin Evans
a3614f6747
Added `native/` back into path prefix
2 years ago
Martin Evans
77003d763e
Added new symbols from llama.h
2 years ago
Martin Evans
37466956c7
Added new binaries.
- Built by this run: https://github.com/SciSharp/LLamaSharp/actions/runs/6921572568
- commit: `e937066420b79a757bf80e9836eb12b88420a218`
- Rearranged paths
2 years ago
Martin Evans
48c5039054
Improved test coverage. Discovered some issues:
FixedSizeQueue:
- Enqueue would always stop one short of filling the capacity
- Fill would only _replace_ existing items. It was only used in a place where there were not existing items! Removed the method entirely.
LLamaGrammarElement:
- Converted into a `record` struct, removed all of the (now unnecessary) equality stuff.
2 years ago
Martin Evans
c517cc18a2
Merge pull request #304 from martindevans/obsolete_attribute_eval
Added Obsolete markings to all `Eval` overloads
2 years ago
Martin Evans
16ab33ba3c
Added Obsolete markings to all `Eval` overloads
2 years ago
Martin Evans
0e51badb38
Exposed `progress_callback` in `LLamaModelParams` (although not in higher level)
2 years ago
Martin Evans
1970023ef4
Merge pull request #292 from martindevans/dotnet8.0
dotnet8.0
2 years ago
Martin Evans
89fef05362
This commit ( 5fe721bdbe) accidentally removed a load of stuff that it shouldn't. Fixed that.
Originally from these PRs:
- https://github.com/SciSharp/LLamaSharp/pull/263
- https://github.com/SciSharp/LLamaSharp/pull/259
2 years ago
Martin Evans
e9f5dbba89
Processing AVX512 branch on all dotnet versions
2 years ago
Martin Evans
e850115b5f
Added dotnet8.0 as a build target
2 years ago
Martin Evans
b44e780b0f
Merge pull request #281 from martindevans/NativeLibraryConfig_improvements
CPU Feature Detection 2
2 years ago
Martin Evans
e3468d04f0
Merge pull request #277 from martindevans/feature/min_p
MinP Sampler
2 years ago
Martin Evans
a9d1f6cb47
- Renamed `NativeLibraryConfig.Default` to `NativeLibraryConfig.Instance`. It's not default any more as soon as you call `WithX`!
- using `Lazy<T>` to initialize it automatically.
- Added in `AVX512` support for all dotnet versions (but not autodetected).
- Added in AVX version auto detection.
2 years ago
Rinne
da6718c387
docs: adjust some descriptions.
2 years ago
Yaohui Liu
d7675f7936
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into cuda_detection
2 years ago
Martin Evans
d743516070
- Added support for the MinP sampler
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2 years ago
Yaohui Liu
cb5fb210b1
feat: optimize apis for cuda feature detection.
2 years ago