Martin Evans
ce1d302e7e
Moved some native methods into `SafeLlamaModelHandle`, these methods are all wrapped in safer accessors with no extra costs so there is no need to expose them.
1 year ago
Martin Evans
1e86755071
- Removed unnecessary `unsafe` block in model metadata loading
- Clarified comments on native metadata loading methods
1 year ago
Martin Evans
402a110a3a
Merge pull request #404 from martindevans/switched_to_LLamaToken_struct
LLamaToken Struct
1 year ago
Martin Evans
1e69e265b6
Moved some native methods to do with creating/destroying resources into their respective handles. There is **no** safe way to call most of these methods, everything must be done through through handles.
1 year ago
Martin Evans
42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
1 year ago
Martin Evans
4e5e994dda
- directly returning a SafeLlamaModelHandle, instead of an IntPtr which is wrapped in a handle.
- made `llama_backend_init` private. This is automatically called, there is no way it can correctly be used externally.
- made `llama_token_to_piece` safe (Span instead of pointer)
1 year ago
Martin Evans
bac3e43498
Fixed handling of empty spans
1 year ago
Martin Evans
c002642268
- Removed some `unsafe` where it wasn't necessary
- Wrapped some native functions which take (pointer, length) in function which take a `span` instead.
1 year ago
Martin Evans
f860f88c36
Code cleanup driven by R# suggestions:
- Made `NativeApi` into a `static class` (it's not intended to be instantiated)
- Moved `LLamaTokenType` enum out into a separate file
- Made `LLamaSeqId` and `LLamaPos` into `record struct`, convenient to have equality etc
1 year ago
Martin Evans
db7ecf5a43
Added a method to create a clone of a grammar instance
1 year ago
Rinne
934358a7b3
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into fix_chinese
2 years ago
Rinne
217c67b757
fix: chinese encoding error.
2 years ago
Martin Evans
77003d763e
Added new symbols from llama.h
2 years ago
Martin Evans
37466956c7
Added new binaries.
- Built by this run: https://github.com/SciSharp/LLamaSharp/actions/runs/6921572568
- commit: `e937066420b79a757bf80e9836eb12b88420a218`
- Rearranged paths
2 years ago
Yaohui Liu
cb5fb210b1
feat: optimize apis for cuda feature detection.
2 years ago
Yaohui Liu
bbbfbd20b5
fix: cannot load library under some conditions.
2 years ago
Yaohui Liu
d03e1dbe30
feat: support cuda feature detection.
2 years ago
SignalRT
5fe721bdbe
Revert "Merge branch 'pr/268' into RuntimeDetection"
This reverts commit 091b8d58b3502a99b3bfbec9db457c92cc736beb, reversing
changes made to 9b2ca9cf8e .
2 years ago
SignalRT
200011e186
Revert "Merge feat: add detection template for cuda and avx. #268"
This reverts commit b4b3ea9d99 .
2 years ago
SignalRT
b4b3ea9d99
Merge feat: add detection template for cuda and avx. #268
Just merge cuda and avx detection and change layout.
2 years ago
Yaohui Liu
b893c6f609
feat: add detection template for cuda and avx.
2 years ago
Martin Evans
c7fdb9712c
Added binaries, built from ` 6961c4bd0b`
2 years ago
Martin Evans
a024d2242e
It works!
had to update binary to `b1426`
2 years ago
Martin Evans
8cd81251b4
initial setup
2 years ago
Martin Evans
15db194c17
Added multi GPU support
2 years ago
Martin Evans
e89ca5cc17
Fixed a few minor warnings
2 years ago
Martin Evans
1f8c94e386
Added in the `special` parameter to the tokenizer (introduced in https://github.com/ggerganov/llama.cpp/pull/3538 )
2 years ago
Martin Evans
0d40338692
Fixed out-of-context handling in stateless executor
2 years ago
Martin Evans
9e958e896b
safe handle for batch
2 years ago
Martin Evans
ce1fc51163
Added some more native methods
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
Haiping
10678a83d6
Merge pull request #65 from martindevans/alternative_dependency_loading
CPU Feature Detection
2 years ago
sa_ddam213
09d8f434f2
Extract LLamaLogLevel, Remove Logger class
2 years ago
Martin Evans
8f58a40fb9
Added Linux dependency loading
2 years ago
Martin Evans
dd4957471f
Changed paths to match what the GitHub build action produces
2 years ago
Martin Evans
756a1ad0ba
Added a new way to load dependencies, performing CPU feature detection
2 years ago
Martin Evans
bcf06e2652
Added some comments on various native methods
2 years ago
Martin Evans
2022b82947
Added binaries generated by this action: https://github.com/SciSharp/LLamaSharp/actions/runs/6002797872/job/16279896150
Based on this version: 6b73ef1201
2 years ago
Martin Evans
0c98ae1955
Passing ctx to `llama_token_nl(_ctx)`
2 years ago
Martin Evans
6ffa28f964
Removed `LLAMA_MAX_DEVICES` (not used)
2 years ago
Martin Evans
2056078aef
Initial changes required for GGUF support
2 years ago
Martin Evans
829f32b27d
- Added `Obsolete` attributes to the entire `OldVersion` namespace, so they can be removed in the future
- Minor changes to cleanup some of the compiler warnings
2 years ago
Martin Evans
d7f971fc22
Improved `NativeApi` file a bit:
- Added some more comments
- Modified `llama_tokenize` to not allocate
- Modified `llama_tokenize_native` to take a pointer instead of an array, allowing use with no allocations
- Removed GgmlInitParams (not used)
2 years ago
sa_ddam213
726987b761
Add native logging output
2 years ago
Martin Evans
2b2d3af26b
Moved `Eval` out of `Utils` and into `SafeLLamaContextHandle`
2 years ago
Martin Evans
2d811b2603
- Moved `GetLogits` into `SafeLLamaContextHandle`
- Added disposal check into `SafeLLamaContextHandle`
2 years ago
Martin Evans
cd3cf2b77d
- Moved tokenization from `Utils.Tokenize` into `SafeLLamaContextHandle.Tokenize`, one less thing in `Utils`.
- Also refactored it to return an `int[]` instead of an `IEnumerable<int>`, solving the "multiple enumeration" problems at the source!
2 years ago
Yaohui Liu
bb46a990d0
fix: add bug info for native api.
2 years ago
Martin Evans
afb9d24f3a
Added model `Tokenize` method
2 years ago
Martin Evans
369c915afe
Added TokenToString conversion on model handle
2 years ago