Martin Evans
5cf481dc8e
Merge pull request #454 from martindevans/kv_cache_instance_methods
kv_cache_instance_methods
1 year ago
Martin Evans
92b9bbe779
Added methods to `SafeLLamaContextHandle` for KV cache manipulation
1 year ago
Martin Evans
8dfd07f67b
Merge pull request #453 from martindevans/fix_bad_merge_nseqmax
Fix Master Build Fail
1 year ago
Martin Evans
a690db5d3e
Fixed build error caused by extra unnecessary parameter
1 year ago
Martin Evans
96c26c25f5
Merge pull request #445 from martindevans/stateless_executor_llama_decode
Swapped `StatelessExecutor` to use `llama_decode`!
1 year ago
Martin Evans
1bc61472a8
Merge pull request #449 from SciSharp/dependabot/nuget/xunit-2.6.6
build(deps): bump xunit from 2.6.5 to 2.6.6
1 year ago
Martin Evans
d5b3650235
Merge pull request #451 from xbotter/deps/sk_1.1.0
bump sk & km
1 year ago
xbotter
90815ae7d8
bump sk & km
- bump semantic kernel to 1.1.0
- bump kernel memory to 0.26
1 year ago
dependabot[bot]
3d4c3c5509
build(deps): bump xunit from 2.6.5 to 2.6.6
Bumps [xunit](https://github.com/xunit/xunit ) from 2.6.5 to 2.6.6.
- [Commits](https://github.com/xunit/xunit/compare/2.6.5...2.6.6 )
---
updated-dependencies:
- dependency-name: xunit
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
1 year ago
Martin Evans
0074320a31
Merge pull request #447 from martindevans/grow_nseqmax_batch
LLamaBatch Grow n_seq_max automatically
1 year ago
Martin Evans
9fe878ae1f
- Fixed example
- Growing more than double, if necessary
1 year ago
Martin Evans
9ede1bedc2
Automatically growing batch n_seq_max when exceeded. This means no parameters need to be picked when the batch is created.
1 year ago
Martin Evans
a2e29d393c
Swapped `StatelessExecutor` to use `llama_decode`!
- Added `logits_i` argument to `Context.ApplyPenalty`
- Added a new exception type for `llama_decode` return code
1 year ago
Martin Evans
892e841da3
Merge pull request #444 from martindevans/batched_sampling_example_cleanup
Improved the BatchedDecoding demo
1 year ago
Martin Evans
5b6e82a594
Improved the BatchedDecoding demo:
- using less `NativeHandle`
- Using `StreamingTokenDecoder` instead of obsolete detokenize method
1 year ago
Martin Evans
250c20bd56
Merge pull request #443 from martindevans/llama_batch_self_grow
LLamaBatch Automatically Grow Capacity
1 year ago
Martin Evans
99969e538e
- Removed some unused `eval` methods.
- Added a `DecodeAsync` overload which runs the work in a task
- Replaced some `NativeHandle` usage in `BatchedDecoding` with higher level equivalents.
- Made the `LLamaBatch` grow when token capacity is exceeded, removing the need to manage token capacity externally.
1 year ago
Martin Evans
a0be27d32b
Merge pull request #442 from martindevans/managed_llama_batch
Managed `LLamaBatch`
1 year ago
Martin Evans
36a9335588
Removed `LLamaBatchSafeHandle` (using unmanaged memory, created by llama.cpp) and replaced it with a fully managed `LLamaBatch`. Modified the `BatchedDecoding` example to use new managed batch.
1 year ago
Martin Evans
4b11feddef
Merge pull request #436 from SciSharp/dependabot/nuget/Microsoft.AspNetCore.OpenApi-8.0.1
build(deps): bump Microsoft.AspNetCore.OpenApi from 8.0.0 to 8.0.1
1 year ago
Martin Evans
1cb9bcd55c
Merge pull request #440 from martindevans/additional_special_string_tokenizer_tests
Extra Tokenization Tests
1 year ago
Martin Evans
1472704e12
Added a test with examples of troublesome strings from 0.9.1
1 year ago
Martin Evans
73172bbaba
Merge pull request #438 from martindevans/cleanup_model_unnecessary_unsafe
Model Metadata Loading Cleanup
1 year ago
Martin Evans
ce1d302e7e
Moved some native methods into `SafeLlamaModelHandle`, these methods are all wrapped in safer accessors with no extra costs so there is no need to expose them.
1 year ago
Martin Evans
4ef618012e
Merge pull request #437 from martindevans/check_model_path_exists
Check Model Path Exists
1 year ago
Martin Evans
1e86755071
- Removed unnecessary `unsafe` block in model metadata loading
- Clarified comments on native metadata loading methods
1 year ago
Martin Evans
de2b20aae5
- Added a specific exception for failing to load model weights.
- Checking if model is readable
1 year ago
Martin Evans
096e0e75f8
Check that the model file actually exists immediately before loading it. Improve #395
1 year ago
dependabot[bot]
853fccd9d0
build(deps): bump Microsoft.AspNetCore.OpenApi from 8.0.0 to 8.0.1
Bumps [Microsoft.AspNetCore.OpenApi](https://github.com/dotnet/aspnetcore ) from 8.0.0 to 8.0.1.
- [Release notes](https://github.com/dotnet/aspnetcore/releases )
- [Changelog](https://github.com/dotnet/aspnetcore/blob/main/docs/ReleasePlanning.md )
- [Commits](https://github.com/dotnet/aspnetcore/compare/v8.0.0...v8.0.1 )
---
updated-dependencies:
- dependency-name: Microsoft.AspNetCore.OpenApi
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
1 year ago
Martin Evans
3c6af909dd
Merge pull request #434 from martindevans/stateless_eos_check
Added a check for EOS token in LLamaStatelessExecutor
1 year ago
Martin Evans
5b41c8e5d5
Merge pull request #433 from martindevans/tokenizer_fixes_newline
Tokenizer Fixes For Issue 430
1 year ago
Martin Evans
f160fbd6d1
Added a check for EOS token in LLamaStatelessExecutor
1 year ago
Martin Evans
2ea2048b78
- Added a test for tokenizing just a new line (reproduce issue https://github.com/SciSharp/LLamaSharp/issues/430 )
- Properly displaying `LLamaToken`
- Removed all tokenisation code in `SafeLLamaContextHandle` - just pass it all through to the `SafeLlamaModelHandle`
- Improved `SafeLlamaModelHandle` tokenisation:
- Renting an array, for one less allocation
- Not using `&tokens[0]` to take a pointer to an array, this is redundant and doesn't work on empty arrays
1 year ago
Martin Evans
ba477b83a0
Merge pull request #428 from martindevans/fix_streaming_text_decoder
StreamingTextDecoder Fix & Tests
1 year ago
Martin Evans
98635a0d5a
Fixed decoding of large tokens (over 16 bytes) in streaming text decoder
1 year ago
Martin Evans
54dffe7e03
Merge pull request #427 from SciSharp/Oceania2018-patch-1
Update README.md
1 year ago
Haiping
e5c8126c1c
Update README.md
1 year ago
Martin Evans
402a110a3a
Merge pull request #404 from martindevans/switched_to_LLamaToken_struct
LLamaToken Struct
1 year ago
Martin Evans
d9b4e1fde7
Merge pull request #424 from Kaotic3/GpuLayerCountChange
Gpu layer count change
1 year ago
Steven Kennedy
988f2fa302
Reverted Net8.0
1 year ago
Kaotic3
984632bfed
Merge branch 'SciSharp:master' into master
1 year ago
Steven Kennedy
cf2e9e35f8
Updating the GpuLayerCount to mirror the Python Port of Llama.cpp
1 year ago
Martin Evans
267523e22a
Merge pull request #420 from SciSharp/dependabot/nuget/xunit-2.6.5
build(deps): bump xunit from 2.6.4 to 2.6.5
1 year ago
dependabot[bot]
4a6f07943c
build(deps): bump xunit from 2.6.4 to 2.6.5
Bumps [xunit](https://github.com/xunit/xunit ) from 2.6.4 to 2.6.5.
- [Commits](https://github.com/xunit/xunit/compare/2.6.4...2.6.5 )
---
updated-dependencies:
- dependency-name: xunit
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
1 year ago
Martin Evans
b2334b888e
Merge pull request #418 from martindevans/grammar_test
Grammar Test (#394 repro)
1 year ago
Martin Evans
eebe4cb120
Added a new test (commented out for now) which reproduces the issue reported in #394
1 year ago
Martin Evans
b620e749a1
Merge pull request #415 from martindevans/readme_example_code_fix
Fixed example code in readme
1 year ago
Martin Evans
9573e2c9ba
Merge pull request #417 from martindevans/safe_handle_initialisation
Safer Handle Initialisation
1 year ago
Martin Evans
1e69e265b6
Moved some native methods to do with creating/destroying resources into their respective handles. There is **no** safe way to call most of these methods, everything must be done through through handles.
1 year ago
Martin Evans
c696cda44b
Merge pull request #416 from martindevans/IModelParams_better_exception
Improved exceptions in IModelParams for unknown KV override types.
1 year ago