Martin Evans
b0acecf080
Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix).
Conversations can be "forked", to create a copy of a conversation at a given point. This allows e.g. prompting a conversation with a system prefix just once and then forking it again and again for each individual conversation. Conversations can also be "rewound" to an earlier state.
Added two new examples, demonstrating forking and rewinding.
1 year ago
Martin Evans
92b9bbe779
Added methods to `SafeLLamaContextHandle` for KV cache manipulation
1 year ago
Martin Evans
96c26c25f5
Merge pull request #445 from martindevans/stateless_executor_llama_decode
Swapped `StatelessExecutor` to use `llama_decode`!
1 year ago
xbotter
90815ae7d8
bump sk & km
- bump semantic kernel to 1.1.0
- bump kernel memory to 0.26
1 year ago
Martin Evans
9fe878ae1f
- Fixed example
- Growing more than double, if necessary
1 year ago
Martin Evans
a2e29d393c
Swapped `StatelessExecutor` to use `llama_decode`!
- Added `logits_i` argument to `Context.ApplyPenalty`
- Added a new exception type for `llama_decode` return code
1 year ago
Martin Evans
5b6e82a594
Improved the BatchedDecoding demo:
- using less `NativeHandle`
- Using `StreamingTokenDecoder` instead of obsolete detokenize method
1 year ago
Martin Evans
99969e538e
- Removed some unused `eval` methods.
- Added a `DecodeAsync` overload which runs the work in a task
- Replaced some `NativeHandle` usage in `BatchedDecoding` with higher level equivalents.
- Made the `LLamaBatch` grow when token capacity is exceeded, removing the need to manage token capacity externally.
1 year ago
Martin Evans
36a9335588
Removed `LLamaBatchSafeHandle` (using unmanaged memory, created by llama.cpp) and replaced it with a fully managed `LLamaBatch`. Modified the `BatchedDecoding` example to use new managed batch.
1 year ago
Martin Evans
42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
1 year ago
Martin Evans
a408335c44
Fixed broken build on master (just removing a namespace that no longer exists)
1 year ago
dependabot[bot]
f02b0500b5
build(deps): bump Microsoft.KernelMemory.Core and Microsoft.KernelMemory.Abstractions
Bumps [Microsoft.KernelMemory.Core](https://github.com/microsoft/kernel-memory ) and [Microsoft.KernelMemory.Abstractions](https://github.com/microsoft/kernel-memory ). These dependencies needed to be updated together.
Updates `Microsoft.KernelMemory.Core` from 0.18.231209.1-preview to 0.24.231228.5
- [Release notes](https://github.com/microsoft/kernel-memory/releases )
- [Commits](https://github.com/microsoft/kernel-memory/compare/dotnet-0.18.231209.1-preview...0.24.231228.5 )
Updates `Microsoft.KernelMemory.Abstractions` from 0.18.231209.1-preview to 0.24.231228.5
- [Release notes](https://github.com/microsoft/kernel-memory/releases )
- [Commits](https://github.com/microsoft/kernel-memory/compare/dotnet-0.18.231209.1-preview...0.24.231228.5 )
---
updated-dependencies:
- dependency-name: Microsoft.KernelMemory.Core
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: Microsoft.KernelMemory.Abstractions
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
1 year ago
Martin Evans
f0d7468b22
Merge pull request #356 from xbotter/deps/sk-rc3
bump sk to 1.0.1 & km to 0.18
1 year ago
xbotter
40ac944fb5
Bump sk to 1.0.1
1 year ago
Martin Evans
b868b056f7
Added metadata overrides to `IModelParams`
1 year ago
xbotter
8766fb1b03
Merge branch 'deps/sk-rc3' of https://github.com/xbotter/LLamaSharp into deps/sk-rc3
1 year ago
xbotter
213b4be723
bump sk-1.0.0-rc4
1 year ago
xbotter
ce20b30e06
Merge branch 'SciSharp:master' into deps/sk-rc3
1 year ago
Martin Evans
bab6b65b61
Added a safe handle for LLamaKvCacheView
1 year ago
Rinne
fb75e06293
fix: output prefix of Chinese example.
1 year ago
Rinne
836f071cd0
fix: Chinese example.
1 year ago
xbotter
13a312b4ec
update sk to 1.0.0-rc3 & km to 0.18
1 year ago
Philipp Bauer
f669a4f5a7
Update the Chinese chat sample to use new ChatSession integration
1 year ago
Philipp Bauer
2cc01efdae
Merge branch 'SciSharp:master' into master
1 year ago
Martin Evans
4fc743c9ba
Merge branch 'master' into master
1 year ago
Philipp Bauer
422605d980
Re-add ChatSession examples
2 years ago
Philipp Bauer
73d1725954
Modified / updated ChatSession examples
2 years ago
Philipp Bauer
f1eac82ecc
Update target frameworks with .NET 8
2 years ago
xbotter
a2b26faa7a
🔧 Refactor chat completion implementation
- Refactored the chat completion implementation in `LLamaSharpChatCompletion.cs` to use `StatelessExecutor` instead of `InteractiveExecutor`.
- Updated the chat history prompt in `LLamaSharpChatCompletion.cs` to include a conversation between the assistant and the user.
- Modified the `HistoryTransform` class in `HistoryTransform.cs` to append the assistant role to the chat history prompt.
- Updated the constructor of `LLamaSharpChatCompletion` to accept optional parameters for `historyTransform` and `outputTransform`.
- Modified the `GetChatCompletionsAsync` and `GetChatCompletions` methods in `LLamaSharpChatCompletion.cs` to use the new `StatelessExecutor` and `outputTransform`.
- Updated the `ExtensionMethods.cs` file to include the assistant and system roles in the list of anti-prompts.
1 year ago
dependabot[bot]
36adc4af32
build(deps): bump Spectre.Console from 0.47.0 to 0.48.0
Bumps [Spectre.Console](https://github.com/spectreconsole/spectre.console ) from 0.47.0 to 0.48.0.
- [Release notes](https://github.com/spectreconsole/spectre.console/releases )
- [Commits](https://github.com/spectreconsole/spectre.console/compare/0.47.0...0.48.0 )
---
updated-dependencies:
- dependency-name: Spectre.Console
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
Rinne
605d1ddb3f
revert a change.
2 years ago
Rinne
934358a7b3
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into fix_chinese
2 years ago
Rinne
217c67b757
fix: chinese encoding error.
2 years ago
xbotter
d1e2a4750b
🔧 Update KernelMemory configuration
- Update LLamaSharpTextEmbeddingGeneration and LLamaSharpTextGeneration
- Add Microsoft.KernelMemory.Core package reference
- Update Microsoft.KernelMemory.Abstractions package reference
2 years ago
dependabot[bot]
20a0dc7ac6
build(deps): bump Microsoft.Extensions.Logging.Console
Bumps [Microsoft.Extensions.Logging.Console](https://github.com/dotnet/runtime ) from 7.0.0 to 8.0.0.
- [Release notes](https://github.com/dotnet/runtime/releases )
- [Commits](https://github.com/dotnet/runtime/compare/v7.0.0...v8.0.0 )
---
updated-dependencies:
- dependency-name: Microsoft.Extensions.Logging.Console
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
Rinne
c94aeabc4b
Merge pull request #307 from xbotter/sm-default-config
Add DefaultInferenceParams to Kernel Memory
2 years ago
xbotter
286904920b
update DefaultInferenceParams in WithLLamaSharpDefaults
2 years ago
xbotter
1056e13414
fix examples
2 years ago
xbotter
15db931a66
bump semantic kernel 1.0.0-beta8
2 years ago
Martin Evans
9905baa7c4
Merge pull request #300 from martindevans/minor_demos_cleanup
Minor cleanup on example code
2 years ago
Martin Evans
1dd4c6ab7a
Merge pull request #296 from xbotter/deps/sk_1.0.0-beta6
bump semantic kernel to 1.0.0-beta-6
2 years ago
Martin Evans
479779e908
Some minor cleanup on example code:
- Removed special case for exit
- Added a wait at the end of batched decoding
2 years ago
xbotter
6c31f69720
bump semantic kernel to 1.0.0-beta-6
2 years ago
xbotter
521e36903c
🔀 Remove unused code and update examples
- Renamed files and updated namespaces in Examples folder.
- Moved files from NewVersion folder to Examples folder.
- Removed TestRunner.cs file.
- Updated Runner.cs to include new examples.
- Update Program.cs to use the new Runner class instead of NewVersionTestRunner
- Update LLama.Examples namespace in Program.cs
- Update await NewVersionTestRunner.Run() in Program.cs to await Runner.Run()
2 years ago
Martin Evans
a9d1f6cb47
- Renamed `NativeLibraryConfig.Default` to `NativeLibraryConfig.Instance`. It's not default any more as soon as you call `WithX`!
- using `Lazy<T>` to initialize it automatically.
- Added in `AVX512` support for all dotnet versions (but not autodetected).
- Added in AVX version auto detection.
2 years ago
Yaohui Liu
d7675f7936
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into cuda_detection
2 years ago
Yaohui Liu
cb5fb210b1
feat: optimize apis for cuda feature detection.
2 years ago
SignalRT
fb95bbb4e0
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2 years ago
SignalRT
6de8d6219a
Change SemanticKernel version to beta1 on Examples
2 years ago
Rinne
14b5994c93
Merge branch 'master' into add_kernel_memory_pkg
2 years ago