SignalRT
43677c511c
Change interface to support multiple images and add the capabitlity to render the image in the console
1 year ago
SignalRT
e8732efadd
Example InteractiveExecutor
Add an Example and modifications to the interactive executor to enable Llava Models.
Just a preview / demo
1 year ago
eublefar
af796fc3e9
Change List types in executor state to arrays to enforce copy on get/set operations
1 year ago
eublefar
e05d5d4e14
Remove resetting state ops and make SessionState.ExecutorState and SessionState.ContextState no nullable
1 year ago
eublefar
35153a77dd
Chat session Get/Load in-memory state operations, reset state ops for stateful executors and context
1 year ago
Martin Evans
8ac1634233
Removed `llama_eval`. It is going to be completely removed in the next version of llama.cpp ( #553 )
1 year ago
Martin Evans
c7d0dc915a
Assorted small changes to clean up some code warnings
1 year ago
Martin Evans
b0acecf080
Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix).
Conversations can be "forked", to create a copy of a conversation at a given point. This allows e.g. prompting a conversation with a system prefix just once and then forking it again and again for each individual conversation. Conversations can also be "rewound" to an earlier state.
Added two new examples, demonstrating forking and rewinding.
1 year ago
Martin Evans
15a98b36d8
Updated everything to work with llama.cpp ce32060198
1 year ago
Martin Evans
a2e29d393c
Swapped `StatelessExecutor` to use `llama_decode`!
- Added `logits_i` argument to `Context.ApplyPenalty`
- Added a new exception type for `llama_decode` return code
1 year ago
Martin Evans
42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
1 year ago
Martin Evans
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
1 year ago
Martin Evans
48c5039054
Improved test coverage. Discovered some issues:
FixedSizeQueue:
- Enqueue would always stop one short of filling the capacity
- Fill would only _replace_ existing items. It was only used in a place where there were not existing items! Removed the method entirely.
LLamaGrammarElement:
- Converted into a `record` struct, removed all of the (now unnecessary) equality stuff.
2 years ago
Martin Evans
d743516070
- Added support for the MinP sampler
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2 years ago
Martin Evans
a024d2242e
It works!
had to update binary to `b1426`
2 years ago
Martin Evans
f1e5a8f995
- Passing the `ILogger` through to every call of `CreateContext`
- Passing `ILogger` into executors
2 years ago
sa_ddam213
4ec9aed47a
Revert LLamasSharp project changes
2 years ago
sa_ddam213
9b8de007dc
Propagate ILogger
2 years ago
Martin Evans
3f80190f85
Minimal changes required to remove non-async inference.
2 years ago
Martin Evans
a09aa86324
Merge pull request #153 from martindevans/fix_savestate_OpenOrCreate
Changed `OpenOrCreate` to `Create`
2 years ago
Martin Evans
f366aa3abe
Changed `OpenOrCreate` to `Create` to fix #151
2 years ago
Martin Evans
77bd090150
Simplified `LLamaInteractExecutor` antiprompt matching by using new extension method
2 years ago
Martin Evans
33035c82bf
- Removed `LLamaNewlineTokens` from `InteractiveExecutorState`. This is always set in the constructor from the context, so there's no point serializing it.
2 years ago
sa_ddam213
a5d742b72c
Fix Tokenize of new line, Remove space inserts
2 years ago
Martin Evans
31287b5e6e
Rewritten TokenToSpan/TokenToString to better fit the new way it's done in llama.cpp with a few different options:
- Just convert it to a `string`, nice and simple
- Write the bytes to a `Span<byte>` no allocations
- Write the chars to a `StringBuilder` potentially no allocations
2 years ago
Martin Evans
0c98ae1955
Passing ctx to `llama_token_nl(_ctx)`
2 years ago
Martin Evans
829f32b27d
- Added `Obsolete` attributes to the entire `OldVersion` namespace, so they can be removed in the future
- Minor changes to cleanup some of the compiler warnings
2 years ago
Martin Evans
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
- Deleted `NativeInfo` (internal class, not used anywhere)
2 years ago
Martin Evans
759ae26f36
Merge branch 'master' into grammar_basics
2 years ago
Martin Evans
4738c26299
- Reduced context size of test, to speed it up
- Removed some unnecessary `ToArray` calls
- Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from
2 years ago
Martin Evans
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
- Integrated grammar into sampling
- Added a test for the grammar sampling
2 years ago
Martin Evans
fda7e1c038
Fixed mirostat/mirostate
2 years ago
Martin Evans
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2 years ago
Martin Evans
270c6d55ef
Merge pull request #88 from martindevans/fix_serialization_nan
Fix serialization error due to NaN
2 years ago
Martin Evans
b5de3ee5aa
Fixed some final mentions of "mirostate" instead of "mirostat"
2 years ago
Martin Evans
0e5e00e300
Moved `TokenToString` from Utils into `SafeLLamaContextHandle` (thin wrappers around the same method in `SafeLlamaModelHandle`)
2 years ago
Martin Evans
cd3cf2b77d
- Moved tokenization from `Utils.Tokenize` into `SafeLLamaContextHandle.Tokenize`, one less thing in `Utils`.
- Also refactored it to return an `int[]` instead of an `IEnumerable<int>`, solving the "multiple enumeration" problems at the source!
2 years ago
sa_ddam213
bac9cba01a
InferenceParams abstractions
2 years ago
Martin Evans
c64507cb41
Correctly passing through mu value to mirostate instead of resetting it every time.
2 years ago
Yaohui Liu
6c400e64c2
docs: publiash documentation 0.4.
2 years ago
Yaohui Liu
2eb2d6df83
test: add 9 examples of the new version.
2 years ago
Marcel
762fd7c1ae
Fixed a typo in FixedSizeQueue
2 years ago
Yaohui Liu
a3b8186f20
feat: support save and load chat session.
2 years ago
Yaohui Liu
3bf74ec9b9
feat: add chat session for refactored code.
2 years ago
Yaohui Liu
908b79e855
feat: add stateless executor.
2 years ago
Yaohui Liu
e603a09137
fix: state loading and saving not working.
2 years ago
Yaohui Liu
5679e08718
feat: add ILLamaExecutor.InferAsync.
2 years ago
Yaohui Liu
264fb9a706
refactor: LLamaModel and LLamaExecutor.
2 years ago