Martin Evans
36735f7908
Fixed spelling of "mirostat" instead of "mirostate"
2 years ago
Martin Evans
f3fa73de2b
Implemented a new `LlamaModel.State` handle which internally stores the state as natively allocated memory. This allows it to exceed the 2GB limit on C# arrays.
2 years ago
Martin Evans
4d72420a04
Replaced `SaveState` and `LoadState` implementations. These new implementations map the file into memory and then pass the pointer directly into the native API. This improves things in two ways:
- A C# array cannot exceed 2,147,483,591 bytes. In my own use of LlamaSharp I encountered this limit.
- This saves an extra copy of the entire state data into a C# `byte[]`, so it should be faster.
This does _not_ fix some other places where `GetStateData` is used. I'll look at those in a separate PR.
2 years ago
Yaohui Liu
6c400e64c2
docs: publiash documentation 0.4.
2 years ago
Yaohui Liu
b20b6f209e
docs: add some xml comments.
2 years ago
Yaohui Liu
bdbd6aa824
feat: add transforms for chat session.
2 years ago
Yaohui Liu
b567399b65
refactor: allow customized logger.
2 years ago
Yaohui Liu
3bf74ec9b9
feat: add chat session for refactored code.
2 years ago
Yaohui Liu
908b79e855
feat: add stateless executor.
2 years ago
Yaohui Liu
e603a09137
fix: state loading and saving not working.
2 years ago
Yaohui Liu
5679e08718
feat: add ILLamaExecutor.InferAsync.
2 years ago
Yaohui Liu
264fb9a706
refactor: LLamaModel and LLamaExecutor.
2 years ago
Yaohui Liu
3a62f087fe
fix: encoding error when using other languages.
2 years ago
Yaohui Liu
e77afa76d0
feat: change default param of n_gpu_layers to 20.
2 years ago
Yaohui Liu
513d566361
refactor: remove dependency for third-party logger.
2 years ago
Yaohui Liu
18c2ff2395
refactor: instruct mode and examples.
2 years ago
Yaohui Liu
421e3f32c7
feat: add tokenize and detokenize apis to LLamaModel.
2 years ago
Yaohui Liu
e926b0690f
docs: add comments to LLamaModel methods.
2 years ago
Yaohui Liu
4e1b6cf4e9
fix: optimize loading and saving state.
2 years ago
Yaohui Liu
19979f664a
feat: support loading and saving state.
2 years ago
Yaohui Liu
2490cf17f4
build: update to v0.2.3.
2 years ago
Yaohui Liu
00d91cf99e
refactor: some parts of code of LLamaModel.
2 years ago
Yaohui Liu
afedd3c949
fix: errors when input is not English or too long.
2 years ago
Yaohui Liu
ea5f9d38ac
fix: always add bos when inference.
2 years ago
Yaohui Liu
4314f64b9c
feat: add check for backend package.
2 years ago
Yaohui Liu
bcd4c5605b
feat: add n_gpu_layers and prompt_cache_all params.
2 years ago
Yaohui Liu
9c0f3aedba
refactor: change some file names.
2 years ago
Yaohui Liu
aa2b064d1d
fix: add IDisposable to model classes.
2 years ago
Yaohui Liu
d76619c01b
docs: add more comments to obselete class LLamaModelV1.
2 years ago
Yaohui Liu
fce10f3c4f
feat: add ChatSession.
2 years ago
Yaohui Liu
d6a7997e46
feat: add gpt model.
2 years ago
Yaohui Liu
5a79edeb51
feat: add the framework and basic usages.
2 years ago