scisharp
/
LLamaSharp
Not watched
1
0
Fork 0
Code
Releases 17 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
1198 Commits
8 Branches
333 MB
43 Download
Tree: 96bf214427
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '96bf214427'
${ noResults }
LLamaSharp/LLama.Examples/Program.cs

Program.cs 1.3 kB
Raw Normal View History

LLama.Examples: improve model path prompt (#526) * LLama.Examples: RepoUtils.cs → ConsoleLogger.cs * LLama.Examples: Examples/Runner.cs → ExampleRunner.cs * LLama.Examples: delete unused console logger * LLama.Examples: improve splash screen appearance the llama_empty_call() no longer shows configuration information on startup, but it will display it automatically the first time a model is engaged * LLama.Examples: Runner → ExampleRunner * LLama.Examples: improve model path prompt The last used model is stored in a config file and is re-used when a blank path is provided * LLama.Examples: NativeApi.llama_empty_call() at startup * LLama.Examples: reduce console noise when saving model path
1 year ago
Used `AnsiConsole` in a few more places: (#534) - UserSettings, simplifying the validation/re-ask loop down to one call - Program, adding colour to figlet title - Batched examples, showing default prompt - ExampleRunner, resetting state after running an example
1 year ago
docs: add examples for running LLamaSharp.
2 years ago
Used `AnsiConsole` in a few more places: (#534) - UserSettings, simplifying the validation/re-ask loop down to one call - Program, adding colour to figlet title - Batched examples, showing default prompt - ExampleRunner, resetting state after running an example
1 year ago
​
​
LLama.Examples: improve model path prompt (#526) * LLama.Examples: RepoUtils.cs → ConsoleLogger.cs * LLama.Examples: Examples/Runner.cs → ExampleRunner.cs * LLama.Examples: delete unused console logger * LLama.Examples: improve splash screen appearance the llama_empty_call() no longer shows configuration information on startup, but it will display it automatically the first time a model is engaged * LLama.Examples: Runner → ExampleRunner * LLama.Examples: improve model path prompt The last used model is stored in a config file and is re-used when a blank path is provided * LLama.Examples: NativeApi.llama_empty_call() at startup * LLama.Examples: reduce console noise when saving model path
1 year ago
​
​
Used `AnsiConsole` in a few more places: (#534) - UserSettings, simplifying the validation/re-ask loop down to one call - Program, adding colour to figlet title - Batched examples, showing default prompt - ExampleRunner, resetting state after running an example
1 year ago
LLama.Examples: improve model path prompt (#526) * LLama.Examples: RepoUtils.cs → ConsoleLogger.cs * LLama.Examples: Examples/Runner.cs → ExampleRunner.cs * LLama.Examples: delete unused console logger * LLama.Examples: improve splash screen appearance the llama_empty_call() no longer shows configuration information on startup, but it will display it automatically the first time a model is engaged * LLama.Examples: Runner → ExampleRunner * LLama.Examples: improve model path prompt The last used model is stored in a config file and is re-used when a blank path is provided * LLama.Examples: NativeApi.llama_empty_call() at startup * LLama.Examples: reduce console noise when saving model path
1 year ago
​
​
Used `AnsiConsole` in a few more places: (#534) - UserSettings, simplifying the validation/re-ask loop down to one call - Program, adding colour to figlet title - Batched examples, showing default prompt - ExampleRunner, resetting state after running an example
1 year ago
LLama.Examples: improve model path prompt (#526) * LLama.Examples: RepoUtils.cs → ConsoleLogger.cs * LLama.Examples: Examples/Runner.cs → ExampleRunner.cs * LLama.Examples: delete unused console logger * LLama.Examples: improve splash screen appearance the llama_empty_call() no longer shows configuration information on startup, but it will display it automatically the first time a model is engaged * LLama.Examples: Runner → ExampleRunner * LLama.Examples: improve model path prompt The last used model is stored in a config file and is re-used when a blank path is provided * LLama.Examples: NativeApi.llama_empty_call() at startup * LLama.Examples: reduce console noise when saving model path
1 year ago
docs: add examples for running LLamaSharp.
2 years ago
LLama.Examples: improve model path prompt (#526) * LLama.Examples: RepoUtils.cs → ConsoleLogger.cs * LLama.Examples: Examples/Runner.cs → ExampleRunner.cs * LLama.Examples: delete unused console logger * LLama.Examples: improve splash screen appearance the llama_empty_call() no longer shows configuration information on startup, but it will display it automatically the first time a model is engaged * LLama.Examples: Runner → ExampleRunner * LLama.Examples: improve model path prompt The last used model is stored in a config file and is re-used when a blank path is provided * LLama.Examples: NativeApi.llama_empty_call() at startup * LLama.Examples: reduce console noise when saving model path
1 year ago
refactor: instruct mode and examples.
2 years ago
`SetDllImportResolver` based loading (#603) - Modified library loading to be based on `SetDllImportResolver`. This replaces the built in loading system and ensures there can't be two libraries loaded at once. - llava and llama are loaded separately, as needed. - All the previous loading logic is still used, within the `SetDllImportResolver` - Split out CUDA, AVX and MacOS paths to separate helper methods. - `Description` now specifies if it is for `llama` or `llava`
1 year ago
Added a safe handle for LLamaKvCacheView
1 year ago
​
​
`SetDllImportResolver` based loading (#603) - Modified library loading to be based on `SetDllImportResolver`. This replaces the built in loading system and ensures there can't be two libraries loaded at once. - llava and llama are loaded separately, as needed. - All the previous loading logic is still used, within the `SetDllImportResolver` - Split out CUDA, AVX and MacOS paths to separate helper methods. - `Description` now specifies if it is for `llama` or `llava`
1 year ago
refactor: instruct mode and examples.
2 years ago
`SetDllImportResolver` based loading (#603) - Modified library loading to be based on `SetDllImportResolver`. This replaces the built in loading system and ensures there can't be two libraries loaded at once. - llava and llama are loaded separately, as needed. - All the previous loading logic is still used, within the `SetDllImportResolver` - Split out CUDA, AVX and MacOS paths to separate helper methods. - `Description` now specifies if it is for `llama` or `llava`
1 year ago
Initial changes to match the llama.cpp changes
2 years ago
test: update the examples after refactoring.
2 years ago
`SetDllImportResolver` based loading (#603) - Modified library loading to be based on `SetDllImportResolver`. This replaces the built in loading system and ensures there can't be two libraries loaded at once. - llava and llama are loaded separately, as needed. - All the previous loading logic is still used, within the `SetDllImportResolver` - Split out CUDA, AVX and MacOS paths to separate helper methods. - `Description` now specifies if it is for `llama` or `llava`
1 year ago
​
 1234567891011121314151617181920212223242526272829 
  1. using LLama.Native;

  2. using Spectre.Console;

  3. 

  4. AnsiConsole.MarkupLineInterpolated(

  5.     $"""

  6.     [purple]======================================================================================================[/]

  7.      __       __                                       ____     __

  8.     /\ \     /\ \                                     /\  _`\  /\ \

  9.     \ \ \    \ \ \         __       ___ ___       __  \ \,\L\_\\ \ \___       __     _ __   _____

  10.      \ \ \  __\ \ \  __  /'__`\   /' __` __`\   /'__`\ \/_\__ \ \ \  _ `\   /'__`\  /\` __\/\  __`\

  11.       \ \ \L\ \\ \ \L\ \/\ \L\.\_ /\ \/\ \/\ \ /\ \L\.\_ /\ \L\ \\ \ \ \ \ /\ \L\.\_\ \ \/ \ \ \L\ \

  12.        \ \____/ \ \____/\ \__/.\_\\ \_\ \_\ \_\\ \__/.\_\\ `\____\\ \_\ \_\\ \__/.\_\\ \_\  \ \ ,__/

  13.         \/___/   \/___/  \/__/\/_/ \/_/\/_/\/_/ \/__/\/_/ \/_____/ \/_/\/_/ \/__/\/_/ \/_/   \ \ \/

  14.     [purple]=========================================================================================[/] \ \_\ [purple]======[/]

  15.                                                                                                \/_/

  16. 

  17.     """);

  18. 

  19. // Configure native library to use

  20. NativeLibraryConfig

  21.    .Instance

  22.    .WithCuda()

  23.    .WithLogs(LLamaLogLevel.Info);

  24. 

  25. // Calling this method forces loading to occur now.

  26. NativeApi.llama_empty_call();

  27. 

  28. await ExampleRunner.Run();

  29.

C#/.NET上易用的LLM高性能推理框架，支持LLaMA和LLaVA系列模型。