From a7b9d29083677fc9571c52885c4b4bba00a5fd1e Mon Sep 17 00:00:00 2001 From: Yaohui Liu Date: Wed, 17 May 2023 03:27:14 +0800 Subject: [PATCH] docs: update the readme file. --- README.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 899174a7..55b414be 100644 --- a/README.md +++ b/README.md @@ -13,21 +13,29 @@ both Windows and Linux and does NOT require compiling llama.cpp yourself. ## Installation -Just search `LLamaSharp` or `LLamaSharp.Cpu` in nuget package manager and install it! +Firstly, search `LLamaSharp` in nuget package manager and install it. ``` PM> Install-Package LLamaSharp ``` -If you want to run it only on CPUs, please install `LLamaSharp.Cpu`. The only difference between them is native library. +Then, search and install one of the following backends: -If you have GPU on windows but fail to run with the error `failed to load libllama.dll or one of its dependencies`, please try downloading the cublas support dlls [here](https://github.com/ggerganov/llama.cpp/releases). For example, [cudart-llama-bin-win-cu11.7.1-x64.zip -](https://github.com/ggerganov/llama.cpp/releases/download/master-773ee24/cudart-llama-bin-win-cu11.7.1-x64.zip) is the one for CUDA 11.7. +``` +LLamaSharp.Backend.Cpu +LLamaSharp.Backend.Cuda11 +LLamaSharp.Backend.Cuda12 +``` + +Note that version v0.2.1 has a package named `LLamaSharp.Cpu`. After v0.2.2 it will be dropped. + +We publish the backend with cpu, cuda11 and cuda12 because they are the most popular ones. If none of them matches, please compile the [llama.cpp](https://github.com/ggerganov/llama.cpp) +from source and put the `libllama` under your project's output path. When building from source, please add `-DBUILD_SHARED_LIBS=ON` to enable the library generation. ## Simple Benchmark Currently it's only a simple benchmark to indicate that the performance of `LLamaSharp` is close to `llama.cpp`. Experiments run on a computer -with Intel i7-12700, 3060Ti with 7B model. Note that the benchmark uses `LLamaModel` instead of `LLamaModelV1`. +with Intel i7-12700, 3060Ti with 7B model. Note that the benchmark uses `LLamaModel` instead of `LLamaModelV1`. #### Windows @@ -82,6 +90,8 @@ else } ``` +For more usages, please refer to [Examples](./LLama.Examples). + #### Web API We provide the integration of ASP.NET core [here](./LLama.WebAPI). Since currently the API is not stable, please clone the repo and use it. In the future we'll publish it on NuGet.