You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

NativeApi.Quantize.cs 813 B

12345678910111213141516171819202122
  1. using System;
  2. using System.Collections.Generic;
  3. using System.Runtime.InteropServices;
  4. using System.Text;
  5. namespace LLama.Native
  6. {
  7. internal partial class NativeApi
  8. {
  9. /// <summary>
  10. /// Returns 0 on success
  11. /// </summary>
  12. /// <param name="fname_inp"></param>
  13. /// <param name="fname_out"></param>
  14. /// <param name="ftype"></param>
  15. /// <param name="nthread">how many threads to use. If <=0, will use std::thread::hardware_concurrency(), else the number given</param>
  16. /// <remarks>not great API - very likely to change</remarks>
  17. /// <returns>Returns 0 on success</returns>
  18. [DllImport(libraryName)]
  19. public static extern int llama_model_quantize(string fname_inp, string fname_out, LLamaFtype ftype, int nthread);
  20. }
  21. }

C#/.NET上易用的LLM高性能推理框架,支持LLaMA和LLaVA系列模型。