You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

Quantize.cs 620 B

12345678910111213141516171819202122232425262728
  1. using System;
  2. using System.Collections.Generic;
  3. using System.Linq;
  4. using System.Text;
  5. using System.Threading.Tasks;
  6. namespace LLama.Examples
  7. {
  8. public class Quantize
  9. {
  10. public Quantize()
  11. {
  12. }
  13. public void Run(string srcFileName, string dstFilename, string ftype, int nthread = -1)
  14. {
  15. if(Quantizer.Quantize(srcFileName, dstFilename, ftype, nthread))
  16. {
  17. Console.WriteLine("Quantization succeed!");
  18. }
  19. else
  20. {
  21. Console.WriteLine("Quantization failed!");
  22. }
  23. }
  24. }
  25. }

C#/.NET上易用的LLM高性能推理框架,支持LLaMA和LLaVA系列模型。