You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

llama.native.llamaftype.md 1.7 kB

1234567891011121314151617181920212223242526272829303132333435
  1. # LLamaFtype
  2. Namespace: LLama.Native
  3. Supported model file types
  4. ```csharp
  5. public enum LLamaFtype
  6. ```
  7. Inheritance [Object](https://docs.microsoft.com/en-us/dotnet/api/system.object) → [ValueType](https://docs.microsoft.com/en-us/dotnet/api/system.valuetype) → [Enum](https://docs.microsoft.com/en-us/dotnet/api/system.enum) → [LLamaFtype](./llama.native.llamaftype.md)<br>
  8. Implements [IComparable](https://docs.microsoft.com/en-us/dotnet/api/system.icomparable), [IFormattable](https://docs.microsoft.com/en-us/dotnet/api/system.iformattable), [IConvertible](https://docs.microsoft.com/en-us/dotnet/api/system.iconvertible)
  9. ## Fields
  10. | Name | Value | Description |
  11. | --- | --: | --- |
  12. | LLAMA_FTYPE_ALL_F32 | 0 | All f32 |
  13. | LLAMA_FTYPE_MOSTLY_F16 | 1 | Mostly f16 |
  14. | LLAMA_FTYPE_MOSTLY_Q8_0 | 7 | Mostly 8 bit |
  15. | LLAMA_FTYPE_MOSTLY_Q4_0 | 2 | Mostly 4 bit |
  16. | LLAMA_FTYPE_MOSTLY_Q4_1 | 3 | Mostly 4 bit |
  17. | LLAMA_FTYPE_MOSTLY_Q4_1_SOME_F16 | 4 | Mostly 4 bit, tok_embeddings.weight and output.weight are f16 |
  18. | LLAMA_FTYPE_MOSTLY_Q5_0 | 8 | Mostly 5 bit |
  19. | LLAMA_FTYPE_MOSTLY_Q5_1 | 9 | Mostly 5 bit |
  20. | LLAMA_FTYPE_MOSTLY_Q2_K | 10 | K-Quant 2 bit |
  21. | LLAMA_FTYPE_MOSTLY_Q3_K_S | 11 | K-Quant 3 bit (Small) |
  22. | LLAMA_FTYPE_MOSTLY_Q3_K_M | 12 | K-Quant 3 bit (Medium) |
  23. | LLAMA_FTYPE_MOSTLY_Q3_K_L | 13 | K-Quant 3 bit (Large) |
  24. | LLAMA_FTYPE_MOSTLY_Q4_K_S | 14 | K-Quant 4 bit (Small) |
  25. | LLAMA_FTYPE_MOSTLY_Q4_K_M | 15 | K-Quant 4 bit (Medium) |
  26. | LLAMA_FTYPE_MOSTLY_Q5_K_S | 16 | K-Quant 5 bit (Small) |
  27. | LLAMA_FTYPE_MOSTLY_Q5_K_M | 17 | K-Quant 5 bit (Medium) |
  28. | LLAMA_FTYPE_MOSTLY_Q6_K | 18 | K-Quant 6 bit |
  29. | LLAMA_FTYPE_GUESSED | 1024 | File type was not specified |