You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

llama.statelessexecutor.md 2.1 kB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980
  1. # StatelessExecutor
  2. Namespace: LLama
  3. This executor infer the input as one-time job. Previous inputs won't impact on the
  4. response to current input.
  5. ```csharp
  6. public class StatelessExecutor : LLama.Abstractions.ILLamaExecutor
  7. ```
  8. Inheritance [Object](https://docs.microsoft.com/en-us/dotnet/api/system.object) → [StatelessExecutor](./llama.statelessexecutor.md)<br>
  9. Implements [ILLamaExecutor](./llama.abstractions.illamaexecutor.md)
  10. ## Properties
  11. ### **Model**
  12. The mode used by the executor when running the inference.
  13. ```csharp
  14. public LLamaModel Model { get; }
  15. ```
  16. #### Property Value
  17. [LLamaModel](./llama.llamamodel.md)<br>
  18. ## Constructors
  19. ### **StatelessExecutor(LLamaModel)**
  20. ```csharp
  21. public StatelessExecutor(LLamaModel model)
  22. ```
  23. #### Parameters
  24. `model` [LLamaModel](./llama.llamamodel.md)<br>
  25. The LLama model.
  26. ## Methods
  27. ### **Infer(String, InferenceParams, CancellationToken)**
  28. ```csharp
  29. public IEnumerable<string> Infer(string text, InferenceParams inferenceParams, CancellationToken cancellationToken)
  30. ```
  31. #### Parameters
  32. `text` [String](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>
  33. `inferenceParams` [InferenceParams](./llama.common.inferenceparams.md)<br>
  34. `cancellationToken` [CancellationToken](https://docs.microsoft.com/en-us/dotnet/api/system.threading.cancellationtoken)<br>
  35. #### Returns
  36. [IEnumerable&lt;String&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.ienumerable-1)<br>
  37. ### **InferAsync(String, InferenceParams, CancellationToken)**
  38. ```csharp
  39. public IAsyncEnumerable<string> InferAsync(string text, InferenceParams inferenceParams, CancellationToken token)
  40. ```
  41. #### Parameters
  42. `text` [String](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>
  43. `inferenceParams` [InferenceParams](./llama.common.inferenceparams.md)<br>
  44. `token` [CancellationToken](https://docs.microsoft.com/en-us/dotnet/api/system.threading.cancellationtoken)<br>
  45. #### Returns
  46. [IAsyncEnumerable&lt;String&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.iasyncenumerable-1)<br>

C#/.NET上易用的LLM高性能推理框架,支持LLaMA和LLaVA系列模型。