LLamaSharp/docs/xmldocs/llama.native.nativelibraryconfig.md

• NativeLibraryConfig
• • Properties
  • • Instance
    • • Property Value
    • LibraryHasLoaded
    • • Property Value
  • Methods
  • • WithLibrary(String, String)
    • • Parameters
      • Returns
      • Exceptions
    • WithCuda(Boolean)
    • • Parameters
      • Returns
      • Exceptions
    • WithAvx(AvxLevel)
    • • Parameters
      • Returns
      • Exceptions
    • WithAutoFallback(Boolean)
    • • Parameters
      • Returns
      • Exceptions
    • SkipCheck(Boolean)
    • • Parameters
      • Returns
      • Exceptions
    • WithLogs(Boolean)
    • • Parameters
      • Returns
      • Exceptions
    • WithLogs(LLamaLogLevel)
    • • Parameters
      • Returns
      • Exceptions
    • WithSearchDirectories(IEnumerable<String>)
    • • Parameters
      • Returns
    • WithSearchDirectory(String)
    • • Parameters
      • Returns
    • CheckAndGatherDescription(LibraryName)
    • • Parameters
      • Returns
    • AvxLevelToString(AvxLevel)
    • • Parameters
      • Returns

NativeLibraryConfig

Namespace: LLama.Native

Allows configuration of the native llama.cpp libraries to load and use.
All configuration must be done before using any other LLamaSharp methods!

public sealed class NativeLibraryConfig

Inheritance Object → NativeLibraryConfig

Properties

Instance

Get the config instance

public static NativeLibraryConfig Instance { get; }

Property Value

NativeLibraryConfig

LibraryHasLoaded

Check if the native library has already been loaded. Configuration cannot be modified if this is true.

public static bool LibraryHasLoaded { get; internal set; }

Property Value

Boolean

Methods

WithLibrary(String, String)

Load a specified native library as backend for LLamaSharp.
When this method is called, all the other configurations will be ignored.

public NativeLibraryConfig WithLibrary(string llamaPath, string llavaPath)

Parameters

llamaPath String

The full path to the llama library to load.

llavaPath String

The full path to the llava library to load.

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

WithCuda(Boolean)

Configure whether to use cuda backend if possible.

public NativeLibraryConfig WithCuda(bool enable)

Parameters

enable Boolean

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

WithAvx(AvxLevel)

Configure the prefferred avx support level of the backend.

public NativeLibraryConfig WithAvx(AvxLevel level)

Parameters

level AvxLevel

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

WithAutoFallback(Boolean)

Configure whether to allow fallback when there's no match for preferred settings.

public NativeLibraryConfig WithAutoFallback(bool enable)

Parameters

enable Boolean

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

SkipCheck(Boolean)

Whether to skip the check when you don't allow fallback. This option
may be useful under some complex conditions. For example, you're sure
you have your cublas configured but LLamaSharp take it as invalid by mistake.

public NativeLibraryConfig SkipCheck(bool enable)

Parameters

enable Boolean

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

WithLogs(Boolean)

Whether to output the logs to console when loading the native library with your configuration.

public NativeLibraryConfig WithLogs(bool enable)

Parameters

enable Boolean

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

WithLogs(LLamaLogLevel)

Enable console logging with the specified log logLevel.

public NativeLibraryConfig WithLogs(LLamaLogLevel logLevel)

Parameters

logLevel LLamaLogLevel

Returns

NativeLibraryConfig

Exceptions

InvalidOperationException

Thrown if LibraryHasLoaded is true.

WithSearchDirectories(IEnumerable<String>)

Add self-defined search directories. Note that the file stucture of the added
directories must be the same as the default directory. Besides, the directory
won't be used recursively.

public NativeLibraryConfig WithSearchDirectories(IEnumerable<string> directories)

Parameters

directories IEnumerable<String>

Returns

NativeLibraryConfig

WithSearchDirectory(String)

Add self-defined search directories. Note that the file stucture of the added
directories must be the same as the default directory. Besides, the directory
won't be used recursively.

public NativeLibraryConfig WithSearchDirectory(string directory)

Parameters

directory String

Returns

NativeLibraryConfig

CheckAndGatherDescription(LibraryName)

internal static Description CheckAndGatherDescription(LibraryName library)

Parameters

library LibraryName

Returns

Description

AvxLevelToString(AvxLevel)

internal static string AvxLevelToString(AvxLevel level)

Parameters

level AvxLevel

Returns

String