Function: getLlama()
getLlama(options)
function getLlama(options?: LlamaOptions): Promise<Llama>
Get a llama.cpp
binding.
Defaults to use a local binary built using the source download
or source build
CLI commands if one exists, otherwise, uses a prebuilt binary, and fallbacks to building from source if a prebuilt binary is not found.
Pass "lastBuild"
to default to use the last successful build created using the source download
or source build
CLI commands if one exists.
The difference between using "lastBuild"
and not using it is that "lastBuild"
will use the binary built using a CLI command with the configuration used to build that binary (like using its GPU type), while not using "lastBuild"
will only attempt to only use a binary that complies with the given options.
For example, if your machine supports both CUDA and Vulkan, and you run the source download --gpu vulkan
command, calling getLlama("lastBuild")
will return the binary you built with Vulkan, while calling getLlama()
will return a binding from a pre-built binary with CUDA, since CUDA is preferable on systems that support it.
For example, if your machine supports CUDA, and you run the source download --gpu cuda
command, calling getLlama("lastBuild")
will return the binary you built with CUDA, and calling getLlama()
will also return that same binary you built with CUDA.
You should prefer to use getLlama()
without "lastBuild"
unless you have a specific reason to use the last build.
Parameters
Parameter | Type |
---|---|
options ? | LlamaOptions |
Returns
Defined in
getLlama(type, lastBuildOptions)
function getLlama(type: "lastBuild", lastBuildOptions?: LastBuildOptions): Promise<Llama>
Parameters
Parameter | Type |
---|---|
type | "lastBuild" |
lastBuildOptions ? | LastBuildOptions |