# node-llama-cpp

> Run AI models locally on your machine

node.js bindings for llama.cpp, and much more

## Table of Contents

### Guide

- [Getting Started](/guide.md): Get started with node-llama-cpp
- [Using `LlamaChatSession`](/guide/chat-session.md): Chatting with a text generation model
- [Chat Wrapper](/guide/chat-wrapper.md): Chat with a model without having to worry about any parsing or formatting
- [Using Grammar](/guide/grammar.md): Using grammar
- [Using Function Calling](/guide/function-calling.md): Using function calling
- [Using Embedding](/guide/embedding.md): Using embeddings with node-llama-cpp
- [Text Completion {#title}](/guide/text-completion.md): Generating text completions with node-llama-cpp
- [Choosing a Model](/guide/choosing-a-model.md): Learn how to choose the right model for your use case
- [Downloading Models](/guide/downloading-models.md): Downloading models with node-llama-cpp

### Advanced

- [Building From Source](/guide/building-from-source.md): Building llama.cpp from source for node-llama-cpp
- [Metal Support](/guide/Metal.md): Metal support in node-llama-cpp
- [CUDA Support](/guide/CUDA.md): CUDA support in node-llama-cpp
- [Using Vulkan](/guide/Vulkan.md): Vulkan support in node-llama-cpp
- [Using in Electron](/guide/electron.md): Using node-llama-cpp in Electron applications
- [Using `node-llama-cpp` in Docker](/guide/docker.md): Using node-llama-cpp in Docker
- [Using Tokens](/guide/tokens.md): The basics of working with tokens in node-llama-cpp
- [Using LlamaText](/guide/llama-text.md): The basics of using LlamaText in node-llama-cpp
- [External Chat State](/guide/external-chat-state.md): Chat with a model and manage the chat state externally
- [Using Token Bias {#title}](/guide/token-bias.md): Using token bias to adjust the probabilities of tokens in the generated response
- [Objects Lifecycle](/guide/objects-lifecycle.md): Objects lifecycle in node-llama-cpp
- [Chat Context Shift Strategy {#background}](/guide/chat-context-shift.md)
- [Using Batching](/guide/batching.md): Using batching in node-llama-cpp
- [Using Token Predictors](/guide/token-prediction.md): Using token predictors to speed up the generation process in node-llama-cpp
- [Low Level API](/guide/low-level-api.md): Learn how to use the low-level API of node-llama-cpp
- [Awesome `node-llama-cpp`](/guide/awesome.md): Awesome projects that use node-llama-cpp
- [Troubleshooting](/guide/troubleshooting.md): Troubleshooting common issues with node-llama-cpp
- [Tips and Tricks](/guide/tips-and-tricks.md): Tips and tricks for using node-llama-cpp

### Contributing

- [Developing `node-llama-cpp`](/guide/development.md): Developing node-llama-cpp
- [Opening a PR on `node-llama-cpp`](/guide/contributing.md): Contributing to node-llama-cpp

### Blog

- [Blog](/blog.md): node-llama-cpp blog
- [Gemma 4 is here!](/blog/v3.19-gemma-4.md): node-llama-cpp v3.19 is here, with full support for Gemma 4 models and improved performance!
- [gpt-oss is here!](/blog/v3.12-gpt-oss.md): Learn how to use gpt-oss to its full potential with node-llama-cpp
- [DeepSeek R1 with function calling](/blog/v3.6-deepseek-r1.md): node-llama-cpp v3.6 is here, with full support for DeepSeek R1, including function calling!
- [node-llama-cpp v3.0](/blog/v3.md): Learn more about the new features in node-llama-cpp v3.0!

### CLI

- [`init` command](/cli/init.md): 'init' command reference
- [`chat` command](/cli/chat.md): 'chat' command reference
- [`pull` command](/cli/pull.md): 'pull' command reference
- [`source` command](/cli/source.md): 'source' command reference
- [`complete` command](/cli/complete.md): 'complete' command reference
- [`infill` command](/cli/infill.md): 'infill' command reference
- [`inspect` command](/cli/inspect.md): 'inspect' command reference

### Functions

- [Function: getLlama()](/api/functions/getLlama.md)
- [Function: resolveModelFile()](/api/functions/resolveModelFile.md)
- [Function: defineChatSessionFunction()](/api/functions/defineChatSessionFunction.md)
- [Function: createModelDownloader()](/api/functions/createModelDownloader.md)
- [Function: resolveChatWrapper()](/api/functions/resolveChatWrapper.md)
- [Function: tokenizeText()](/api/functions/tokenizeText.md)
- [Function: readGgufFileInfo()](/api/functions/readGgufFileInfo.md)
- [Function: getLlamaGpuTypes()](/api/functions/getLlamaGpuTypes.md)
- [Function: jsonDumps()](/api/functions/jsonDumps.md)
- [Function: appendUserMessageToChatHistory()](/api/functions/appendUserMessageToChatHistory.md)
- [Function: combineModelDownloaders()](/api/functions/combineModelDownloaders.md)
- [Function: getModuleVersion()](/api/functions/getModuleVersion.md)

#### Log levels

- [Function: LlamaLogLevelGreaterThan()](/api/functions/LlamaLogLevelGreaterThan.md)
- [Function: LlamaLogLevelGreaterThanOrEqual()](/api/functions/LlamaLogLevelGreaterThanOrEqual.md)

#### Type guards

- [Function: isGgufMetadataOfArchitectureType()](/api/functions/isGgufMetadataOfArchitectureType.md)
- [Function: isChatModelResponseFunctionCall()](/api/functions/isChatModelResponseFunctionCall.md)
- [Function: isChatModelResponseSegment()](/api/functions/isChatModelResponseSegment.md)
- [Function: isLlamaText()](/api/functions/isLlamaText.md)

### Classes

- [Class: Llama](/api/classes/Llama.md)
- [Class: LlamaModel](/api/classes/LlamaModel.md)
- [Class: LlamaContext](/api/classes/LlamaContext.md)
- [Class: LlamaContextSequence](/api/classes/LlamaContextSequence.md)
- [Class: LlamaChatSession](/api/classes/LlamaChatSession.md)
- [Class: LlamaCompletion](/api/classes/LlamaCompletion.md)
- [Class: LlamaEmbeddingContext](/api/classes/LlamaEmbeddingContext.md)
- [Class: LlamaEmbedding](/api/classes/LlamaEmbedding.md)
- [Class: LlamaRankingContext](/api/classes/LlamaRankingContext.md)
- [Class: LlamaGrammar](/api/classes/LlamaGrammar.md)
- [Class: LlamaJsonSchemaGrammar\<T, Defs>](/api/classes/LlamaJsonSchemaGrammar.md)
- [Class: LlamaText](/api/classes/LlamaText.md)
- [Class: TokenBias](/api/classes/TokenBias.md)
- [Class: GgufInsights](/api/classes/GgufInsights.md)
- [Class: LlamaChat](/api/classes/LlamaChat.md)
- [Class: TokenMeter](/api/classes/TokenMeter.md)
- [Class: TokenAttributes](/api/classes/TokenAttributes.md)
- [Class: ModelDownloader](/api/classes/ModelDownloader.md)
- [Class: ChatModelFunctionsDocumentationGenerator](/api/classes/ChatModelFunctionsDocumentationGenerator.md)
- [Abstract Class: TokenPredictor](/api/classes/TokenPredictor.md)
- [Class: DraftSequenceTokenPredictor](/api/classes/DraftSequenceTokenPredictor.md)
- [Class: InputLookupTokenPredictor](/api/classes/InputLookupTokenPredictor.md)
- [Class: LlamaGrammarEvaluationState](/api/classes/LlamaGrammarEvaluationState.md)
- [Class: GgufInsightsTokens](/api/classes/GgufInsightsTokens.md)
- [Class: CombinedModelDownloader](/api/classes/CombinedModelDownloader.md)
- [Abstract Class: ChatWrapper](/api/classes/ChatWrapper.md)

#### LlamaModel

- [Class: LlamaModelTokens](/api/classes/LlamaModelTokens.md)

##### LlamaModelTokens

- [Class: LlamaModelInfillTokens](/api/classes/LlamaModelInfillTokens.md)

#### LlamaChatSession

- [Class: LlamaChatSessionPromptCompletionEngine](/api/classes/LlamaChatSessionPromptCompletionEngine.md)

#### LlamaText

- [Class: SpecialTokensText](/api/classes/SpecialTokensText.md)
- [Class: SpecialToken](/api/classes/SpecialToken.md)

#### GgufInsights

- [Class: GgufInsightsConfigurationResolver](/api/classes/GgufInsightsConfigurationResolver.md)

#### Chat wrappers

- [Class: GeneralChatWrapper](/api/classes/GeneralChatWrapper.md)
- [Class: TemplateChatWrapper](/api/classes/TemplateChatWrapper.md)
- [Class: JinjaTemplateChatWrapper](/api/classes/JinjaTemplateChatWrapper.md)
- [Class: QwenChatWrapper](/api/classes/QwenChatWrapper.md)
- [Class: HarmonyChatWrapper](/api/classes/HarmonyChatWrapper.md)
- [Class: SeedChatWrapper](/api/classes/SeedChatWrapper.md)
- [Class: DeepSeekChatWrapper](/api/classes/DeepSeekChatWrapper.md)
- [Class: Llama3\_1ChatWrapper](/api/classes/Llama3_1ChatWrapper.md)
- [Class: Llama3\_2LightweightChatWrapper](/api/classes/Llama3_2LightweightChatWrapper.md)
- [Class: Llama3ChatWrapper](/api/classes/Llama3ChatWrapper.md)
- [Class: Llama2ChatWrapper](/api/classes/Llama2ChatWrapper.md)
- [Class: MistralChatWrapper](/api/classes/MistralChatWrapper.md)
- [Class: Gemma4ChatWrapper](/api/classes/Gemma4ChatWrapper.md)
- [Class: GemmaChatWrapper](/api/classes/GemmaChatWrapper.md)
- [Class: ChatMLChatWrapper](/api/classes/ChatMLChatWrapper.md)
- [Class: FalconChatWrapper](/api/classes/FalconChatWrapper.md)
- [Class: AlpacaChatWrapper](/api/classes/AlpacaChatWrapper.md)
- [Class: FunctionaryChatWrapper](/api/classes/FunctionaryChatWrapper.md)
- [Class: EmptyChatWrapper](/api/classes/EmptyChatWrapper.md)
- [Class: MuseChatWrapper](/api/classes/MuseChatWrapper.md)

#### Errors

- [Class: NoBinaryFoundError](/api/classes/NoBinaryFoundError.md)
- [Class: InsufficientMemoryError](/api/classes/InsufficientMemoryError.md)
- [Class: UnsupportedError](/api/classes/UnsupportedError.md)
- [Class: LlamaJsonSchemaValidationError](/api/classes/LlamaJsonSchemaValidationError.md)

### Types

- [Type Alias: Token](/api/type-aliases/Token.md)
- [Type Alias: Tokenizer](/api/type-aliases/Tokenizer.md)
- [Type Alias: Detokenizer](/api/type-aliases/Detokenizer.md)
- [Type Alias: LlamaGpuType](/api/type-aliases/LlamaGpuType.md)
- [Type Alias: LlamaNuma](/api/type-aliases/LlamaNuma.md)
- [Type Alias: RamState](/api/type-aliases/RamState.md)
- [Type Alias: SpecializedChatWrapperTypeName](/api/type-aliases/SpecializedChatWrapperTypeName.md)
- [Type Alias: TemplateChatWrapperTypeName](/api/type-aliases/TemplateChatWrapperTypeName.md)
- [Type Alias: ResolvableChatWrapperTypeName](/api/type-aliases/ResolvableChatWrapperTypeName.md)
- [Type Alias: BuiltInChatWrapperType](/api/type-aliases/BuiltInChatWrapperType.md)
- [Type Alias: LlamaChatResponseChunk](/api/type-aliases/LlamaChatResponseChunk.md)
- [Type Alias: LlamaChatResponseTextChunk](/api/type-aliases/LlamaChatResponseTextChunk.md)
- [Type Alias: LlamaChatResponseSegmentChunk](/api/type-aliases/LlamaChatResponseSegmentChunk.md)
- [Type Alias: LlamaChatResponseFunctionCallParamsChunk](/api/type-aliases/LlamaChatResponseFunctionCallParamsChunk.md)
- [Type Alias: LlamaChatResponseSegment](/api/type-aliases/LlamaChatResponseSegment.md)
- [Type Alias: LlamaChatLoadAndCompleteUserResponse](/api/type-aliases/LlamaChatLoadAndCompleteUserResponse.md)
- [Type Alias: LlamaChatSessionDryRepeatPenalty](/api/type-aliases/LlamaChatSessionDryRepeatPenalty.md)
- [Type Alias: LlamaCompletionResponse](/api/type-aliases/LlamaCompletionResponse.md)
- [Type Alias: LlamaContextSequenceRepeatPenalty](/api/type-aliases/LlamaContextSequenceRepeatPenalty.md)
- [Type Alias: LlamaContextSequenceDryRepeatPenalty](/api/type-aliases/LlamaContextSequenceDryRepeatPenalty.md)
- [Type Alias: ContextTokensDeleteRange](/api/type-aliases/ContextTokensDeleteRange.md)
- [Type Alias: SequenceEvaluateOutput\<Options>](/api/type-aliases/SequenceEvaluateOutput.md)
- [Type Alias: ControlledEvaluateInputItem](/api/type-aliases/ControlledEvaluateInputItem.md)
- [Type Alias: ControlledEvaluateIndexOutput](/api/type-aliases/ControlledEvaluateIndexOutput.md)
- [Type Alias: EvaluationPriority](/api/type-aliases/EvaluationPriority.md)
- [Type Alias: LlamaEmbeddingJSON](/api/type-aliases/LlamaEmbeddingJSON.md)
- [Type Alias: TokenMeterState](/api/type-aliases/TokenMeterState.md)
- [Type Alias: GgufInsightsResourceRequirements](/api/type-aliases/GgufInsightsResourceRequirements.md)
- [Type Alias: GgufFileInfoSource](/api/type-aliases/GgufFileInfoSource.md)
- [Type Alias: GgufFileInfoSourceData](/api/type-aliases/GgufFileInfoSourceData.md)
- [Type Alias: ChatWrapperSettings](/api/type-aliases/ChatWrapperSettings.md)
- [Type Alias: ChatWrapperSettingsSegment](/api/type-aliases/ChatWrapperSettingsSegment.md)
- [Type Alias: ChatWrapperGeneratedContextState](/api/type-aliases/ChatWrapperGeneratedContextState.md)
- [Type Alias: ChatWrapperGeneratedPrefixTriggersContextState](/api/type-aliases/ChatWrapperGeneratedPrefixTriggersContextState.md)
- [Type Alias: ChatWrapperGeneratedInitiallyEngagedFunctionsContextState](/api/type-aliases/ChatWrapperGeneratedInitiallyEngagedFunctionsContextState.md)
- [Type Alias: ChatModelSegmentType](/api/type-aliases/ChatModelSegmentType.md)
- [Type Alias: ChatModelSegment](/api/type-aliases/ChatModelSegment.md)
- [Type Alias: LLamaContextualRepeatPenalty](/api/type-aliases/LLamaContextualRepeatPenalty.md)
- [Type Alias: LLamaContextualDryRepeatPenalty](/api/type-aliases/LLamaContextualDryRepeatPenalty.md)
- [Type Alias: OverridesObject\<T, AllowedValueTypes>](/api/type-aliases/OverridesObject.md)
- [Type Alias: LlamaClasses](/api/type-aliases/LlamaClasses.md)
- [Type Alias: ModelDownloadEndpoints](/api/type-aliases/ModelDownloadEndpoints.md)
- [Type Alias: ModelFileAccessTokens](/api/type-aliases/ModelFileAccessTokens.md)
- [Type Alias: GbnfJsonSchema\<Defs>](/api/type-aliases/GbnfJsonSchema.md)
- [Type Alias: ChatHistoryItem](/api/type-aliases/ChatHistoryItem.md)
- [Type Alias: LlamaChatResponse\<Functions>](/api/type-aliases/LlamaChatResponse.md)
- [Type Alias: GgufFileInfo](/api/type-aliases/GgufFileInfo.md)

#### GbnfJsonSchema

- [Type Alias: GbnfJsonSchemaImmutableType](/api/type-aliases/GbnfJsonSchemaImmutableType.md)
- [Type Alias: GbnfJsonDefList\<Defs>](/api/type-aliases/GbnfJsonDefList.md)
- [Type Alias: GbnfJsonBasicSchema](/api/type-aliases/GbnfJsonBasicSchema.md)
- [Type Alias: GbnfJsonConstSchema](/api/type-aliases/GbnfJsonConstSchema.md)
- [Type Alias: GbnfJsonEnumSchema](/api/type-aliases/GbnfJsonEnumSchema.md)
- [Type Alias: GbnfJsonOneOfSchema\<Defs>](/api/type-aliases/GbnfJsonOneOfSchema.md)
- [Type Alias: GbnfJsonStringSchema](/api/type-aliases/GbnfJsonStringSchema.md)
- [Type Alias: GbnfJsonBasicStringSchema](/api/type-aliases/GbnfJsonBasicStringSchema.md)
- [Type Alias: GbnfJsonFormatStringSchema](/api/type-aliases/GbnfJsonFormatStringSchema.md)
- [Type Alias: GbnfJsonObjectSchema\<Keys, Defs>](/api/type-aliases/GbnfJsonObjectSchema.md)
- [Type Alias: GbnfJsonArraySchema\<Defs>](/api/type-aliases/GbnfJsonArraySchema.md)
- [Type Alias: GbnfJsonRefSchema\<Defs>](/api/type-aliases/GbnfJsonRefSchema.md)
- [Type Alias: GbnfJsonSchemaToType\<T>](/api/type-aliases/GbnfJsonSchemaToType.md)

#### ChatHistoryItem

- [Type Alias: ChatSystemMessage](/api/type-aliases/ChatSystemMessage.md)
- [Type Alias: ChatUserMessage](/api/type-aliases/ChatUserMessage.md)
- [Type Alias: ChatModelResponse](/api/type-aliases/ChatModelResponse.md)

##### ChatModelResponse

- [Type Alias: ChatModelFunctionCall](/api/type-aliases/ChatModelFunctionCall.md)

#### LlamaChatResponse

- [Type Alias: LlamaChatResponseFunctionCall\<Functions, FunctionCallName, Params>](/api/type-aliases/LlamaChatResponseFunctionCall.md)

#### LlamaText

- [Type Alias: LlamaTextValue](/api/type-aliases/LlamaTextValue.md)
- [Type Alias: LlamaTextInputValue](/api/type-aliases/LlamaTextInputValue.md)
- [Type Alias: LlamaTextJSON](/api/type-aliases/LlamaTextJSON.md)
- [Type Alias: LlamaTextJSONValue](/api/type-aliases/LlamaTextJSONValue.md)
- [Type Alias: LlamaTextSpecialTokensTextJSON](/api/type-aliases/LlamaTextSpecialTokensTextJSON.md)
- [Type Alias: LlamaTextSpecialTokenJSON](/api/type-aliases/LlamaTextSpecialTokenJSON.md)
- [Type Alias: BuiltinSpecialTokenValue](/api/type-aliases/BuiltinSpecialTokenValue.md)

#### GgufFileInfo

- [Type Alias: GgufTensorInfo](/api/type-aliases/GgufTensorInfo.md)
- [Type Alias: GgufMetadata\<A>](/api/type-aliases/GgufMetadata.md)

##### GgufMetadata

- [Type Alias: GgufMetadataLlmToType](/api/type-aliases/GgufMetadataLlmToType.md)
- [Type Alias: GgufMetadataGeneral\<A>](/api/type-aliases/GgufMetadataGeneral.md)
- [Type Alias: GgufMetadataTokenizer](/api/type-aliases/GgufMetadataTokenizer.md)
- [Type Alias: GgufMetadataDefaultArchitectureType](/api/type-aliases/GgufMetadataDefaultArchitectureType.md)
- [Type Alias: GgufMetadataLlmLLaMA](/api/type-aliases/GgufMetadataLlmLLaMA.md)
- [Type Alias: GgufMetadataMPT](/api/type-aliases/GgufMetadataMPT.md)
- [Type Alias: GgufMetadataGPTNeoX](/api/type-aliases/GgufMetadataGPTNeoX.md)
- [Type Alias: GgufMetadataGPTJ](/api/type-aliases/GgufMetadataGPTJ.md)
- [Type Alias: GgufMetadataGPT2](/api/type-aliases/GgufMetadataGPT2.md)
- [Type Alias: GgufMetadataBloom](/api/type-aliases/GgufMetadataBloom.md)
- [Type Alias: GgufMetadataFalcon](/api/type-aliases/GgufMetadataFalcon.md)
- [Type Alias: GgufMetadataMamba](/api/type-aliases/GgufMetadataMamba.md)

#### Options

- [Type Alias: LlamaOptions](/api/type-aliases/LlamaOptions.md)
- [Type Alias: LastBuildOptions](/api/type-aliases/LastBuildOptions.md)
- [Type Alias: LlamaExperimentalOptions](/api/type-aliases/LlamaExperimentalOptions.md)
- [Type Alias: TemplateChatWrapperSegmentsOptions](/api/type-aliases/TemplateChatWrapperSegmentsOptions.md)
- [Type Alias: ResolveChatWrapperWithModelOptions](/api/type-aliases/ResolveChatWrapperWithModelOptions.md)
- [Type Alias: LlamaChatOptions](/api/type-aliases/LlamaChatOptions.md)
- [Type Alias: LLamaChatGenerateResponseOptions\<Functions>](/api/type-aliases/LLamaChatGenerateResponseOptions.md)
- [Type Alias: LLamaChatLoadAndCompleteUserMessageOptions\<Functions>](/api/type-aliases/LLamaChatLoadAndCompleteUserMessageOptions.md)
- [Type Alias: LLamaChatContextShiftOptions](/api/type-aliases/LLamaChatContextShiftOptions.md)
- [Type Alias: LLamaChatCompletePromptOptions](/api/type-aliases/LLamaChatCompletePromptOptions.md)
- [Type Alias: LLamaChatPreloadPromptOptions](/api/type-aliases/LLamaChatPreloadPromptOptions.md)
- [Type Alias: LLamaChatPromptCompletionEngineOptions](/api/type-aliases/LLamaChatPromptCompletionEngineOptions.md)
- [Type Alias: LlamaCompletionOptions](/api/type-aliases/LlamaCompletionOptions.md)
- [Type Alias: LlamaCompletionGenerationOptions](/api/type-aliases/LlamaCompletionGenerationOptions.md)
- [Type Alias: LlamaInfillGenerationOptions](/api/type-aliases/LlamaInfillGenerationOptions.md)
- [Type Alias: ContextShiftOptions](/api/type-aliases/ContextShiftOptions.md)
- [Type Alias: SequenceEvaluateOptions](/api/type-aliases/SequenceEvaluateOptions.md)
- [Type Alias: SequenceEvaluateMetadataOptions](/api/type-aliases/SequenceEvaluateMetadataOptions.md)
- [Type Alias: LlamaEmbeddingOptions](/api/type-aliases/LlamaEmbeddingOptions.md)
- [Type Alias: LlamaEmbeddingContextOptions](/api/type-aliases/LlamaEmbeddingContextOptions.md)
- [Type Alias: LlamaGrammarOptions](/api/type-aliases/LlamaGrammarOptions.md)
- [Type Alias: LlamaGrammarEvaluationStateOptions](/api/type-aliases/LlamaGrammarEvaluationStateOptions.md)
- [Type Alias: LlamaModelOptions](/api/type-aliases/LlamaModelOptions.md)
- [Type Alias: LlamaRankingContextOptions](/api/type-aliases/LlamaRankingContextOptions.md)
- [Type Alias: ChatWrapperGenerateContextStateOptions](/api/type-aliases/ChatWrapperGenerateContextStateOptions.md)
- [Type Alias: ChatWrapperGenerateInitialHistoryOptions](/api/type-aliases/ChatWrapperGenerateInitialHistoryOptions.md)
- [Type Alias: ModelDownloaderOptions](/api/type-aliases/ModelDownloaderOptions.md)
- [Type Alias: CombinedModelDownloaderOptions](/api/type-aliases/CombinedModelDownloaderOptions.md)
- [Type Alias: ResolveModelFileOptions](/api/type-aliases/ResolveModelFileOptions.md)
- [Type Alias: LlamaContextOptions](/api/type-aliases/LlamaContextOptions.md)
- [Type Alias: LlamaChatSessionOptions](/api/type-aliases/LlamaChatSessionOptions.md)
- [Type Alias: LLamaChatPromptOptions\<Functions>](/api/type-aliases/LLamaChatPromptOptions.md)

##### LlamaContextOptions

- [Type Alias: BatchingOptions](/api/type-aliases/BatchingOptions.md)

###### BatchingOptions

- [Type Alias: CustomBatchingDispatchSchedule()](/api/type-aliases/CustomBatchingDispatchSchedule.md)
- [Type Alias: CustomBatchingPrioritizationStrategy()](/api/type-aliases/CustomBatchingPrioritizationStrategy.md)
- [Type Alias: BatchItem](/api/type-aliases/BatchItem.md)
- [Type Alias: PrioritizedBatchItem](/api/type-aliases/PrioritizedBatchItem.md)

##### LlamaChatSessionOptions

- [Type Alias: LlamaChatSessionContextShiftOptions](/api/type-aliases/LlamaChatSessionContextShiftOptions.md)
- [Type Alias: ChatSessionModelFunction\<Params>](/api/type-aliases/ChatSessionModelFunction.md)

##### LLamaChatPromptOptions

- [Type Alias: LlamaChatSessionRepeatPenalty](/api/type-aliases/LlamaChatSessionRepeatPenalty.md)
- [Type Alias: ChatModelFunctions](/api/type-aliases/ChatModelFunctions.md)
- [Type Alias: ChatSessionModelFunctions](/api/type-aliases/ChatSessionModelFunctions.md)

##### Chat Wrapper Options

- [Type Alias: TemplateChatWrapperOptions](/api/type-aliases/TemplateChatWrapperOptions.md)
- [Type Alias: ChatHistoryFunctionCallMessageTemplate](/api/type-aliases/ChatHistoryFunctionCallMessageTemplate.md)
- [Type Alias: ResolveChatWrapperOptions](/api/type-aliases/ResolveChatWrapperOptions.md)
- [Type Alias: JinjaTemplateChatWrapperOptions](/api/type-aliases/JinjaTemplateChatWrapperOptions.md)

###### JinjaTemplateChatWrapperOptions

- [Type Alias: JinjaTemplateChatWrapperOptionsConvertMessageFormat](/api/type-aliases/JinjaTemplateChatWrapperOptionsConvertMessageFormat.md)

### Enums

- [Enumeration: LlamaLogLevel](/api/enumerations/LlamaLogLevel.md)
- [Enumeration: LlamaVocabularyType](/api/enumerations/LlamaVocabularyType.md)
- [Enumeration: GgufArchitectureType](/api/enumerations/GgufArchitectureType.md)
- [Enumeration: GgufFileType](/api/enumerations/GgufFileType.md)
- [Enumeration: GgufMetadataTokenizerTokenType](/api/enumerations/GgufMetadataTokenizerTokenType.md)
- [Enumeration: GgufMetadataArchitecturePoolingType](/api/enumerations/GgufMetadataArchitecturePoolingType.md)
- [Enumeration: GgmlType](/api/enumerations/GgmlType.md)

### Variables

- [Variable: specializedChatWrapperTypeNames](/api/variables/specializedChatWrapperTypeNames.md)
- [Variable: templateChatWrapperTypeNames](/api/variables/templateChatWrapperTypeNames.md)
- [Variable: resolvableChatWrapperTypeNames](/api/variables/resolvableChatWrapperTypeNames.md)
- [Variable: chatWrappers](/api/variables/chatWrappers.md)
- [Variable: ggufFileQuantNames](/api/variables/ggufFileQuantNames.md)