Type Alias: LLamaChatLoadAndCompleteUserMessageOptions<Functions>

type LLamaChatLoadAndCompleteUserMessageOptions<Functions>: {
  initialUserPrompt: string;
  stopOnAbortSignal: boolean;
  onTextChunk: LLamaChatGenerateResponseOptions<Functions>["onTextChunk"];
  onToken: LLamaChatGenerateResponseOptions<Functions>["onToken"];
  signal: LLamaChatGenerateResponseOptions<Functions>["signal"];
  maxTokens: LLamaChatGenerateResponseOptions<Functions>["maxTokens"];
  temperature: LLamaChatGenerateResponseOptions<Functions>["temperature"];
  minP: LLamaChatGenerateResponseOptions<Functions>["minP"];
  topK: LLamaChatGenerateResponseOptions<Functions>["topK"];
  topP: LLamaChatGenerateResponseOptions<Functions>["topP"];
  seed: LLamaChatGenerateResponseOptions<Functions>["seed"];
  trimWhitespaceSuffix: LLamaChatGenerateResponseOptions<Functions>["trimWhitespaceSuffix"];
  repeatPenalty: LLamaChatGenerateResponseOptions<Functions>["repeatPenalty"];
  tokenBias: LLamaChatGenerateResponseOptions<Functions>["tokenBias"];
  evaluationPriority: LLamaChatGenerateResponseOptions<Functions>["evaluationPriority"];
  contextShift: LLamaChatGenerateResponseOptions<Functions>["contextShift"];
  customStopTriggers: LLamaChatGenerateResponseOptions<Functions>["customStopTriggers"];
  lastEvaluationContextWindow: LLamaChatGenerateResponseOptions<Functions>["lastEvaluationContextWindow"];
  grammar: LlamaGrammar;
  functions: Functions | ChatModelFunctions;
  documentFunctionParams: boolean;
};

Type Parameters

Type Parameter	Default type
Functions extends ChatModelFunctions | undefined	undefined

Type declaration

initialUserPrompt?

optional initialUserPrompt: string;

Complete the given user prompt without adding it or the completion to the returned context window.

stopOnAbortSignal?

optional stopOnAbortSignal: boolean;

When a completion already started being generated and then the signal is aborted, the generation will stop and the completion will be returned as is instead of throwing an error.

Defaults to false.

onTextChunk?

optional onTextChunk: LLamaChatGenerateResponseOptions<Functions>["onTextChunk"];

Called as the model generates a completion with the generated text chunk.

Useful for streaming the generated completion as it's being generated.

onToken?

optional onToken: LLamaChatGenerateResponseOptions<Functions>["onToken"];

Called as the model generates a completion with the generated tokens.

Preferably, you'd want to use onTextChunk instead of this.

signal?

optional signal: LLamaChatGenerateResponseOptions<Functions>["signal"];

maxTokens?

optional maxTokens: LLamaChatGenerateResponseOptions<Functions>["maxTokens"];

temperature?

optional temperature: LLamaChatGenerateResponseOptions<Functions>["temperature"];

minP?

optional minP: LLamaChatGenerateResponseOptions<Functions>["minP"];

topK?

optional topK: LLamaChatGenerateResponseOptions<Functions>["topK"];

topP?

optional topP: LLamaChatGenerateResponseOptions<Functions>["topP"];

seed?

optional seed: LLamaChatGenerateResponseOptions<Functions>["seed"];

trimWhitespaceSuffix?

optional trimWhitespaceSuffix: LLamaChatGenerateResponseOptions<Functions>["trimWhitespaceSuffix"];

repeatPenalty?

optional repeatPenalty: LLamaChatGenerateResponseOptions<Functions>["repeatPenalty"];

tokenBias?

optional tokenBias: LLamaChatGenerateResponseOptions<Functions>["tokenBias"];

evaluationPriority?

optional evaluationPriority: LLamaChatGenerateResponseOptions<Functions>["evaluationPriority"];

contextShift?

optional contextShift: LLamaChatGenerateResponseOptions<Functions>["contextShift"];

customStopTriggers?

optional customStopTriggers: LLamaChatGenerateResponseOptions<Functions>["customStopTriggers"];

lastEvaluationContextWindow?

optional lastEvaluationContextWindow: LLamaChatGenerateResponseOptions<Functions>["lastEvaluationContextWindow"];

grammar?

optional grammar: LlamaGrammar;

functions?

optional functions: Functions | ChatModelFunctions;

Functions are not used by the model here, but are used for keeping the instructions given to the model about the functions in the current context state, to avoid context shifts.

It's best to provide the same functions that were used for the previous prompt here.

documentFunctionParams?

optional documentFunctionParams: boolean;

Functions are not used by the model here, but are used for keeping the instructions given to the model about the functions in the current context state, to avoid context shifts.

It's best to provide the same value that was used for the previous prompt here.

Defined in

evaluator/LlamaChat/LlamaChat.ts:187