node-llama-cpp

Appearance

Type Alias: LlamaChatGenerateDecisionsOptions ​

ts
type LlamaChatGenerateDecisionsOptions = {
  document?: string;
  signal?: AbortSignal;
  evaluationPriority?: EvaluationPriority;
  contextShift?: LLamaChatContextShiftOptions;
  lastEvaluationContextWindow?: {
     history?: ChatHistoryItem[];
  };
  functions?: ChatModelFunctions;
  documentFunctionParams?: boolean;
};

Defined in: evaluator/LlamaChat/LlamaChat.ts:476

Properties ​

document? ​

ts
optional document: string;

Defined in: evaluator/LlamaChat/LlamaChat.ts:486

An optional document to add to the context window before a question. Only added for generating decisions in this call; won't be appended to the actual context window chat history returned from this evaluation.

Will be put in the same user message as the question, with "\n\n" in between the document and the question.

Note that a long document can incur a context shift, so make sure to not use a too big document.

signal? ​

ts
optional signal: AbortSignal;

Defined in: evaluator/LlamaChat/LlamaChat.ts:488

evaluationPriority? ​

ts
optional evaluationPriority: EvaluationPriority;

Defined in: evaluator/LlamaChat/LlamaChat.ts:493

See the parameter evaluationPriority on the LlamaContextSequence.evaluate() function for more information.

contextShift? ​

ts
optional contextShift: LLamaChatContextShiftOptions;

Defined in: evaluator/LlamaChat/LlamaChat.ts:495

lastEvaluationContextWindow? ​

ts
optional lastEvaluationContextWindow: {
  history?: ChatHistoryItem[];
};

Defined in: evaluator/LlamaChat/LlamaChat.ts:501

The evaluation context window returned from the last evaluation. This is an optimization to utilize existing context sequence state better when possible.

history? ​

ts
optional history: ChatHistoryItem[];

The history of the last evaluation.

functions? ​

ts
optional functions: ChatModelFunctions;

Defined in: evaluator/LlamaChat/LlamaChat.ts:513

Functions are not used by the model here, but are used for keeping the instructions given to the model about the functions in the current context state, to avoid context shifts.

It's best to provide the same functions that were used for the previous prompt here.

documentFunctionParams? ​

ts
optional documentFunctionParams: boolean;

Defined in: evaluator/LlamaChat/LlamaChat.ts:522

Functions are not used by the model here, but are used for keeping the instructions given to the model about the functions in the current context state, to avoid context shifts.

It's best to provide the same value that was used for the previous prompt here.