node-llama-cpp

Appearance

Type Alias: LlamaChatSessionContextShiftOptions ​

ts
type LlamaChatSessionContextShiftOptions = {
  size?: LLamaChatContextShiftOptions["size"];
  strategy?: LLamaChatContextShiftOptions["strategy"];
};

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:52

Properties ​

size? ​

ts
optional size: LLamaChatContextShiftOptions["size"];

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:57

The number of tokens to delete from the context window to make space for new ones. Defaults to 10% of the context size.

strategy? ​

ts
optional strategy: LLamaChatContextShiftOptions["strategy"];

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:64

The strategy to use when deleting tokens from the context window.

Defaults to "eraseFirstResponseAndKeepFirstSystem".