node-llama-cpp

Appearance

Type Alias: LlamaChatSessionContextShiftOptions ​

ts
type LlamaChatSessionContextShiftOptions = {
  size: LLamaChatContextShiftOptions["size"];
  strategy: LLamaChatContextShiftOptions["strategy"];
};

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:51

Properties ​

size? ​

ts
optional size: LLamaChatContextShiftOptions["size"];

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:56

The number of tokens to delete from the context window to make space for new ones. Defaults to 10% of the context size.

strategy? ​

ts
optional strategy: LLamaChatContextShiftOptions["strategy"];

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:63

The strategy to use when deleting tokens from the context window.

Defaults to "eraseFirstResponseAndKeepFirstSystem".