node-llama-cpp

Appearance

Type Alias: LlamaChatSessionOptions ​

ts
type LlamaChatSessionOptions = {
  contextSequence: LlamaContextSequence;
  chatWrapper: "auto" | ChatWrapper;
  systemPrompt: string;
  forceAddSystemPrompt: boolean;
  autoDisposeSequence: boolean;
  contextShift: LlamaChatSessionContextShiftOptions;
};

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:23

Properties ​

contextSequence ​

ts
contextSequence: LlamaContextSequence;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:24

chatWrapper? ​

ts
optional chatWrapper: "auto" | ChatWrapper;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:27

"auto" is used by default

systemPrompt? ​

ts
optional systemPrompt: string;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:29

forceAddSystemPrompt? ​

ts
optional forceAddSystemPrompt: boolean;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:39

Add the system prompt even on models that don't support a system prompt.

Each chat wrapper has its own workaround for adding a system prompt to a model that doesn't support it, but forcing the system prompt on unsupported models may not always work as expected.

Use with caution.

autoDisposeSequence? ​

ts
optional autoDisposeSequence: boolean;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:46

Automatically dispose the sequence when the session is disposed.

Defaults to false.

contextShift? ​

ts
optional contextShift: LlamaChatSessionContextShiftOptions;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:48