Skip to content

Class: LlamaChatSession

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:349

See

Using LlamaChatSession tutorial

Constructors

Constructor

ts
new LlamaChatSession(options: LlamaChatSessionOptions): LlamaChatSession;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:365

Parameters

ParameterType
optionsLlamaChatSessionOptions

Returns

LlamaChatSession

Properties

onDispose

ts
readonly onDispose: EventRelay<void>;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:363

Accessors

disposed

Get Signature

ts
get disposed(): boolean;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:422

Returns

boolean


chatWrapper

Get Signature

ts
get chatWrapper(): ChatWrapper;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:426

Returns

ChatWrapper


sequence

Get Signature

ts
get sequence(): LlamaContextSequence;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:433

Returns

LlamaContextSequence


context

Get Signature

ts
get context(): LlamaContext;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:440

Returns

LlamaContext


model

Get Signature

ts
get model(): LlamaModel;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:444

Returns

LlamaModel

Methods

dispose()

ts
dispose(__namedParameters: {
  disposeSequence?: boolean;
}): void;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:407

Parameters

ParameterType
__namedParameters{ disposeSequence?: boolean; }
__namedParameters.disposeSequence?boolean

Returns

void


prompt()

ts
prompt<Functions>(prompt: string, options: LLamaChatPromptOptions<Functions>): Promise<string>;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:448

Type Parameters

Type ParameterDefault type
Functions extends | undefined | ChatSessionModelFunctionsundefined

Parameters

ParameterType
promptstring
optionsLLamaChatPromptOptions<Functions>

Returns

Promise<string>


promptWithMeta()

ts
promptWithMeta<Functions>(prompt: string, options?: LLamaChatPromptOptions<Functions>): Promise<
  | {
  response: (
     | string
     | ChatModelFunctionCall
    | ChatModelSegment)[];
  responseText: string;
  stopReason: "customStopTrigger";
  customStopTrigger: (string | Token)[];
  remainingGenerationAfterStop: undefined | string | Token[];
}
  | {
  customStopTrigger?: undefined;
  response: (
     | string
     | ChatModelFunctionCall
    | ChatModelSegment)[];
  responseText: string;
  stopReason:   | "abort"
     | "maxTokens"
     | "eogToken"
     | "stopGenerationTrigger"
     | "functionCalls";
  remainingGenerationAfterStop: undefined | string | Token[];
}>;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:497

Type Parameters

Type ParameterDefault type
Functions extends | undefined | ChatSessionModelFunctionsundefined

Parameters

ParameterTypeDescription
promptstring
options?LLamaChatPromptOptions<Functions>

Returns

Promise< | { response: ( | string | ChatModelFunctionCall | ChatModelSegment)[]; responseText: string; stopReason: "customStopTrigger"; customStopTrigger: (string | Token)[]; remainingGenerationAfterStop: undefined | string | Token[]; } | { customStopTrigger?: undefined; response: ( | string | ChatModelFunctionCall | ChatModelSegment)[]; responseText: string; stopReason: | "abort" | "maxTokens" | "eogToken" | "stopGenerationTrigger" | "functionCalls"; remainingGenerationAfterStop: undefined | string | Token[]; }>


preloadPrompt()

ts
preloadPrompt(prompt: string, options?: LLamaChatPreloadPromptOptions): Promise<void>;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:786

Preload a user prompt into the current context sequence state to make later inference of the model response begin sooner and feel faster.

Note: Preloading a long user prompt can incur context shifts, so consider limiting the length of prompts you preload

Parameters

ParameterTypeDescription
promptstringthe prompt to preload
options?LLamaChatPreloadPromptOptions

Returns

Promise<void>


completePrompt()

ts
completePrompt(prompt: string, options?: LLamaChatCompletePromptOptions): Promise<string>;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:803

Preload a user prompt into the current context sequence state and generate a completion for it.

Note: Preloading a long user prompt and completing a user prompt with a high number of maxTokens can incur context shifts, so consider limiting the length of prompts you preload.

Also, it's recommended to limit the number of tokens generated to a reasonable amount by configuring maxTokens.

Parameters

ParameterTypeDescription
promptstringthe prompt to preload
options?LLamaChatCompletePromptOptions

Returns

Promise<string>


createPromptCompletionEngine()

ts
createPromptCompletionEngine(options?: LLamaChatPromptCompletionEngineOptions): LlamaChatSessionPromptCompletionEngine;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:816

Create a smart completion engine that caches the prompt completions and reuses them when the user prompt matches the beginning of the cached prompt or completion.

All completions are made and cache is used only for the current chat session state. You can create a single completion engine for an entire chat session.

Parameters

ParameterType
options?LLamaChatPromptCompletionEngineOptions

Returns

LlamaChatSessionPromptCompletionEngine


completePromptWithMeta()

ts
completePromptWithMeta(prompt: string, options?: LLamaChatCompletePromptOptions): Promise<
  | {
  completion: string;
  stopReason: "customStopTrigger";
  customStopTrigger: (string | Token)[];
  remainingGenerationAfterStop: undefined | string | Token[];
}
  | {
  customStopTrigger?: undefined;
  completion: string;
  stopReason: "abort" | "maxTokens" | "eogToken" | "stopGenerationTrigger";
  remainingGenerationAfterStop: undefined | string | Token[];
}>;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:825

See completePrompt for more information.

Parameters

ParameterTypeDescription
promptstring
options?LLamaChatCompletePromptOptions

Returns

Promise< | { completion: string; stopReason: "customStopTrigger"; customStopTrigger: (string | Token)[]; remainingGenerationAfterStop: undefined | string | Token[]; } | { customStopTrigger?: undefined; completion: string; stopReason: "abort" | "maxTokens" | "eogToken" | "stopGenerationTrigger"; remainingGenerationAfterStop: undefined | string | Token[]; }>


getChatHistory()

ts
getChatHistory(): ChatHistoryItem[];

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:929

Returns

ChatHistoryItem[]


getLastEvaluationContextWindow()

ts
getLastEvaluationContextWindow(): null | ChatHistoryItem[];

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:933

Returns

null | ChatHistoryItem[]


setChatHistory()

ts
setChatHistory(chatHistory: ChatHistoryItem[]): void;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:940

Parameters

ParameterType
chatHistoryChatHistoryItem[]

Returns

void


resetChatHistory()

ts
resetChatHistory(): void;

Defined in: evaluator/LlamaChatSession/LlamaChatSession.ts:948

Clear the chat history and reset it to the initial state.

Returns

void