node-llama-cpp

Appearance

Type Alias: JinjaTemplateChatWrapperOptions ​

ts
type JinjaTemplateChatWrapperOptions = {
  template: string;
  modelRoleName: string;
  userRoleName: string;
  systemRoleName: string;
  convertUnsupportedSystemMessagesToUserMessages:   | "auto"
     | boolean
     | JinjaTemplateChatWrapperOptionsConvertMessageFormat;
  functionCallMessageTemplate:   | "auto"
     | "noJinja"
     | ChatHistoryFunctionCallMessageTemplate;
  joinAdjacentMessagesOfTheSameType: boolean;
  trimLeadingWhitespaceInResponses: boolean;
  additionalRenderParameters: Record<string, any>;
  segments: TemplateChatWrapperSegmentsOptions;
  tokenizer: Tokenizer;
};

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:25

Properties ​

template ​

ts
template: string;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:26

modelRoleName? ​

ts
optional modelRoleName: string;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:31

Defaults to "assistant".

userRoleName? ​

ts
optional userRoleName: string;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:36

Defaults to "user".

systemRoleName? ​

ts
optional systemRoleName: string;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:41

Defaults to "system".

convertUnsupportedSystemMessagesToUserMessages? ​

ts
optional convertUnsupportedSystemMessagesToUserMessages: 
  | "auto"
  | boolean
  | JinjaTemplateChatWrapperOptionsConvertMessageFormat;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:58

Some Jinja templates may not support system messages, and in such cases, it'll be detected and system messages can be converted to user messages.

You can specify the format of the converted user message.

  • "auto": Convert system messages to user messages only if the template does not support system messages.
  • true: Always convert system messages to user messages.
  • false: Never convert system messages to user messages. May throw an error if some system messages don't appear in the template.
  • {use: "ifNeeded", format: "..."}: Convert system messages to user messages only if the template does not support system messages with the specified format.
  • {use: "always", format: "..."}: Always convert system messages to user messages with the specified format.

Defaults to "auto".

functionCallMessageTemplate? ​

ts
optional functionCallMessageTemplate: 
  | "auto"
  | "noJinja"
  | ChatHistoryFunctionCallMessageTemplate;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:71

Template format for how functions can be called by the model and how their results are fed to the model after function calls.

  • "auto": Extract the function call message template from the Jinja template. Fallback to the default template if not found.
  • "noJinja": Use the default template.
  • Custom template: Use the specified template. See `ChatHistoryFunctionCallMessageTemplate` for more details.

Defaults to "auto".

joinAdjacentMessagesOfTheSameType? ​

ts
optional joinAdjacentMessagesOfTheSameType: boolean;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:79

Whether to join adjacent messages of the same type. Some Jinja templates may throw an error if this is not set to true.

Defaults to true.

trimLeadingWhitespaceInResponses? ​

ts
optional trimLeadingWhitespaceInResponses: boolean;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:86

Whether to trim leading whitespace in responses.

Defaults to true.

additionalRenderParameters? ​

ts
optional additionalRenderParameters: Record<string, any>;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:91

Additional parameters to use for rendering the Jinja template.

segments? ​

ts
optional segments: TemplateChatWrapperSegmentsOptions;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:96

Format of the segments generated by the model (like thought segments)

tokenizer? ​

ts
optional tokenizer: Tokenizer;

Defined in: chatWrappers/generic/JinjaTemplateChatWrapper.ts:103

Pass a model's tokenizer to attempt to detect common tokens used for chat formatting from it.

Currently only used for detecting support for <think> tags for thought segments.