node-llama-cpp

Appearance

Type Alias: LlamaChatResponseSegmentChunk ​

ts
type LlamaChatResponseSegmentChunk = {
  type: "segment";
  segmentType: ChatModelSegmentType;
  text: string;
  tokens: Token[];
  segmentStartTime: Date;
  segmentEndTime: Date;
};

Defined in: evaluator/LlamaChat/LlamaChat.ts:72

Properties ​

type ​

ts
type: "segment";

Defined in: evaluator/LlamaChat/LlamaChat.ts:73

segmentType ​

ts
segmentType: ChatModelSegmentType;

Defined in: evaluator/LlamaChat/LlamaChat.ts:76

Segment type

text ​

ts
text: string;

Defined in: evaluator/LlamaChat/LlamaChat.ts:86

The generated text chunk.

Detokenized from the tokens property, but with the context of the previous generation (for better spacing of the text with some models).

Prefer using this property over tokens when streaming the generated response as text.

tokens ​

ts
tokens: Token[];

Defined in: evaluator/LlamaChat/LlamaChat.ts:89

The generated tokens

segmentStartTime? ​

ts
optional segmentStartTime: Date;

Defined in: evaluator/LlamaChat/LlamaChat.ts:97

When the current chunk is the start of a segment, this field will be set.

It's possible that a chunk with no tokens and empty text will be emitted just to set this field to signify that the segment has started.

segmentEndTime? ​

ts
optional segmentEndTime: Date;

Defined in: evaluator/LlamaChat/LlamaChat.ts:105

When the current chunk is the last one of a segment (meaning the current segment has ended), this field will be set.

It's possible that a chunk with no tokens and empty text will be emitted just to set this field to signify that the segment has ended.