node-llama-cpp

Appearance

Type Alias: ControlledEvaluateIndexOutput ​

ts
type ControlledEvaluateIndexOutput = {
  next: {
     token?: Token | null;
     confidence?: number;
     probabilities?: Map<Token, number>;
     logits?: Map<Token, number>;
     totalLogitWeight?: number;
  };
};

Defined in: evaluator/LlamaContext/types.ts:675

Properties ​

next ​

ts
next: {
  token?: Token | null;
  confidence?: number;
  probabilities?: Map<Token, number>;
  logits?: Map<Token, number>;
  totalLogitWeight?: number;
};

Defined in: evaluator/LlamaContext/types.ts:676

token? ​

ts
optional token: Token | null;

confidence? ​

ts
optional confidence: number;

The confidence (probability) of the selected token (the token field in this object).

Same as next.probabilities.get(next.token).

If you need only this value, you can skip getting the full probabilities list to improve performance.

This value might be slightly different when evaluated on different GPUs and configurations.

probabilities? ​

ts
optional probabilities: Map<Token, number>;

The probabilities of the tokens from the vocabulary to be the next token.

A probability is a number from 0 to 1.

The probabilities might be slightly different when evaluated on different GPUs and configurations.

The map is sorted by the probability of the tokens from the highest to the lowest, and is reflected in the order of the entries when iterating over the map. Use .entries().next().value to get the top probability pair (learn more).

logits? ​

ts
optional logits: Map<Token, number>;

The raw logits of the tokens from the vocabulary to be the next token.

A logit is a number representing the unnormalized score for a token.

The logits might be slightly different when evaluated on different GPUs and configurations.

The map is sorted by the logit of the tokens from the highest to the lowest, and is reflected in the order of the entries when iterating over the map. Use .entries().next().value to get the top logit pair (learn more).

totalLogitWeight? ​

ts
optional totalLogitWeight: number;

The total logit weight of the tokens from the vocabulary to be the next token, after applying the given options.

Equals to the sum of exp(logit - maxLogit).