node-llama-cpp

Appearance

Type Alias: ControlledEvaluateIndexOutput ​

ts
type ControlledEvaluateIndexOutput = {
  next: {
     token: Token | null;
     confidence: number;
     probabilities: Map<Token, number>;
    };
};

Defined in: evaluator/LlamaContext/types.ts:444

Type declaration ​

next ​

ts
next: {
  token: Token | null;
  confidence: number;
  probabilities: Map<Token, number>;
};

next.token? ​

ts
optional token: Token | null;

next.confidence? ​

ts
optional confidence: number;

The confidence (probability) of the selected token (the token field in this object).

Same as next.probabilities.get(next.token).

If you need only this value, you can skip getting the full probabilities list to improve performance.

This value might be slightly different when evaluated on different GPUs and configurations.

next.probabilities? ​

ts
optional probabilities: Map<Token, number>;

The probabilities of the tokens from the vocabulary to be the next token.

A probability is a number from 0 to 1.

The probabilities might be slightly different when evaluated on different GPUs and configurations.

The map is sorted by the probability of the tokens from the highest to the lowest, and is reflected in the order of the entries when iterating over the map. Use .entries().next().value to get the top probability pair (learn more).