Type Alias: SequenceEvaluateMetadataOptions

type SequenceEvaluateMetadataOptions = {
  confidence?: boolean;
  probabilities?: boolean;
};

Defined in: evaluator/LlamaContext/types.ts:351

Properties

confidence?

readonly optional confidence: boolean;

Defined in: evaluator/LlamaContext/types.ts:361

Get the confidence (probability) of the selected token.

Same as probabilities.get(token) from the output.

If you need only this value, you can skip getting the full probabilities list to improve performance.

This value might be slightly different when evaluated on different GPUs and configurations.

---

probabilities?

readonly optional probabilities: boolean;

Defined in: evaluator/LlamaContext/types.ts:370

Get the full probabilities list of tokens from the vocabulary to be the next token, after applying the given options.

Only enable when needed, as it impacts the performance.

Defaults to false.