node-llama-cpp

Appearance

Type Alias: SequenceEvaluateMetadataOptions ​

ts
type SequenceEvaluateMetadataOptions = {
  confidence: boolean;
  probabilities: boolean;
};

Defined in: evaluator/LlamaContext/types.ts:328

Type declaration ​

confidence? ​

ts
readonly optional confidence: boolean;

Get the confidence (probability) of the selected token.

Same as probabilities.get(token) from the output.

If you need only this value, you can skip getting the full probabilities list to improve performance.

This value might be slightly different when evaluated on different GPUs and configurations.

probabilities? ​

ts
readonly optional probabilities: boolean;

Get the full probabilities list of tokens from the vocabulary to be the next token, after applying the given options.

Only enable when needed, as it impacts the performance.

Defaults to false.