node-llama-cpp

Appearance

Type Alias: SequenceEvaluateMetadataOptions ​

ts
type SequenceEvaluateMetadataOptions = {
  confidence: boolean;
  probabilities: boolean;
};

Defined in: evaluator/LlamaContext/types.ts:328

Properties ​

confidence? ​

ts
readonly optional confidence: boolean;

Defined in: evaluator/LlamaContext/types.ts:338

Get the confidence (probability) of the selected token.

Same as probabilities.get(token) from the output.

If you need only this value, you can skip getting the full probabilities list to improve performance.

This value might be slightly different when evaluated on different GPUs and configurations.

probabilities? ​

ts
readonly optional probabilities: boolean;

Defined in: evaluator/LlamaContext/types.ts:347

Get the full probabilities list of tokens from the vocabulary to be the next token, after applying the given options.

Only enable when needed, as it impacts the performance.

Defaults to false.