Type Alias: LlamaInfillGenerationOptions

type LlamaInfillGenerationOptions: LlamaCompletionGenerationOptions & {
  minPrefixKeepTokens: number | (sequence: LlamaContextSequence) => number | Promise<number>;
};

Type declaration

minPrefixKeepTokens?

optional minPrefixKeepTokens: number | (sequence: LlamaContextSequence) => number | Promise<number>;

The minimum number of tokens to keep from the prefix input when making a context shift. Defaults to 10% of the context size.

Defined in

evaluator/LlamaCompletion.ts:150