Skip to content

Type Alias: LlamaInfillGenerationOptions

ts
type LlamaInfillGenerationOptions = LlamaCompletionGenerationOptions & {
  minPrefixKeepTokens:   | number
     | (sequence: LlamaContextSequence) => 
     | number
     | Promise<number>;
};

Defined in: evaluator/LlamaCompletion.ts:151

Type declaration

minPrefixKeepTokens?

ts
optional minPrefixKeepTokens: 
  | number
  | (sequence: LlamaContextSequence) => 
  | number
  | Promise<number>;

The minimum number of tokens to keep from the prefix input when making a context shift. Defaults to 10% of the context size.