Type Alias: LlamaInfillGenerationOptions
ts
type LlamaInfillGenerationOptions = LlamaCompletionGenerationOptions & {
minPrefixKeepTokens: | number
| (sequence: LlamaContextSequence) =>
| number
| Promise<number>;
};
Defined in: evaluator/LlamaCompletion.ts:151
Type declaration
minPrefixKeepTokens?
ts
optional minPrefixKeepTokens:
| number
| (sequence: LlamaContextSequence) =>
| number
| Promise<number>;
The minimum number of tokens to keep from the prefix input when making a context shift. Defaults to 10% of the context size.