node-llama-cpp

Appearance

Type Alias: LlamaExperimentalOptions ​

ts
type LlamaExperimentalOptions = {
  metalSkipDisablingResidencySets?: boolean;
};

Defined in: bindings/getLlama.ts:384

Properties ​

metalSkipDisablingResidencySets? ​

ts
optional metalSkipDisablingResidencySets: boolean;

Defined in: bindings/getLlama.ts:394

Experimental

Skip disabling Metal residency sets on macOS when using Metal, which will keep the model data wired (force it to stay in memory) but can negatively affect system performance.

Doing this may cause issues where the system thinks it has less available memory to load a model even after disposing a previous one.

Defaults to false. This is an experimental option that may be removed in the future without a major version bump, so use with caution.