Class: LlamaDecisionContext

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:115

See

Using Structured Decisions tutorial

Properties

onDispose

readonly onDispose: EventRelay<void>;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:123

Accessors

disposed

Get Signature

get disposed(): boolean;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:167

Returns

boolean

---

model

Get Signature

get model(): LlamaModel;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:171

Returns

LlamaModel

---

contextSize

Get Signature

get contextSize(): number;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:175

Returns

number

---

batchSize

Get Signature

get batchSize(): number;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:179

Returns

number

---

flashAttention

Get Signature

get flashAttention(): boolean | "auto";

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:183

Returns

boolean | "auto"

---

memoryUsage

Get Signature

get memoryUsage(): {
  ram: number;
  vram: number;
};

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:188

Assumed memory footprint of the context in bytes

Returns

{
  ram: number;
  vram: number;
}

ram

ram: number;

vram

vram: number;

Methods

dispose()

dispose(): Promise<void>;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:153

Returns

Promise<void>

---

warmup()

warmup(__namedParameters?: {
  signal?: AbortSignal;
}): Promise<void>;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:193

Warmup the model, so that the next evaluation is faster

Parameters

Parameter	Type
__namedParameters	{ signal?: AbortSignal; }
__namedParameters.signal?	AbortSignal

Returns

Promise<void>

---

decide()

decide<Questions>(
   document: string, 
   questions: Questions, 
   options?: LlamaDecisionContextDecideOptions): Promise<DecisionAnswers<Questions>>;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:211

Type Parameters

Type Parameter
Questions extends DecisionQuestions

Parameters

Parameter	Type
document	string
questions	Questions
options	LlamaDecisionContextDecideOptions

Returns

Promise<DecisionAnswers<Questions>>

---

decideWithMeta()

decideWithMeta<Questions>(
   document: string, 
   questions: Questions, 
   options?: LlamaDecisionContextDecideOptions): Promise<LlamaDecisionContextDecideResponse<Questions>>;

Defined in: evaluator/LlamaDecisionContext/LlamaDecisionContext.ts:219

Type Parameters

Type Parameter
Questions extends DecisionQuestions

Parameters

Parameter	Type
document	string
questions	Questions
options	LlamaDecisionContextDecideOptions

Returns

Promise<LlamaDecisionContextDecideResponse<Questions>>