Type Alias: BatchingOptions

type BatchingOptions = {
  dispatchSchedule:   | "nextCycle"
     | CustomBatchingDispatchSchedule;
  itemPrioritizationStrategy:   | "maximumParallelism"
     | "firstInFirstOut"
     | CustomBatchingPrioritizationStrategy;
};

Defined in: evaluator/LlamaContext/types.ts:222

Type declaration

dispatchSchedule?

optional dispatchSchedule: 
  | "nextCycle"
  | CustomBatchingDispatchSchedule;

The strategy used to dispatch items to be processed when there are items pending to be processed.

• "nextCycle" - dispatch the items on the next event loop cycle. You can provide a custom function to define a custom dispatch schedule.

Defaults to "nextCycle".

itemPrioritizationStrategy?

optional itemPrioritizationStrategy: 
  | "maximumParallelism"
  | "firstInFirstOut"
  | CustomBatchingPrioritizationStrategy;

The strategy used to prioritize pending items to be processed.

• "maximumParallelism" - process as many different sequences in parallel as possible.
• "firstInFirstOut" - process items in the order they were added.
• Custom prioritization function - a custom function that prioritizes the items to be processed. See the CustomBatchingPrioritizationStrategy type for more information.

Defaults to "maximumParallelism".