For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
CompletedBatchStats
CompletedBatchStats
class max.pipelines.modeling.types.pipeline_variants.CompletedBatchStats(batch_type, batch_size, num_input_tokens, num_context_tokens, execution_time_s, num_output_tokens=None, draft_tokens_generated=0, draft_tokens_accepted=0, avg_acceptance_length=0.0, max_acceptance_length=0, acceptance_rate_per_position=<factory>, early_sync_duration_s=None)
Bases: object
Execution stats for a batch whose outputs have been synchronized.
-
Parameters:
-
- batch_type (BatchType)
- batch_size (int)
- num_input_tokens (int)
- num_context_tokens (int)
- execution_time_s (float)
- num_output_tokens (int | None)
- draft_tokens_generated (int)
- draft_tokens_accepted (int)
- avg_acceptance_length (float)
- max_acceptance_length (int)
- acceptance_rate_per_position (list[float])
- early_sync_duration_s (float | None)
acceptance_rate_per_position
Per-position draft acceptance rates for the completed batch (speculative decoding).
avg_acceptance_length
avg_acceptance_length: float = 0.0
Average acceptance length for the completed batch (speculative decoding).
batch_size
batch_size: int
Number of requests in the completed batch.
batch_type
batch_type: BatchType
Type of the completed batch.
draft_tokens_accepted
draft_tokens_accepted: int = 0
Draft tokens accepted by the completed batch (speculative decoding).
draft_tokens_generated
draft_tokens_generated: int = 0
Draft tokens generated by the completed batch (speculative decoding).
early_sync_duration_s
Wall-clock time spent in the early-sync guard’s blocking
sync_and_process_outputs() call (see
_should_early_sync_prev_batch), when it fired for this batch.
None when the guard did not fire.
execution_time_s
execution_time_s: float
Host executor service interval in seconds. With overlap, completion minus the later of enqueue and the previous successful completion. Includes host observation delays, including other batches’ scheduling and advancement.
generation_throughput
property generation_throughput: float
Generation throughput of the completed batch in tokens/second.
Uses the returned output-token count, including prefill and speculative outputs; falls back to one token per request when unknown.
max_acceptance_length
max_acceptance_length: int = 0
Maximum possible acceptance length, i.e. the configured number of speculative tokens (speculative decoding).
num_context_tokens
num_context_tokens: int
Number of context tokens in the completed batch.
num_input_tokens
num_input_tokens: int
Number of input tokens in the completed batch.
num_output_tokens
Tokens returned by the completed batch before scheduler filtering:
includes prefill outputs and accepted plus bonus speculative outputs.
None supports callers without returned-output accounting.
prompt_throughput
property prompt_throughput: float
Prompt-side throughput of the completed batch in tokens/second.