For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
CompletedBatchStats
CompletedBatchStats
class max.pipelines.modeling.types.CompletedBatchStats(batch_type, batch_size, num_input_tokens, num_context_tokens, execution_time_s, num_output_tokens=None, draft_tokens_generated=0, draft_tokens_accepted=0, avg_acceptance_length=0.0, max_acceptance_length=0, acceptance_rate_per_position=<factory>, early_sync_duration_s=None)
Bases: object
Execution stats for a batch whose outputs have been synchronized.
-
Parameters:
-
- batch_type (BatchType)
- batch_size (int)
- num_input_tokens (int)
- num_context_tokens (int)
- execution_time_s (float)
- num_output_tokens (int | None)
- draft_tokens_generated (int)
- draft_tokens_accepted (int)
- avg_acceptance_length (float)
- max_acceptance_length (int)
- acceptance_rate_per_position (list[float])
- early_sync_duration_s (float | None)
acceptance_rate_per_position
Per-position draft acceptance rates for the completed batch (speculative decoding).
avg_acceptance_length
avg_acceptance_length: float = 0.0
Average acceptance length for the completed batch (speculative decoding).
batch_size
batch_size: int
Number of requests in the completed batch.
batch_type
batch_type: BatchType
Type of the completed batch.
draft_tokens_accepted
draft_tokens_accepted: int = 0
Draft tokens accepted by the completed batch (speculative decoding).
draft_tokens_generated
draft_tokens_generated: int = 0
Draft tokens generated by the completed batch (speculative decoding).
early_sync_duration_s
Wall-clock time spent in the early-sync guard’s blocking
sync_and_process_outputs() call (see
_should_early_sync_prev_batch), when it fired for this batch.
None when the guard did not fire.
execution_time_s
execution_time_s: float
Execution time of the completed batch, in seconds.
generation_throughput
property generation_throughput: float
Generation throughput of the completed batch in tokens/second.
Uses the known output-token count when available (speculative decoding, TG batches); otherwise counts one token per request.
max_acceptance_length
max_acceptance_length: int = 0
Maximum possible acceptance length, i.e. the configured number of speculative tokens (speculative decoding).
num_context_tokens
num_context_tokens: int
Number of context tokens in the completed batch.
num_input_tokens
num_input_tokens: int
Number of input tokens in the completed batch.
num_output_tokens
Output tokens produced by the completed batch, when known (currently
only reported by speculative decoding). None otherwise.
prompt_throughput
property prompt_throughput: float
Prompt-side throughput of the completed batch in tokens/second.