For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
DistributedInputs
DistributedInputs
class max.pipelines.speculative.driver.DistributedInputs(host_input_row_offsets, host_merged_offsets, host_query_offsets, data_parallel_splits)
Bases: Generic[_T]
The CPU-side inputs only a distributed=True signature declares.
They arrive together or not at all, and a model that reads one reads all of them: the host offset mirrors let a sharded model size its collectives without waiting on the device, and the splits say which rows each replica owns. Grouping them is what lets a single-device graph – the unified EAGLE and DFlash ones – run the same driver while declaring none.
-
Parameters:
-
- host_input_row_offsets (_T)
- host_merged_offsets (_T)
- host_query_offsets (_T)
- data_parallel_splits (_T)
data_parallel_splits
data_parallel_splits: _T
Per-replica batch boundaries.
host_input_row_offsets
host_input_row_offsets: _T
host_merged_offsets
host_merged_offsets: _T
CPU mirror of SequentialBatch.merged_offsets.
host_query_offsets
host_query_offsets: _T
CPU mirror of SequentialBatch.query_offsets_per_dev.