IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

DistributedInputs

DistributedInputs​

class max.pipelines.speculative.driver.DistributedInputs(host_input_row_offsets, host_merged_offsets, host_query_offsets, data_parallel_splits)

source

Bases: Generic[_T]

The CPU-side inputs only a distributed=True signature declares.

They arrive together or not at all, and a model that reads one reads all of them: the host offset mirrors let a sharded model size its collectives without waiting on the device, and the splits say which rows each replica owns. Grouping them is what lets a single-device graph – the unified EAGLE and DFlash ones – run the same driver while declaring none.

Parameters:

  • host_input_row_offsets (_T)
  • host_merged_offsets (_T)
  • host_query_offsets (_T)
  • data_parallel_splits (_T)

data_parallel_splits​

data_parallel_splits: _T

source

Per-replica batch boundaries.

host_input_row_offsets​

host_input_row_offsets: _T

source

host_merged_offsets​

host_merged_offsets: _T

source

CPU mirror of SequentialBatch.merged_offsets.

host_query_offsets​

host_query_offsets: _T

source

CPU mirror of SequentialBatch.query_offsets_per_dev.