IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

gather_variable_rows

gather_variable_rows()​

max.pipelines.speculative.driver.gather_variable_rows(h, input_row_offsets, return_n_logits, mesh, data_parallel_degree=1)

source

Gathers each request’s last return_n_logits rows of h.

Also returns their offsets, return_n_logits apart, matching the graph-API deepseek_logits_postprocess.

Under data parallelism (data_parallel_degree > 1), h holds each replica’s own rows and input_row_offsets is in each replica’s local frame. Each replica gathers from its own rows, and one allgather joins them in replica order, which is global request order, so only the gathered rows cross devices. The offsets then span the joined batch.

Parameters:

Return type:

tuple[Tensor, Tensor]