For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
AxisAssignment
AxisAssignment
class max.experimental.sharding.AxisAssignment(needed_inputs, outputs)
Bases: NamedTuple
One placement choice that a sharding rule offers for its op.
A sharding rule is a function with the same parameters as its op. It
receives the op’s arguments with each tensor, at any depth, replaced by
its TensorLayout, and returns a list of rows. Each row gives a
placement for every tensor operand, in argument order, and for every
result. Some rows of the matmul rule:
from max.experimental.sharding import (
AxisAssignment,
Partial,
Replicated,
Sharded,
TensorLayout,
)
def matmul_rule(
lhs: TensorLayout, rhs: TensorLayout
) -> list[AxisAssignment]:
return [
AxisAssignment((Replicated(), Replicated()), (Replicated(),)),
# Column parallel: each device holds some of the columns.
AxisAssignment((Replicated(), Sharded(1)), (Sharded(1),)),
# Row parallel: each device holds part of the sum.
AxisAssignment((Sharded(1), Sharded(0)), (Partial(),)),
]On each mesh axis, the op picks the row whose input moves cost least, moves its operands there, and gives its results the row’s placements. A rule never changes the op’s other arguments.
Create new instance of AxisAssignment(needed_inputs, outputs)
needed_inputs
The placement of each tensor operand, in argument order.
outputs
The placement of each result, in result order.