IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

shard_and_stack

Implements shard-and-stack: shards a tensor across devices and stacks the shards into a higher-rank output.

comptime values​

ShardStackInput​

comptime ShardStackInput[dtype: DType, rank: Int] = TileTensor[dtype, Layout[TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)](), TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)]()], ImmUntrackedOrigin]

An input of shard_and_stack: a host tensor with a dynamic layout.

Parameters​

ShardStackOutput​

comptime ShardStackOutput[dtype: DType, rank: Int] = TileTensor[dtype, Layout[TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)](), TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)]()], MutUntrackedOrigin]

An output of shard_and_stack: one device's stack of input shards.

Parameters​

Functions​

  • ​shard_and_stack: Shard weight tensors across multiple devices for tensor parallelism.

Was this page helpful?