For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
shard_and_stack
Implements shard-and-stack: shards a tensor across devices and stacks the shards into a higher-rank output.
comptime values
ShardStackInput
comptime ShardStackInput[dtype: DType, rank: Int] = TileTensor[dtype, Layout[TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)](), TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)]()], ImmUntrackedOrigin]
An input of shard_and_stack: a host tensor with a dynamic layout.
Parameters
ShardStackOutput
comptime ShardStackOutput[dtype: DType, rank: Int] = TileTensor[dtype, Layout[TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)](), TypeList[#kgen.param_list.tabulate(rank, [idx: __mlir_type.index] Int64)]()], MutUntrackedOrigin]
An output of shard_and_stack: one device's stack of input shards.
Parameters
Functions
-
shard_and_stack: Shard weight tensors across multiple devices for tensor parallelism.