For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
PackMatrixCols
struct PackMatrixCols[original_mut: Bool, //, dtype: DType, simd_size: Int, column_inner_size: Int, use_vnni: Bool, use_i8mm: Bool, packed_origin: MutOrigin, original_origin: Origin[mut=original_mut], packed_layout: TensorLayout, original_layout: TensorLayout]
Pack columns from a matrix into the mlas packed layout and extract inner vectors of columns into the packed inner dimension, e.g. extracts [X, Y] and packs as [Yo][X][Yi].
Parameters
- original_mut (
Bool): True if the original matrix buffer is mutable (inferred). - dtype (
DType): Element type of the matrix being packed. - simd_size (
Int): SIMD vector width fordtype. - column_inner_size (
Int): Size of the inner dimension along columns in the packed layout; must be a multiple ofsimd_size. - use_vnni (
Bool): True to pack for the VNNI instruction layout. - use_i8mm (
Bool): True to pack for the i8mm instruction layout. - packed_origin (
MutOrigin): Origin of the packed outputTileTensor. - original_origin (
Origin[mut=original_mut]): Origin of the original inputTileTensor. - packed_layout (
TensorLayout): Layout of the packed outputTileTensor. - original_layout (
TensorLayout): Layout of the original inputTileTensor.
Fields
- packed_matrix (
TileTensor[dtype, packed_layout, packed_origin]): - original_matrix (
TileTensor[dtype, original_layout, original_origin]): - global_offset (
IndexList[Int(2)]): - pack_tile_dim (
IndexList[Int(2)]): - valid_data_dim (
IndexList[Int(2)]):
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable
Methods
run
static def run(packed_matrix: TileTensor[dtype, packed_layout, packed_origin], original_matrix: TileTensor[dtype, original_layout, original_origin], global_offset: IndexList[Int(2)], pack_tile_dim: IndexList[Int(2)], valid_data_dim: IndexList[Int(2)])
Interface function to run the packing routine. Args: packed_matrix: Pre-allocated buffer space for packed data. original_matrix: Data buffer containing the original matrix to pack. global_offset: Offset to use when indexing the original matrix. pack_tile_dim: 2D dimension tuple describing the size of the packed tile. valid_data_dim: 2D dimension tuple describing the amount of valid data on the global buffer starting from the offset.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!