For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
gpu_qint4_repack_Q4_0
def gpu_qint4_repack_Q4_0(b_tt: TileTensor[.uint8, Engine=b_tt.Engine, linear_idx_type=b_tt.linear_idx_type], b_packed_tt: TileTensor[.uint8, Engine=b_packed_tt.Engine, linear_idx_type=b_packed_tt.linear_idx_type], ctx: DeviceContext)
Launches the GPU kernel that repacks Q4_0 weights into the packed GEMM layout.
Args:
- โb_tt (
TileTensor[.uint8, Engine=b_tt.Engine, linear_idx_type=b_tt.linear_idx_type]): The input Q4_0 quantized weight tile tensor in global memory. - โb_packed_tt (
TileTensor[.uint8, Engine=b_packed_tt.Engine, linear_idx_type=b_packed_tt.linear_idx_type]): The output repacked weight tile tensor in global memory. - โctx (
DeviceContext): The device context used to enqueue the kernel.
Raises:
An error if the input tensors are not rank-2 or the target is not a GPU.