For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

RealRowScales

struct RealRowScales

Row scales read from an array of one bfloat16 per output row.

Fields

• ​ptr (Pointer[BFloat16, ImmutAnyOrigin]): Device pointer to the per-row scales, indexed like the output rows.

Implemented traits

AnyType, Copyable, Deinitable, DevicePassable, ImplicitlyCopyable, Movable, RegisterPassable, RowScales, TrivialRegisterPassable

comptime members

device_type

comptime device_type = RealRowScales

Enabled

comptime Enabled = True

Methods

__init__

def __init__(ptr: Pointer[BFloat16, ImmutAnyOrigin]) -> Self

Wraps a device pointer to the per-row scales.

Args:

• ​ptr (Pointer[BFloat16, ImmutAnyOrigin]): Device pointer to one scale per output row.

load

def load(self, row: Int) -> Float32

Reads the scale of one output row.

Args:

• ​row (Int): Absolute output row (token) index.

Returns:

Float32: The row's scale, widened to float32.

get_type_name

static def get_type_name() -> String

Returns the type name.

Returns:

String: Always "RealRowScales".