IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

use_grouped_cb_ssd

def use_grouped_cb_ssd(total_len: Int, batch: Int, nheads: Int, head_dim: Int, ngroups: Int) -> Bool

Returns shape eligibility for amortized grouped score precomputation.

The caller separately gates the architecture, device name and dtypes. With head and state dimensions 128, eight heads per group bound the extra fp32 score scratch to half of the existing chunk-state scratch. Long average rows and sufficient head-token work amortize the additional allocation/launch.

Returns:

Bool

Was this page helpful?