IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

use_chunked_ssd

def use_chunked_ssd(total_len: Int, batch: Int, head_dim: Int, min_tokens: Int) -> Bool

Returns whether a launch should take the chunked scan over the serial one.

The chunk setup costs a fixed ~0.2 ms, so the launch needs at least min_tokens tokens in total and an average sequence of at least MIN_AVG_SEQ_LEN tokens: a batch of short verify or decode rows stays on the serial kernel. head_dim must be a multiple of TILE.

Returns:

Bool

Was this page helpful?