For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ScratchKVGroupCoordinator

ScratchKVGroupCoordinator

class max.pipelines.kv_cache.paged_kv_cache.scratch_coordinator.ScratchKVGroupCoordinator(pools, leaf_id, group_id, page_size, *, rows=<factory>)

source

Bases: KVGroupCoordinatorInterface

A group holding one never-published block per request.

The block is drawn on the request’s first grow() and freed at release(), and nothing in between moves it: a page boundary neither publishes it nor rotates it, so its id is stable for the request’s whole life. That is what a ring of recent kernel inputs needs, and it is why commit(), advance() and shrink_to_fit() do nothing here.

Carrying no hash keeps the group out of every content-addressed path. It answers no hit, contributes no claimable hash, and asks an external tier for nothing.

Parameters:

• pools (Sequence[JengaBlockPool])
• leaf_id (str)
• group_id (KVCacheGroupId)
• page_size (int)
• rows (dict[RequestID, dict[str, list[LittleKVCacheBlock]]])

advance()

advance(req_id, num_committed_blocks, replica_idx)

source

Frees nothing: the request reads its block until it is released.

Parameters:

• req_id (RequestID)
• num_committed_blocks (int)
• replica_idx (int)

Return type:

None

blocks_held_of_connector_hit()

blocks_held_of_connector_hit(num_hit_blocks)

source

Returns zero: an external tier holds nothing for this group.

Parameters:

num_hit_blocks (int)

Return type:

int

claim_hit_blocks()

claim_hit_blocks(desired_hashes, replica_idx)

source

Returns empty rows: a hit leaves this group nothing to take.

Parameters:

• desired_hashes (Sequence[bytes])
• replica_idx (int)

Return type:

dict[str, list[LittleKVCacheBlock]]

claimable_hashes()

claimable_hashes(desired_hashes)

source

Returns nothing: this group’s block is never published.

Parameters:

desired_hashes (Sequence[bytes])

Return type:

Sequence[bytes]

commit()

commit(req_id, hashes, last_block, replica_idx)

source

Publishes nothing.

Parameters:

• req_id (RequestID)
• hashes (Sequence[bytes])
• last_block (int)
• replica_idx (int)

Return type:

None

forward_blocks()

forward_blocks(batch, num_blocks)

source

Returns the one block each request’s scratch lives in.

Parameters:

• batch (Sequence[TextContext])
• num_blocks (Sequence[int])

Return type:

dict[str, list[Sequence[int]]]

grow()

grow(req_id, num_required_blocks, replica_idx)

source

Draws the request’s one block, if it does not hold it yet.

Parameters:

• req_id (RequestID)
• num_required_blocks (int)
• replica_idx (int)

Return type:

None

grow_with_padding()

grow_with_padding(req_id, num_required_blocks, replica_idx)

source

Points the row at one null block, however long the dummy is.

Parameters:

• req_id (RequestID)
• num_required_blocks (int)
• replica_idx (int)

Return type:

None

shrink_to_fit()

shrink_to_fit(req_id, num_committed_blocks, replica_idx)

source

Keeps the block: there is no committed prefix to trim it to.

Parameters:

• req_id (RequestID)
• num_committed_blocks (int)
• replica_idx (int)

Return type:

None