Skip to content

vllm_omni.worker.sparse_audio

Sparse-audio output protocol: marker classification and payload routing.

A step's per-request multimodal_outputs lists may cover only a SUBSET of the batch (requests that produced audio this step). The producer declares this with two metadata keys (nested meta dict or flattened meta.*):

  • meta.req_id — the request ids that DO have output, in list order;
  • meta.sparse_audio — the marker saying "per-request lists are aligned to meta.req_id order, not batch order".

The marker's carrier form is ENFORCED, not normalized. The design contract is a list of strings (e.g. ["1"] — what every in-tree producer emits); a bare string is also accepted. Interpreting deviant carriers would need a second truth predicate, and predicate divergence is exactly how np.array(["0"]) came to route as sparse while ["0"] routed as dense; per the refactor series' Policy A the runner<->model interface owes no tolerance shim.

A present-but-invalid declaration (illegal marker carrier, or marker set with an unusable meta.req_id) FAILS CLOSED: it most likely belongs to a producer that INTENDED sparse output, and falling back to dense would batch-index-assign its payloads to the WRONG requests. InvalidSparseDeclarationError is raised by the classifiers and caught INSIDE this module by resolve_sparse_mm_routing, which routes no payload for the affected step — the exception never crosses the module boundary, so it can never escape a v1 runner (which does not catch per-request exceptions).

This module is the single home of the contract until the model-state refactor series (G9-C2.6) turns the marker into a typed bool resolved at one adapter site and unifies the nested/flattened encodings. The NPU runner still carries a tolerant pre-enforcement copy — NPU-track handoff item.

logger module-attribute 

logger = init_logger(__name__)

InvalidSparseDeclarationError

Bases: ValueError

Sparse-audio metadata is PRESENT but out of contract.

Internal control signal: raised by is_sparse_audio_marker / resolve_sparse_mm_req_ids and caught by resolve_sparse_mm_routing in this same module, which fails closed. Callers outside this module only ever see the routing result, never the exception.

is_sparse_audio_marker 

is_sparse_audio_marker(value: Any) -> bool

Classify a meta.sparse_audio marker.

Legal carriers (list of strings, bare string) evaluate against the literal truthy set; None means "marker absent" and returns False. An ILLEGAL carrier (tensors, arrays, numbers, lists with non-string items) is a producer bug: logged at error level once per offending type, then raised as InvalidSparseDeclarationError. Deviant carriers are never read element-wise, so a device tensor can never introduce a D2H sync here.

resolve_sparse_mm_req_ids 

resolve_sparse_mm_req_ids(
    multimodal_outputs: Any,
) -> list[str] | None

Resolve the sparse request-id list.

Returns the request-id list when a legal sparse declaration is present, and None when there is no sparse declaration (dense). Raises InvalidSparseDeclarationError when the declaration is present but out of contract (illegal marker carrier, or marker set with an unusable meta.req_id).

resolve_sparse_mm_routing 

resolve_sparse_mm_routing(
    *,
    engine_output_type: str,
    req_ids_output_copy: list[str],
    downstream_req_ids: list[str],
    multimodal_outputs: Any,
) -> tuple[list[str], dict[str, int], bool]

Resolve (downstream_req_ids, sparse_mm_index, audio_sparse_output).

The only entry point runners use; all protocol errors are absorbed here.