Skip to content

vllm_omni.clients.utils

Client-local helpers for duplex JSON metric blocks.

STREAMING_OUTPUT_UNIT_TYPES module-attribute 

STREAMING_OUTPUT_UNIT_TYPES = frozenset(
    {"text", "stream", "audio"}
)

StageModalityFlags

Bases: NamedTuple

How duplex JSON classifies one engine stage.

is_audio_stage instance-attribute 

is_audio_stage: bool

is_image_stage instance-attribute 

is_image_stage: bool

is_internal_stream_stage instance-attribute 

is_internal_stream_stage: bool

is_text_stage instance-attribute 

is_text_stage: bool

is_video_stage instance-attribute 

is_video_stage: bool

distribution_summary 

distribution_summary(
    values: Sequence[float], *, digits: int = 3
) -> dict[str, float | int] | None

Summarize values as {count, mean, p50, p99} for duplex report fields.

metric_mean 

metric_mean(value: object) -> float | None

Read a scalar mean, or the mean field of a distribution summary.

stage_modality_flags 

stage_modality_flags(
    final_output_type: object, output_unit_type: object
) -> StageModalityFlags

Classify a stage from wire final_output_type / output_unit_type.

summarize_stage_metrics 

summarize_stage_metrics(
    request_metrics: Sequence[Mapping[str, object]],
) -> dict[str, dict[str, object]] | None

Roll per-response engine stage blocks into {count, mean, p50, p99}.

Reads stages when present, otherwise stage0_tokens as stage "0". Zero or missing tpot_ms / tpop_ms values are omitted, matching session tpot_ms.