Skip to content

vllm_omni.protocol.duplex.events

vLLM-Omni's half of the Realtime server-event vocabulary.

docs/serving/realtime_duplex_api.md sorts every message into three tiers. Tier 1 --- identical to OpenAI --- is vllm_omni.protocol.realtime.events. The other two are here:

Tier 2, OpenAI names carrying vLLM-Omni extensions. Same wire type, same OpenAI fields, plus additive keys a stock client ignores. Each one subclasses its Tier 1 twin and declares only what it adds, so the OpenAI surface stays honest: a consumer that wants pure GA takes protocol/realtime and gets exactly OpenAI's fields.

Tier 3, vLLM-Omni only. Events OpenAI has no equivalent for, because they only make sense when the model and the user can talk at the same time: the model decides when to listen and when to speak, the client reports how much audio it played, overlapping speech needs a recorded decision, and a dropped socket can re-attach to a live session.

Same-named classes are deliberate --- duplex.events.SessionCreated is realtime.events.SessionCreated with our extensions. Import whichever surface you mean.

DuplexEvent module-attribute 

DuplexEvent = RealtimeEvent

REALTIME_ERROR_TYPES_BY_CODE module-attribute 

REALTIME_ERROR_TYPES_BY_CODE: dict[str, str] = {
    "bad_event": "invalid_request_error",
    "bad_audio": "invalid_request_error",
    "config_timeout": "invalid_request_error",
    "invalid_json": "invalid_request_error",
    "event_too_large": "invalid_request_error",
    "unknown_event": "invalid_request_error",
    "internal_error": "server_error",
    "runtime_append_failed": "server_error",
    "runtime_append_task_failed": "server_error",
    "runtime_signal_failed": "server_error",
    "runtime_abort_failed": "server_error",
    "runtime_data_plane_stream_failed": "server_error",
    "runtime_data_plane_text_without_audio": "server_error",
    "resource_exhausted": "rate_limit_error",
    "session_exists": "invalid_request_error",
    "session_closed": "invalid_request_error",
    "unknown_session": "invalid_request_error",
    "invalid_duplex_runtime_config": "invalid_request_error",
    "instructions_update_unsupported": "invalid_request_error",
    "persona_update_unsupported": "invalid_request_error",
    "voice_update_unsupported": "invalid_request_error",
    "unsupported_nemotron_duplex_mode": "invalid_request_error",
    "unsupported_native_response_options": "invalid_request_error",
    "runtime_touch_failed": "server_error",
    "engine_error": "server_error",
    "input_backpressure": "rate_limit_error",
    "response_already_active": "invalid_request_error",
    "response_not_active": "invalid_request_error",
    "response_create_without_input": "invalid_request_error",
    "text_only_turn_unsupported": "invalid_request_error",
    "commit_aborted": "server_error",
    "input_audio_buffer_empty": "invalid_request_error",
    "missing_item_id": "invalid_request_error",
    "item_not_found": "invalid_request_error",
    "playback_item_mismatch": "invalid_request_error",
    "playback_item_not_found": "invalid_request_error",
    "playback_ack_too_late": "invalid_request_error",
    "unsupported_audio_format": "invalid_request_error",
    "unsupported_turn_detection": "invalid_request_error",
    "unsupported_ref_audio_path": "invalid_request_error",
    "ref_audio_required": "invalid_request_error",
    "model_update_unsupported": "invalid_request_error",
    "voice_update_after_audio_unsupported": "invalid_request_error",
    "ref_audio_update_unsupported": "invalid_request_error",
    "native_text_append_unsupported": "invalid_request_error",
    "invalid_video_frames": "invalid_request_error",
    "invalid_input_modality": "invalid_request_error",
    "invalid_function_call_output": "invalid_request_error",
    "server_vad_unavailable": "server_error",
}

AudioDelta dataclass

Bases: AudioDelta

Tier 2: OpenAI's AudioDelta plus the chunk's own format/rate and the duplex playback metadata.

format class-attribute instance-attribute 

format: str = 'pcm16'

metadata class-attribute instance-attribute 

metadata: Mapping[str, object] | None = None

optional_wire_fields class-attribute instance-attribute 

optional_wire_fields = frozenset(
    {"sample_rate_hz", "metadata"}
)

sample_rate_hz class-attribute instance-attribute 

sample_rate_hz: int | None = None

AudioDone dataclass

Bases: ResponseScopedEvent

wire_type class-attribute instance-attribute 

wire_type = 'response.output_audio.done'

ContentPartAdded dataclass

Bases: ResponseScopedEvent

part class-attribute instance-attribute 

part: Mapping[str, object] = field(default_factory=dict)

wire_type class-attribute instance-attribute 

wire_type = 'response.content_part.added'

ContentPartDone dataclass

Bases: ContentPartAdded

wire_type class-attribute instance-attribute 

wire_type = 'response.content_part.done'

DuplexRawEvent dataclass

Bases: RealtimeEvent

A session-internal event with no dedicated Realtime type (duplex.<internal type>).

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

internal_type class-attribute instance-attribute 

internal_type: str = ''

type property 

type: str

wire_type class-attribute instance-attribute 

wire_type = 'duplex.raw'

to_realtime 

to_realtime() -> dict[str, object]

ErrorEvent dataclass

Bases: ErrorEvent

Tier 2: OpenAI's ErrorEvent plus vLLM-Omni's error codes and any extra keys merged into the envelope.

error property 

error: dict[str, object]

error_type property 

error_type: str

extra class-attribute instance-attribute 

extra: Mapping[str, object] = field(default_factory=dict)

FunctionCallArgumentsDelta dataclass

Bases: RealtimeEvent

call_id class-attribute instance-attribute 

call_id: str = ''

delta class-attribute instance-attribute 

delta: str = ''

item_id class-attribute instance-attribute 

item_id: str | None = None

output_index class-attribute instance-attribute 

output_index: int = 0

response_id class-attribute instance-attribute 

response_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'response.function_call_arguments.delta'

FunctionCallArgumentsDone dataclass

Bases: RealtimeEvent

arguments class-attribute instance-attribute 

arguments: str = ''

call_id class-attribute instance-attribute 

call_id: str = ''

item_id class-attribute instance-attribute 

item_id: str | None = None

output_index class-attribute instance-attribute 

output_index: int = 0

response_id class-attribute instance-attribute 

response_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'response.function_call_arguments.done'

InputCleared dataclass

Bases: RealtimeEvent

wire_type class-attribute instance-attribute 

wire_type = 'input_audio_buffer.cleared'

InputCommitted dataclass

Bases: InputCommitted

Tier 2: OpenAI's InputCommitted plus the native input.committed event echoed under event.

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

InputTranscriptionCompleted dataclass

Bases: RealtimeEvent

content_index class-attribute instance-attribute 

content_index: int = 0

item_id class-attribute instance-attribute 

item_id: str | None = None

text property 

text: str | None

transcript class-attribute instance-attribute 

transcript: str = ''

wire_type class-attribute instance-attribute 

wire_type = (
    "conversation.item.input_audio_transcription.completed"
)

ItemAdded dataclass

Bases: _ItemEvent

wire_type class-attribute instance-attribute 

wire_type = 'conversation.item.added'

ItemCreated dataclass

Bases: _ItemEvent

wire_type class-attribute instance-attribute 

wire_type = 'conversation.item.created'

ItemDeleted dataclass

Bases: ItemDeleted

Tier 2: OpenAI's ItemDeleted plus the originating event echoed under event.

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

ItemDone dataclass

Bases: _ItemEvent

wire_type class-attribute instance-attribute 

wire_type = 'conversation.item.done'

ItemRetrieved dataclass

Bases: RealtimeEvent

item class-attribute instance-attribute 

item: Mapping[str, object] = field(default_factory=dict)

item_id property 

item_id: str | None

wire_type class-attribute instance-attribute 

wire_type = 'conversation.item.retrieved'

ItemTruncated dataclass

Bases: ItemTruncated

Tier 2: OpenAI's ItemTruncated plus the originating event echoed under event.

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

Listen dataclass

Bases: RealtimeEvent

The model decided to keep listening (no spoken response for this turn).

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

optional_wire_fields class-attribute instance-attribute 

optional_wire_fields = frozenset({'response_id'})

response_id class-attribute instance-attribute 

response_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'response.listen'

OutputAudioCleared dataclass

Bases: RealtimeEvent

response_id class-attribute instance-attribute 

response_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'output_audio_buffer.cleared'

OutputItemAdded dataclass

Bases: RealtimeEvent

item class-attribute instance-attribute 

item: Mapping[str, object] = field(default_factory=dict)

item_id property 

item_id: str | None

output_index class-attribute instance-attribute 

output_index: int = 0

response_id class-attribute instance-attribute 

response_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'response.output_item.added'

OutputItemDone dataclass

Bases: OutputItemAdded

wire_type class-attribute instance-attribute 

wire_type = 'response.output_item.done'

OverlapDecision dataclass

Bases: RealtimeEvent

action class-attribute instance-attribute 

action: str | None = None

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

policy class-attribute instance-attribute 

policy: str | None = None

reason class-attribute instance-attribute 

reason: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'overlap.decision'

PlaybackAcknowledged dataclass

Bases: RealtimeEvent

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

wire_type class-attribute instance-attribute 

wire_type = 'playback.acknowledged'

RateLimitsUpdated dataclass

Bases: RealtimeEvent

rate_limits class-attribute instance-attribute 

rate_limits: tuple[Mapping[str, object], ...] = ()

wire_type class-attribute instance-attribute 

wire_type = 'rate_limits.updated'

RealtimeEvent dataclass

Base of every public session event.

audio property 

audio: bytes | None

epoch class-attribute instance-attribute 

epoch: int | None = None

event_id class-attribute instance-attribute 

event_id: str = field(default_factory=new_event_id)

is_terminal property 

is_terminal: bool

item_id property 

item_id: str | None

optional_wire_fields class-attribute 

optional_wire_fields: frozenset[str] = frozenset()

response_id property 

response_id: str | None

session_id class-attribute instance-attribute 

session_id: str = ''

text property 

text: str | None

type property 

type: str

wire_type class-attribute 

wire_type: str = ''

to_realtime 

to_realtime() -> dict[str, object]

The Realtime wire JSON object for this event (derived from the fields).

ResponseCreated dataclass

Bases: ResponseCreated

Tier 2: OpenAI's ResponseCreated plus a top-level response_id.

OpenAI carries the id nested as response.id; the duplex lane also reports it at the top level, where every other response-scoped event already has one.

response_id class-attribute instance-attribute 

response_id: str | None = None

ResponseDone dataclass

Bases: ResponseDone

Tier 2: OpenAI's ResponseDone plus a top-level response_id.

OpenAI carries the id nested as response.id; the duplex lane also reports it at the top level, where every other response-scoped event already has one.

response_id class-attribute instance-attribute 

response_id: str | None = None

ResponseScopedEvent dataclass

Bases: RealtimeEvent

Events addressed to one response (and usually one output item / content part).

content_index class-attribute instance-attribute 

content_index: int = 0

item_id class-attribute instance-attribute 

item_id: str | None = None

output_index class-attribute instance-attribute 

output_index: int = 0

response_id class-attribute instance-attribute 

response_id: str | None = None

SessionClosed dataclass

Bases: RealtimeEvent

details class-attribute instance-attribute 

details: Mapping[str, object] = field(default_factory=dict)

is_terminal property 

is_terminal: bool

reason class-attribute instance-attribute 

reason: str = 'closed'

wire_type class-attribute instance-attribute 

wire_type = 'session.closed'

SessionCreated dataclass

Bases: SessionCreated

Tier 2: OpenAI's SessionCreated plus the resume credentials a reconnecting client needs.

attachment_generation class-attribute instance-attribute 

attachment_generation: int | None = None

optional_wire_fields class-attribute instance-attribute 

optional_wire_fields = frozenset(
    {"attachment_generation", "resume_token"}
)

resume_token class-attribute instance-attribute 

resume_token: str | None = None

SessionExpired dataclass

Bases: SessionClosed

wire_type class-attribute instance-attribute 

wire_type = 'session.expired'

SessionHeartbeatAck dataclass

Bases: RealtimeEvent

wire_type class-attribute instance-attribute 

wire_type = 'session.heartbeat_ack'

SessionReplaced dataclass

Bases: RealtimeEvent

A newer connection took the session over (sent to the replaced socket).

attachment_generation class-attribute instance-attribute 

attachment_generation: int = 0

wire_type class-attribute instance-attribute 

wire_type = 'session.replaced'

SessionResumed dataclass

Bases: SessionCreated

wire_type class-attribute instance-attribute 

wire_type = 'session.resumed'

SessionResyncRequired dataclass

Bases: RealtimeEvent

The replay journal cannot bridge the gap; the client must start over.

reason class-attribute instance-attribute 

reason: str = 'journal_gap'

wire_type class-attribute instance-attribute 

wire_type = 'session.resync_required'

SessionUpdated dataclass

Bases: RealtimeEvent

session class-attribute instance-attribute 

session: Mapping[str, object] = field(default_factory=dict)

wire_type class-attribute instance-attribute 

wire_type = 'session.updated'

Speak dataclass

Bases: ResponseScopedEvent

metadata class-attribute instance-attribute 

metadata: Mapping[str, object] = field(default_factory=dict)

wire_type class-attribute instance-attribute 

wire_type = 'response.speak'

SpeechStarted dataclass

Bases: RealtimeEvent

audio_start_ms class-attribute instance-attribute 

audio_start_ms: int = 0

item_id class-attribute instance-attribute 

item_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'input_audio_buffer.speech_started'

SpeechStopped dataclass

Bases: RealtimeEvent

audio_end_ms class-attribute instance-attribute 

audio_end_ms: int = 0

item_id class-attribute instance-attribute 

item_id: str | None = None

wire_type class-attribute instance-attribute 

wire_type = 'input_audio_buffer.speech_stopped'

TextDelta dataclass

Bases: ResponseScopedEvent

delta class-attribute instance-attribute 

delta: str = ''

text property 

text: str | None

wire_type class-attribute instance-attribute 

wire_type = 'response.output_text.delta'

TextDone dataclass

Bases: ResponseScopedEvent

text class-attribute instance-attribute 

text: str = ''

wire_type class-attribute instance-attribute 

wire_type = 'response.output_text.done'

TranscriptDelta dataclass

Bases: TranscriptDelta

Tier 2: OpenAI's TranscriptDelta plus duplex request-start metrics on text-only units.

metadata class-attribute instance-attribute 

metadata: Mapping[str, object] | None = None

optional_wire_fields class-attribute instance-attribute 

optional_wire_fields = frozenset({'metadata'})

TranscriptDone dataclass

Bases: ResponseScopedEvent

text property 

text: str | None

transcript class-attribute instance-attribute 

transcript: str = ''

wire_type class-attribute instance-attribute 

wire_type = 'response.output_audio_transcript.done'

TurnEvent dataclass

Bases: RealtimeEvent

Turn-state transition (turn.event).

event class-attribute instance-attribute 

event: str = ''

turn_state class-attribute instance-attribute 

turn_state: str = ''

wire_type class-attribute instance-attribute 

wire_type = 'turn.event'

error_event 

error_event(
    code: str,
    message: str,
    *,
    event_id: object | None = None,
    param: object | None = None,
    extra: Mapping[str, object] | None = None,
) -> ErrorEvent

Build the error event for an internal code; event_id is the client event id.

new_event_id 

new_event_id() -> str

wire_value 

wire_value(value: object) -> object