Skip to content

vllm_omni.engine.duplex_orchestrator

DuplexOrchestrator: the duplex sibling of Orchestrator.

It hosts the DuplexSessionManager (admission, leases, session runners) on the orchestrator loop and implements the DuplexStagePort the runners use to submit resumable Stage0 requests. Generic stage forwarding, prewarm and cleanup stay in OrchestratorBase; this class only fills the template seams and applies the session-owned policy.

logger module-attribute 

logger = init_logger(__name__)

DuplexOrchestrator

Bases: Orchestrator, DuplexStagePort

Stage management for a duplex deployment; owns one DuplexSessionManager.

Extends the turn-based orchestrator rather than sitting beside it, so one engine serves both a duplex session and an ordinary request. The dependency runs duplex -> turn-based and never the reverse, which is what keeps Orchestrator free of duplex code.

duplex_session_config instance-attribute 

duplex_session_config = duplex_session_config

plugin instance-attribute 

plugin = plugin

session_manager instance-attribute 

session_manager = DuplexSessionManager(
    plugin=plugin,
    stage_port=self,
    output_sink=self.output_async_queue,
    result_sink=self.rpc_async_queue,
    runtime_config=self.duplex_session_config,
    model_config=model_config,
    log_stats=self.log_stats,
)

stage_count property 

stage_count: int

abort_requests async 

abort_requests(request_ids: list[str]) -> None

cleanup async 

cleanup(
    request_ids: list[str], *, abort: bool = False
) -> None

ensure_request 

ensure_request(context: DuplexStageRequestContext) -> None

sampling_defaults 

sampling_defaults() -> tuple[object, ...]

submit async 

submit(
    submission: DuplexStageSubmission,
) -> DuplexStageSubmissionResult

DuplexOrchestratorRequestState dataclass

Bases: OrchestratorRequestState

Request bookkeeping for a session-owned (resumable duplex) stage request.

config_generation class-attribute instance-attribute 

config_generation: int = -1

fence class-attribute instance-attribute 

fence: DuplexFence | None = None

session_id class-attribute instance-attribute 

session_id: str = ''

stage_fences class-attribute instance-attribute 

stage_fences: dict[int, DuplexFence] = field(
    default_factory=dict
)