Summary¶

Entry Points¶

Main entry points for vLLM-Omni inference and serving.

• vllm_omni.entrypoints.async_omni.AsyncOmni
• vllm_omni.entrypoints.async_omni_base.AsyncEventResolver
• vllm_omni.entrypoints.async_omni_base.AsyncOmniBase
• vllm_omni.entrypoints.cli.benchmark.base.OmniBenchmarkSubcommandBase
• vllm_omni.entrypoints.cli.benchmark.cli_args.preprocess_serve_args
• vllm_omni.entrypoints.cli.benchmark.main.OmniBenchmarkSubcommand
• vllm_omni.entrypoints.cli.benchmark.omni_duplex_eval.OmniDuplexEvalSubcommand
• vllm_omni.entrypoints.cli.benchmark.serve.OmniBenchmarkServingSubcommand
• vllm_omni.entrypoints.cli.serve.OmniServeCommand
• vllm_omni.entrypoints.client_request_state.ClientRequestState
• vllm_omni.entrypoints.duplex.realtime_input.RealtimeEnvelope
• vllm_omni.entrypoints.duplex.realtime_input.RealtimeHandshake
• vllm_omni.entrypoints.duplex.realtime_input.ResumeRequest
• vllm_omni.entrypoints.duplex.realtime_input.parse_resume_request
• vllm_omni.entrypoints.duplex.serving.OmniDuplexSessionHandler
• vllm_omni.entrypoints.duplex.session_attachment.DuplexEventJournal
• vllm_omni.entrypoints.duplex.session_attachment.DuplexJournalGapError
• vllm_omni.entrypoints.duplex.session_attachment.DuplexJournalOverflowError
• vllm_omni.entrypoints.duplex.session_attachment.DuplexResumeCredential
• vllm_omni.entrypoints.duplex.session_attachment.DuplexSessionAttachmentCreated
• vllm_omni.entrypoints.duplex.session_attachment.DuplexSessionAttachmentRegistry
• vllm_omni.entrypoints.duplex.session_attachment.DuplexSessionResumeResult
• vllm_omni.entrypoints.duplex.session_attachment.DuplexTransportAttachment
• vllm_omni.entrypoints.duplex.session_attachment.InvalidResumeTokenError
• vllm_omni.entrypoints.duplex.session_attachment.JournalEntry
• vllm_omni.entrypoints.duplex.session_attachment.ResumeToken
• vllm_omni.entrypoints.duplex_omni.DuplexOmni
• vllm_omni.entrypoints.duplex_omni.DuplexSessionHandle
• vllm_omni.entrypoints.omni.Omni
• vllm_omni.entrypoints.omni_base.OmniBase
• vllm_omni.entrypoints.omni_base.OmniEngineDeadError
• vllm_omni.entrypoints.openpi.connection.RobotRealtimeConnection
• vllm_omni.entrypoints.openpi.serving.PolicyServerConfig
• vllm_omni.entrypoints.openpi.serving.ServingRealtimeRobotOpenPI
• vllm_omni.entrypoints.pd_utils.PDDisaggregationMixin
• vllm_omni.entrypoints.serve.omni_control.protocol.OmniSleepRequest
• vllm_omni.entrypoints.serve.omni_control.protocol.OmniWakeupRequest
• vllm_omni.entrypoints.serve.profile.protocol.ProfileRequest
• vllm_omni.entrypoints.utils.PureDiffusionLauncherAdapter
• vllm_omni.entrypoints.utils.StageConfigInputs
• vllm_omni.entrypoints.utils.parse_stage_overrides

Clients¶

Client libraries for full-duplex sessions: the Realtime WebSocket client, the in-process client over DuplexOmni, and the per-model session presets.

• vllm_omni.clients.duplex.AudioDelta
• vllm_omni.clients.duplex.AudioFormat
• vllm_omni.clients.duplex.ConnectionResumed
• vllm_omni.clients.duplex.DuplexClient
• vllm_omni.clients.duplex.DuplexClientBase
• vllm_omni.clients.duplex.DuplexClientError
• vllm_omni.clients.duplex.DuplexConnectionError
• vllm_omni.clients.duplex.DuplexEvent
• vllm_omni.clients.duplex.DuplexProtocolError
• vllm_omni.clients.duplex.DuplexSessionClosedError
• vllm_omni.clients.duplex.ErrorEvent
• vllm_omni.clients.duplex.EventCollector
• vllm_omni.clients.duplex.ListenDecision
• vllm_omni.clients.duplex.ReconnectPolicy
• vllm_omni.clients.duplex.ResponseCreated
• vllm_omni.clients.duplex.ResponseDone
• vllm_omni.clients.duplex.ResponseHandle
• vllm_omni.clients.duplex.SessionClosed
• vllm_omni.clients.duplex.SessionConfig
• vllm_omni.clients.duplex.SessionCreated
• vllm_omni.clients.duplex.SessionExpired
• vllm_omni.clients.duplex.SessionResumed
• vllm_omni.clients.duplex.SessionUpdated
• vllm_omni.clients.duplex.SpeakDecision
• vllm_omni.clients.duplex.TextDelta
• vllm_omni.clients.duplex.TranscriptDelta
• vllm_omni.clients.duplex.WebSocketTransport
• vllm_omni.clients.inline_duplex.InlineDuplexClient
• vllm_omni.clients.utils.StageModalityFlags

Inputs¶

Input data structures for multi-modal inputs.

• vllm_omni.inputs.data.OmniCustomPrompt
• vllm_omni.inputs.data.OmniDiffusionSamplingParams
• vllm_omni.inputs.data.OmniEmbedsPrompt
• vllm_omni.inputs.data.OmniInteractionEvent
• vllm_omni.inputs.data.OmniInteractionPrompt
• vllm_omni.inputs.data.OmniTextPrompt
• vllm_omni.inputs.data.OmniTokenInputs
• vllm_omni.inputs.data.OmniTokensPrompt
• vllm_omni.inputs.mm_processor.OmniMultiModalProcessor
• vllm_omni.inputs.preprocess.OmniRenderer
• vllm_omni.inputs.preprocess.build_omni_renderer
• vllm_omni.inputs.preprocess.omni_renderer_cls

Engine¶

Engine classes for offline and online inference.

• vllm_omni.diffusion.diffusion_engine.DiffusionEngine
• vllm_omni.diffusion.diffusion_engine.DiffusionExecutionMode
• vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.MooncakeAgentMetadata
• vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.MooncakeTransferEngineConnector
• vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.QueryRequest
• vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.QueryResponse
• vllm_omni.distributed.omni_connectors.connectors.mori_transfer_engine_connector.MoriPullRequest
• vllm_omni.distributed.omni_connectors.connectors.mori_transfer_engine_connector.MoriTransferEngineConnector
• vllm_omni.distributed.omni_connectors.connectors.mori_transfer_engine_connector.QueryRequest
• vllm_omni.distributed.omni_connectors.connectors.mori_transfer_engine_connector.QueryResponse
• vllm_omni.engine.AdditionalInformationEntry
• vllm_omni.engine.AdditionalInformationPayload
• vllm_omni.engine.OmniEngineCoreOutput
• vllm_omni.engine.OmniEngineCoreOutputs
• vllm_omni.engine.OmniEngineCoreRequest
• vllm_omni.engine.PromptEmbedsPayload
• vllm_omni.engine.arg_utils.OmniAsyncEngineArgs
• vllm_omni.engine.arg_utils.OmniEngineArgs
• vllm_omni.engine.arg_utils.OrchestratorArgs
• vllm_omni.engine.async_omni_engine.AsyncOmniEngine
• vllm_omni.engine.cfg_companion_tracker.CfgCompanionTracker
• vllm_omni.engine.duplex.commands.AckPlayback
• vllm_omni.engine.duplex.commands.AppendAudio
• vllm_omni.engine.duplex.commands.AppendText
• vllm_omni.engine.duplex.commands.BargeIn
• vllm_omni.engine.duplex.commands.CancelInput
• vllm_omni.engine.duplex.commands.CancelResponse
• vllm_omni.engine.duplex.commands.ClearInput
• vllm_omni.engine.duplex.commands.ClearOutputAudio
• vllm_omni.engine.duplex.commands.CloseSession
• vllm_omni.engine.duplex.commands.Commit
• vllm_omni.engine.duplex.commands.CreateItem
• vllm_omni.engine.duplex.commands.CreateResponse
• vllm_omni.engine.duplex.commands.DeleteItem
• vllm_omni.engine.duplex.commands.DuplexCommand
• vllm_omni.engine.duplex.commands.DuplexCommandError
• vllm_omni.engine.duplex.commands.Heartbeat
• vllm_omni.engine.duplex.commands.SignalTurn
• vllm_omni.engine.duplex.commands.TruncateItem
• vllm_omni.engine.duplex.commands.UpdateSession
• vllm_omni.engine.duplex.config.DuplexAssistantAudioTextMark
• vllm_omni.engine.duplex.config.DuplexAudioChunk
• vllm_omni.engine.duplex.config.DuplexCapabilities
• vllm_omni.engine.duplex.config.DuplexCommittedInput
• vllm_omni.engine.duplex.config.DuplexConfigError
• vllm_omni.engine.duplex.config.DuplexOverlapPolicy
• vllm_omni.engine.duplex.config.DuplexPlaybackCommitPolicy
• vllm_omni.engine.duplex.config.DuplexPlaybackCursor
• vllm_omni.engine.duplex.config.DuplexPlaybackView
• vllm_omni.engine.duplex.config.DuplexSessionConfig
• vllm_omni.engine.duplex.config.DuplexSessionState
• vllm_omni.engine.duplex.config.DuplexTurnEventType
• vllm_omni.engine.duplex.config.DuplexTurnState
• vllm_omni.engine.duplex.config.ResponseCreateOptions
• vllm_omni.engine.duplex.contracts.DuplexAppendPlan
• vllm_omni.engine.duplex.contracts.DuplexFence
• vllm_omni.engine.duplex.contracts.DuplexOutputAction
• vllm_omni.engine.duplex.contracts.DuplexOutputContext
• vllm_omni.engine.duplex.contracts.DuplexOutputDecision
• vllm_omni.engine.duplex.contracts.DuplexRequestIdentity
• vllm_omni.engine.duplex.contracts.DuplexStagePort
• vllm_omni.engine.duplex.contracts.DuplexStageRequestContext
• vllm_omni.engine.duplex.contracts.DuplexStageSubmission
• vllm_omni.engine.duplex.contracts.DuplexStageSubmissionResult
• vllm_omni.engine.duplex.intermediate.DuplexIntermediateBuffer
• vllm_omni.engine.duplex.messages.CloseDuplexSessionMessage
• vllm_omni.engine.duplex.messages.DuplexControlResultMessage
• vllm_omni.engine.duplex.messages.DuplexSessionCommandMessage
• vllm_omni.engine.duplex.messages.DuplexSessionError
• vllm_omni.engine.duplex.messages.DuplexSessionEventMessage
• vllm_omni.engine.duplex.messages.OpenDuplexSessionMessage
• vllm_omni.engine.duplex.messages.ResumeDuplexSessionMessage
• vllm_omni.engine.duplex.messages.TouchDuplexSessionMessage
• vllm_omni.engine.duplex.plugin.DuplexDataPlane
• vllm_omni.engine.duplex.plugin.DuplexModelPlugin
• vllm_omni.engine.duplex.plugin.DuplexModelSessionState
• vllm_omni.engine.duplex.plugin.DuplexRuntimeConfigError
• vllm_omni.engine.duplex.plugin.PartialStageForward
• vllm_omni.engine.duplex.plugin.PcmAppendBuffer
• vllm_omni.engine.duplex.plugin.PcmAppendReservation
• vllm_omni.engine.duplex.realtime_events.RealtimeProjectionState
• vllm_omni.engine.duplex.realtime_events.ResolvedCommit
• vllm_omni.engine.duplex.realtime_events.ResolvedControl
• vllm_omni.engine.duplex.session.append_task.AppendAttempt
• vllm_omni.engine.duplex.session.commit_policy.CommitAction
• vllm_omni.engine.duplex.session.commit_policy.CommitSnapshot
• vllm_omni.engine.duplex.session.context.DuplexAppendTaskMeta
• vllm_omni.engine.duplex.session.context.DuplexRunState
• vllm_omni.engine.duplex.session.context.DuplexSessionContext
• vllm_omni.engine.duplex.session.context.DuplexSessionTasks
• vllm_omni.engine.duplex.session.context.RunnerServices
• vllm_omni.engine.duplex.session.context.StageOutput
• vllm_omni.engine.duplex.session.control.SessionControl
• vllm_omni.engine.duplex.session.emitter.SessionEmitter
• vllm_omni.engine.duplex.session.engine_session.ConversationHistory
• vllm_omni.engine.duplex.session.engine_session.DuplexAppendReservation
• vllm_omni.engine.duplex.session.engine_session.DuplexEngineSession
• vllm_omni.engine.duplex.session.engine_session.DuplexFenceMismatchError
• vllm_omni.engine.duplex.session.engine_session.DuplexInputAppend
• vllm_omni.engine.duplex.session.engine_session.DuplexRequestResource
• vllm_omni.engine.duplex.session.engine_session.InputBufferState
• vllm_omni.engine.duplex.session.engine_session.PlaybackLedger
• vllm_omni.engine.duplex.session.engine_session.ResponseState
• vllm_omni.engine.duplex.session.lease.DuplexLeaseActivity
• vllm_omni.engine.duplex.session.lease.DuplexLeaseConfig
• vllm_omni.engine.duplex.session.lease.DuplexLeaseState
• vllm_omni.engine.duplex.session.manager.DuplexSessionManager
• vllm_omni.engine.duplex.session.model_channel.ModelChannel
• vllm_omni.engine.duplex.session.model_channel.SilenceContinuationScheduler
• vllm_omni.engine.duplex.session.runner.DuplexSessionRunner
• vllm_omni.engine.duplex.turn_detection.PendingTurnDetectionUpdate
• vllm_omni.engine.duplex.turn_detection.ServerTurnDetector
• vllm_omni.engine.duplex.turn_detection.TurnDetectionConfig
• vllm_omni.engine.duplex.turn_detection.TurnDetectionResult
• vllm_omni.engine.duplex.vad.ServerVADUnavailableError
• vllm_omni.engine.duplex.vad.SileroStreamingVAD
• vllm_omni.engine.duplex.vad.SileroVADBackend
• vllm_omni.engine.duplex.vad.SileroVADBackendProvider
• vllm_omni.engine.duplex.vad.SileroVADConfig
• vllm_omni.engine.duplex.vad.SpeechDetectorBackend
• vllm_omni.engine.duplex.vad.StreamingVADResult
• vllm_omni.engine.duplex.vad.TorchSileroBackend
• vllm_omni.engine.duplex_omni_engine.DuplexOmniEngine
• vllm_omni.engine.duplex_orchestrator.DuplexOrchestrator
• vllm_omni.engine.duplex_orchestrator.DuplexOrchestratorRequestState
• vllm_omni.engine.errors.NativeKVHandoffError
• vllm_omni.engine.membership_controller.MembershipController
• vllm_omni.engine.messages.AbortRequestMessage
• vllm_omni.engine.messages.AbortResultMessage
• vllm_omni.engine.messages.AddCompanionRequestMessage
• vllm_omni.engine.messages.CollectiveRPCRequestMessage
• vllm_omni.engine.messages.CollectiveRPCResultMessage
• vllm_omni.engine.messages.EngineQueueMessage
• vllm_omni.engine.messages.ErrorMessage
• vllm_omni.engine.messages.InteractionMessage
• vllm_omni.engine.messages.OutputMessage
• vllm_omni.engine.messages.RegisterRemoteReplicaMessage
• vllm_omni.engine.messages.ShutdownRequestMessage
• vllm_omni.engine.messages.StageMetricsMessage
• vllm_omni.engine.messages.StageSubmissionMessage
• vllm_omni.engine.messages.UnregisterRemoteReplicaMessage
• vllm_omni.engine.omni_engine_base.OmniEngineBase
• vllm_omni.engine.orchestrator.Orchestrator
• vllm_omni.engine.orchestrator.OrchestratorBase
• vllm_omni.engine.orchestrator.OrchestratorRequestState
• vllm_omni.engine.orchestrator.StreamingInputState
• vllm_omni.engine.orchestrator.StreamingSegmentState
• vllm_omni.engine.orchestrator_monitor.OrchestratorMonitor
• vllm_omni.engine.orchestrator_monitor.OrchestratorMonitorBase
• vllm_omni.engine.rpc_result_router.CorrelatedRpcClient
• vllm_omni.engine.rpc_result_router.RpcResultRouter
• vllm_omni.engine.stage_admission.AdmissionExempt
• vllm_omni.engine.stage_admission.DeviceLedger
• vllm_omni.engine.stage_admission.StageAdmissionError
• vllm_omni.engine.stage_admission.StageDemand
• vllm_omni.engine.stage_client.StageClient
• vllm_omni.engine.stage_client.StageClientBase
• vllm_omni.engine.stage_client.StagePoolClient
• vllm_omni.engine.stage_client.StagePoolDiffusionClient
• vllm_omni.engine.stage_client.StagePoolLLMClient
• vllm_omni.engine.stage_engine_core_client.DPLBStageEngineCoreClient
• vllm_omni.engine.stage_engine_core_client.StageEngineCoreClient
• vllm_omni.engine.stage_engine_core_client.StageEngineCoreClientBase
• vllm_omni.engine.stage_engine_core_proc.StageEngineCoreProc
• vllm_omni.engine.stage_engine_core_proc_manager.StageEngineCoreProcManager
• vllm_omni.engine.stage_engine_startup.OmniMasterServer
• vllm_omni.engine.stage_engine_startup.StageAllocation
• vllm_omni.engine.stage_engine_startup.StageCoordinatorAddresses
• vllm_omni.engine.stage_engine_startup.StageRegistrationResponse
• vllm_omni.engine.stage_engine_startup.StageReplicaResources
• vllm_omni.engine.stage_init_utils.LogicalStageInitPlan
• vllm_omni.engine.stage_init_utils.ReplicaInitPlan
• vllm_omni.engine.stage_init_utils.StageMetadata
• vllm_omni.engine.stage_init_utils.parse_physical_device_ids
• vllm_omni.engine.stage_phase_lock.DeviceLockTimeoutError
• vllm_omni.engine.stage_phase_lock.DevicePhaseLock
• vllm_omni.engine.stage_pool.StagePool
• vllm_omni.engine.stage_pool.StageUnavailableError
• vllm_omni.engine.stage_runtime.DistStageRuntime
• vllm_omni.engine.stage_runtime.OmniClientConfig
• vllm_omni.engine.stage_runtime.StageEngineLaunch
• vllm_omni.engine.stage_runtime.StageRemoteFactoryContext
• vllm_omni.engine.stage_runtime.StageRuntime
• vllm_omni.engine.stage_runtime.StageRuntimeInfo
• vllm_omni.platforms.npu.omni_connectors.yuanrong_transfer_engine_connector.CleanupRequest
• vllm_omni.platforms.npu.omni_connectors.yuanrong_transfer_engine_connector.QueryRequest
• vllm_omni.platforms.npu.omni_connectors.yuanrong_transfer_engine_connector.QueryResponse
• vllm_omni.platforms.npu.omni_connectors.yuanrong_transfer_engine_connector.YuanrongTransferEngineConnector

Core¶

Core scheduling and caching components.

• vllm_omni.core.prefix_cache.controller.OmniPrefixCacheController
• vllm_omni.core.prefix_cache.controller.StagingBufferHolder
• vllm_omni.core.prefix_cache.controller.StagingBufferPool
• vllm_omni.core.prefix_cache.controller.StepD2HClaim
• vllm_omni.core.prefix_cache.controller.TaskState
• vllm_omni.core.prefix_cache.controller.WriteTask
• vllm_omni.core.prefix_cache.interface.ModelCachePolicy
• vllm_omni.core.prefix_cache.interface.OmniPrefixCacheStagingTimeoutError
• vllm_omni.core.prefix_cache.interface.OmniPrefixCacheUnmatchError
• vllm_omni.core.prefix_cache.interface.PrefixCacheConfig
• vllm_omni.core.prefix_cache.interface.StageCacheOutputs
• vllm_omni.core.prefix_cache.interface.WriteSchedule
• vllm_omni.core.prefix_cache.manager.OmniPrefixCacheManager
• vllm_omni.core.prefix_cache.runner_mixin.PrefixCacheRunnerMixin
• vllm_omni.core.sched.omni_ar_scheduler.OmniARAsyncScheduler
• vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
• vllm_omni.core.sched.omni_ar_scheduler.SampledLogprobContractError
• vllm_omni.core.sched.omni_cfg_ar_scheduler.OmniCFGARScheduler
• vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
• vllm_omni.core.sched.omni_scheduler_mixin.OmniSchedulerMixin
• vllm_omni.core.sched.omni_scheduling_coordinator.OmniSchedulingCoordinator
• vllm_omni.core.sched.output.OmniCachedRequestData
• vllm_omni.core.sched.output.OmniChunkRecvHandle
• vllm_omni.core.sched.output.OmniNewRequestData
• vllm_omni.core.sched.output.OmniSchedulerOutput
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.BASECFM
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.CausalConditionalCFM
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.CausalMaskedDiffWithDiT
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.ConditionalCFM
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalConv1d
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalConv1dUpsample
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalConvRNNF0Predictor
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalHiFTGenerator
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.HiFTGenerator
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.SineGen
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.SineGen2
• vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.SourceModuleHnNSF
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.DistributedGroupResidualVectorQuantization
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.DistributedResidualVectorQuantization
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.EuclideanCodebook
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.VectorQuantization
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.preprocess

Configuration¶

Configuration classes.

• vllm_omni.config.composable_parallel.aggregation.AggregationConflictError
• vllm_omni.config.composable_parallel.aggregation.AggregationPattern
• vllm_omni.config.composable_parallel.aggregation.AllGather
• vllm_omni.config.composable_parallel.aggregation.Combine
• vllm_omni.config.composable_parallel.aggregation.FanInByStage
• vllm_omni.config.composable_parallel.aggregation.GatherDim
• vllm_omni.config.composable_parallel.aggregation.StitchPipeline
• vllm_omni.config.composable_parallel.aggregation.StitchSpatial
• vllm_omni.config.composable_parallel.aggregation.TakeRank
• vllm_omni.config.composable_parallel.aggregation.Union
• vllm_omni.config.composable_parallel.apply.StrategyApplyError
• vllm_omni.config.composable_parallel.apply.StrategyApplyResult
• vllm_omni.config.composable_parallel.routing.Broadcast
• vllm_omni.config.composable_parallel.routing.DuplicateWithCondTag
• vllm_omni.config.composable_parallel.routing.PartitionByHash
• vllm_omni.config.composable_parallel.routing.PipelineMicrobatch
• vllm_omni.config.composable_parallel.routing.RouteByExpert
• vllm_omni.config.composable_parallel.routing.RouteByStage
• vllm_omni.config.composable_parallel.routing.RoutingKeyError
• vllm_omni.config.composable_parallel.routing.RoutingPattern
• vllm_omni.config.composable_parallel.routing.ShardSequence
• vllm_omni.config.composable_parallel.routing.ShardSpatial
• vllm_omni.config.composable_parallel.spec.KernelSpec
• vllm_omni.config.composable_parallel.spec.MeshAxisKind
• vllm_omni.config.composable_parallel.spec.MeshAxisSpec
• vllm_omni.config.composable_parallel.spec.SpecMergeConflictError
• vllm_omni.config.composable_parallel.spec.StrategySpec
• vllm_omni.config.composable_parallel.strategy_loader.StrategyLoadError
• vllm_omni.config.composable_parallel.strategy_loader.parse_strategy_specs
• vllm_omni.config.composable_parallel.translator.AxisTranslationError
• vllm_omni.config.composable_parallel.translator.OmniParallelConfig
• vllm_omni.config.config_factory.StageConfigFactory
• vllm_omni.config.endpoint_policy.EndpointRestriction
• vllm_omni.config.endpoint_policy.OmniServingCapability
• vllm_omni.config.endpoint_policy.RouteTarget
• vllm_omni.config.environment_variable_inventory.EnvironmentVariableCategory
• vllm_omni.config.environment_variable_inventory.EnvironmentVariableClassification
• vllm_omni.config.environment_variable_inventory.ModelEnvironmentVariableDisposition
• vllm_omni.config.model.OmniModelArchConfigConvertor
• vllm_omni.config.model.OmniModelConfig
• vllm_omni.config.omni_config.BaseVllmOmniStageConfig
• vllm_omni.config.omni_config.OmniStageCacheConfig
• vllm_omni.config.omni_config.OmniStageConnectorConfig
• vllm_omni.config.omni_config.OmniStageDiffusionParallelConfig
• vllm_omni.config.omni_config.OmniStageLoadConfig
• vllm_omni.config.omni_config.OmniStageModelConfig
• vllm_omni.config.omni_config.OmniStageParallelConfig
• vllm_omni.config.omni_config.OmniStagePoolingConfig
• vllm_omni.config.omni_config.OmniStageRuntimeConfig
• vllm_omni.config.omni_config.OmniStageSchedulerConfig
• vllm_omni.config.omni_config.VllmOmniARStageConfig
• vllm_omni.config.omni_config.VllmOmniConfig
• vllm_omni.config.omni_config.VllmOmniDiffusionStageConfig
• vllm_omni.config.omni_config.VllmOmniGenerationStageConfig
• vllm_omni.config.omni_config.VllmOmniOrchestratorConfig
• vllm_omni.config.resolver.OmniConfigResolution
• vllm_omni.config.server_settings.FileBackend
• vllm_omni.config.server_settings.ServerSettings
• vllm_omni.config.stage_config.DeployConfig
• vllm_omni.config.stage_config.DuplexSessionRuntimeConfig
• vllm_omni.config.stage_config.PipelineConfig
• vllm_omni.config.stage_config.PlatformOverrides
• vllm_omni.config.stage_config.StageConfig
• vllm_omni.config.stage_config.StageDeployConfig
• vllm_omni.config.stage_config.StageExecutionType
• vllm_omni.config.stage_config.StagePipelineConfig
• vllm_omni.config.stage_config.StageType
• vllm_omni.diffusion.cache.cachedit.config.CacheDiTAdapterConfig
• vllm_omni.diffusion.cache.cachedit.config.CacheDiTConfig
• vllm_omni.diffusion.cache.magcache.config.MagCacheConfig
• vllm_omni.diffusion.cache.seacache.config.SeaCacheConfig
• vllm_omni.diffusion.cache.stepcache.config.StepCacheConfig
• vllm_omni.diffusion.cache.teacache.config.TeaCacheConfig
• vllm_omni.diffusion.diffusion_kv.config.DiffusionKVCacheMode
• vllm_omni.diffusion.diffusion_kv.config.parse_diffusion_kv_cache_mode
• vllm_omni.diffusion.models.cosmos3.mixed_precision.config.Cosmos3MixedPrecisionConfig
• vllm_omni.diffusion.models.dmd2.config.DMD2Config
• vllm_omni.diffusion.models.gr00t.configs.gr00t_n1d7.Gr00tN1d7Config
• vllm_omni.diffusion.models.internvla_a1.config.InternVLAA1Config
• vllm_omni.diffusion.models.internvla_a1.config.InternVLAA1TrainMetadata
• vllm_omni.diffusion.models.magi2.configuration_magi2.Magi2GenerationConfig
• vllm_omni.diffusion.models.magi2.configuration_magi2.Magi2MHCConfig
• vllm_omni.diffusion.models.magi2.configuration_magi2.Magi2MoEConfig
• vllm_omni.diffusion.models.magi2.configuration_magi2.Magi2PreviewConfig
• vllm_omni.diffusion.models.pi0.config.Pi0Config
• vllm_omni.diffusion.models.pi05.config.Pi05Config
• vllm_omni.diffusion.models.pi05.config.UnsupportedCheckpointCapabilityError
• vllm_omni.diffusion.models.sana_wm.config.SanaWmConfig
• vllm_omni.diffusion.offloader.config.DLOTransfer
• vllm_omni.diffusion.offloader.config.OffloadMode
• vllm_omni.diffusion.offloader.config.OffloadStrategy
• vllm_omni.diffusion.offloader.config.ParsedDiffusionOffloadConfig
• vllm_omni.diffusion.offloader.config.ResolvedOffload
• vllm_omni.diffusion.offloader.config.parse_diffusion_offload_config
• vllm_omni.diffusion.offloader.config.parse_dlo_transfer
• vllm_omni.diffusion.offloader.config.parse_offload_components
• vllm_omni.distributed.omni_connectors.utils.config.ConnectorSpec
• vllm_omni.distributed.omni_connectors.utils.config.OmniTransferConfig
• vllm_omni.host_weight_runtime.config.CapacityPolicy
• vllm_omni.host_weight_runtime.config.HostWeightRuntimeConfig
• vllm_omni.host_weight_runtime.config.IntegrityPolicy
• vllm_omni.host_weight_runtime.config.ProductionPolicy
• vllm_omni.host_weight_runtime.config.RemoteImportPolicy
• vllm_omni.host_weight_runtime.config.RemoteOnMiss
• vllm_omni.host_weight_runtime.config.RuntimeMode
• vllm_omni.host_weight_runtime.config.StorageClass
• vllm_omni.host_weight_runtime.config.StorageDomainPolicy
• vllm_omni.host_weight_runtime.config.StorageScope
• vllm_omni.host_weight_runtime.config.ValidationLevel
• vllm_omni.host_weight_runtime.config.WaitPolicy
• vllm_omni.model_executor.models.audex.speech_decoder.configuration_audex_causal_speech_decoder.AudexCausalSpeechDecoderConfig
• vllm_omni.model_executor.models.audio8_tts.configuration_audio8_tts.Audio8TTSConfig
• vllm_omni.model_executor.models.audio8_tts.configuration_audio8_tts.Audio8TTSFastARConfig
• vllm_omni.model_executor.models.audio8_tts.configuration_audio8_tts.Audio8TTSSlowARConfig
• vllm_omni.model_executor.models.breeze_tts_2.configuration_breeze.BreezeConfig
• vllm_omni.model_executor.models.covo_audio.config_covo_audio.CovoAudioCode2WavConfig
• vllm_omni.model_executor.models.fish_speech.configuration_fish_speech.FishSpeechConfig
• vllm_omni.model_executor.models.fish_speech.configuration_fish_speech.FishSpeechFastARConfig
• vllm_omni.model_executor.models.fish_speech.configuration_fish_speech.FishSpeechSlowARConfig
• vllm_omni.model_executor.models.gepard.configuration_gepard.GepardConfig
• vllm_omni.model_executor.models.higgs_audio_v2.configuration_higgs_audio_v2.HiggsAudioV2Config
• vllm_omni.model_executor.models.indextts2.configuration_indextts2.IndexTTS25Config
• vllm_omni.model_executor.models.indextts2.configuration_indextts2.IndexTTS2Config
• vllm_omni.model_executor.models.mimo_audio.config_mimo_audio.MiMoAudioConfig
• vllm_omni.model_executor.models.mimo_audio.config_mimo_audio.MiMoAudioTokenizerConfig
• vllm_omni.model_executor.models.ming_tts.config_ming_tts.BailingMoeConfig
• vllm_omni.model_executor.models.ming_tts.config_ming_tts.MingDenseConfig
• vllm_omni.model_executor.models.ming_tts.config_ming_tts.MingMoeConfig
• vllm_omni.model_executor.models.ming_tts.config_ming_tts.MingTTSConfig
• vllm_omni.model_executor.models.moss_tts.configuration_moss_audio_tokenizer_v2.MossAudioTokenizerConfig
• vllm_omni.model_executor.models.moss_tts.configuration_moss_tts.MossTTSDelayConfig
• vllm_omni.model_executor.models.moss_tts.configuration_moss_tts.MossTTSLocalConfig
• vllm_omni.model_executor.models.moss_tts.configuration_moss_tts.MossTTSLocalTransformerConfig
• vllm_omni.model_executor.models.moss_tts.configuration_moss_tts.MossTTSRealtimeConfig
• vllm_omni.model_executor.models.nemotron_voicechat.configuration_nemotron_voicechat.NemotronVoiceChatCode2WavConfig
• vllm_omni.model_executor.models.nemotron_voicechat.configuration_nemotron_voicechat.NemotronVoiceChatConfig
• vllm_omni.model_executor.models.nemotron_voicechat.configuration_nemotron_voicechat.NemotronVoiceChatTalkerConfig
• vllm_omni.model_executor.models.personaplex.configuration_helium.HeliumConfig
• vllm_omni.model_executor.models.personaplex.configuration_personaplex.PersonaPlexConfig
• vllm_omni.model_executor.models.personaplex.configuration_personaplex.PersonaPlexDepformerConfig
• vllm_omni.model_executor.models.personaplex.configuration_personaplex.PersonaPlexMimiConfig
• vllm_omni.model_executor.models.personaplex.duplex.config.PersonaPlexConfig
• vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSConfig
• vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSSpeakerEncoderConfig
• vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSTalkerCodePredictorConfig
• vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSTalkerConfig
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_12hz.configuration_qwen3_tts_tokenizer_v2.Qwen3TTSTokenizerV2Config
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_12hz.configuration_qwen3_tts_tokenizer_v2.Qwen3TTSTokenizerV2DecoderConfig
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1Config
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1DecoderBigVGANConfig
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1DecoderConfig
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1DecoderDiTConfig
• vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1EncoderConfig
• vllm_omni.transformers_utils.configs.auk.AuKConfig
• vllm_omni.transformers_utils.configs.cosyvoice3.CosyVoice3Config
• vllm_omni.transformers_utils.configs.glm_tts.GLMTTSConfig
• vllm_omni.transformers_utils.configs.higgs_audio_v3.HiggsAudioV3Config
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Config
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen2_5_VLConfig
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen2_5_VLTextConfig
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen2_5_VLVisionConfig
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen3VLConfig
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen3VLTextConfig
• vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen3VLVisionConfig
• vllm_omni.transformers_utils.configs.ming_flash_omni.BailingMM2Config
• vllm_omni.transformers_utils.configs.ming_flash_omni.BailingMoeV2Config
• vllm_omni.transformers_utils.configs.ming_flash_omni.MingFlashOmniConfig
• vllm_omni.transformers_utils.configs.ming_flash_omni.MingFlashOmniTalkerConfig
• vllm_omni.transformers_utils.configs.ming_flash_omni.MingImageGenConfig
• vllm_omni.transformers_utils.configs.ming_flash_omni.Qwen3VLMoeVisionConfig
• vllm_omni.transformers_utils.configs.ming_flash_omni.WhisperEncoderConfig
• vllm_omni.transformers_utils.configs.minimax_music3.MiniMaxMusic3Config
• vllm_omni.transformers_utils.configs.omnivoice.OmniVoiceConfig
• vllm_omni.transformers_utils.configs.sensenova_u1.SenseNovaU1Config
• vllm_omni.transformers_utils.configs.sensenova_u1.SenseNovaU1LLMConfig
• vllm_omni.transformers_utils.configs.sensenova_u1.SenseNovaU1VisionConfig
• vllm_omni.transformers_utils.configs.voxcpm2.VoxCPM2Config
• vllm_omni.transformers_utils.configs.voxtral_tts.VoxtralTTSConfig

Workers¶

Worker classes and model runners for distributed inference.

• vllm_omni.diffusion.models.dreamzero.video_export_worker.DreamZeroVideoExportWorkerExtension
• vllm_omni.diffusion.worker.diffusion_model_runner.DiffusionModelRunner
• vllm_omni.diffusion.worker.diffusion_worker.CustomPipelineWorkerExtension
• vllm_omni.diffusion.worker.diffusion_worker.DiffusionWorker
• vllm_omni.diffusion.worker.diffusion_worker.WorkerProc
• vllm_omni.diffusion.worker.diffusion_worker.WorkerWrapperBase
• vllm_omni.diffusion.worker.input_batch.InputBatch
• vllm_omni.diffusion.worker.request_batch.DiffusionRequestBatch
• vllm_omni.diffusion.worker.stage_payload.DiffusionStagePayloadMixin
• vllm_omni.diffusion.worker.utils.BaseRunnerOutput
• vllm_omni.diffusion.worker.utils.BatchRunnerOutput
• vllm_omni.diffusion.worker.utils.RunnerOutput
• vllm_omni.diffusion.worker.utils.StepRequestState
• vllm_omni.platforms.npu.worker.base.OmniNPUWorkerBase
• vllm_omni.platforms.npu.worker.npu_ar_model_runner.ExecuteModelState
• vllm_omni.platforms.npu.worker.npu_ar_model_runner.NPUARModelRunner
• vllm_omni.platforms.npu.worker.npu_ar_worker.NPUARWorker
• vllm_omni.platforms.npu.worker.npu_generation_model_runner.NPUGenerationModelRunner
• vllm_omni.platforms.npu.worker.npu_generation_worker.NPUGenerationWorker
• vllm_omni.platforms.npu.worker.npu_model_runner.OmniNPUModelRunner
• vllm_omni.platforms.xpu.worker.xpu_ar_model_runner.XPUARModelRunner
• vllm_omni.platforms.xpu.worker.xpu_ar_worker.XPUARWorker
• vllm_omni.platforms.xpu.worker.xpu_generation_model_runner.XPUGenerationModelRunner
• vllm_omni.platforms.xpu.worker.xpu_generation_worker.XPUGenerationWorker
• vllm_omni.worker.base.OmniGPUWorkerBase
• vllm_omni.worker.gpu_ar_model_runner.ExecuteModelState
• vllm_omni.worker.gpu_ar_model_runner.GPUARModelRunner
• vllm_omni.worker.gpu_ar_model_runner.OmniAsyncGPUModelRunnerOutput
• vllm_omni.worker.gpu_ar_worker.GPUARWorker
• vllm_omni.worker.gpu_generation_model_runner.GPUGenerationModelRunner
• vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker
• vllm_omni.worker.gpu_memory_utils.parse_cuda_visible_devices
• vllm_omni.worker.gpu_model_runner.OmniGPUModelRunner
• vllm_omni.worker.mixins.OmniWorkerMixin
• vllm_omni.worker.omni_connector_model_runner_mixin.OmniConnectorModelRunnerMixin
• vllm_omni.worker.runner_assisted_metadata.RunnerAssistedAttentionMetadataProvider
• vllm_omni.worker.runner_assisted_metadata.RunnerAssistedFullAttentionMetadataRequest
• vllm_omni.worker.sparse_audio.InvalidSparseDeclarationError
• vllm_omni.worker.sparse_audio.is_sparse_audio_marker
• vllm_omni.worker.sparse_audio.resolve_sparse_mm_req_ids
• vllm_omni.worker.sparse_audio.resolve_sparse_mm_routing
• vllm_omni.worker_v2.delivery.DeliveryCancelledError
• vllm_omni.worker_v2.delivery.DeliveryState
• vllm_omni.worker_v2.delivery.DeliveryTicket
• vllm_omni.worker_v2.delivery.DeliveryTimeoutError
• vllm_omni.worker_v2.delivery.OmniDeliveryManager
• vllm_omni.worker_v2.model_states.intermediate_buffer.OmniIntermediateBuffer
• vllm_omni.worker_v2.model_states.omni_model_state.OmniModelState
• vllm_omni.worker_v2.native_output_worker.NativeAsyncOutput
• vllm_omni.worker_v2.native_output_worker.NativeOutputWorker
• vllm_omni.worker_v2.omni_ar_model_runner.OmniARModelRunner
• vllm_omni.worker_v2.omni_ar_model_runner.OmniAsyncOutput
• vllm_omni.worker_v2.omni_data_plane.OmniRunnerDataPlane
• vllm_omni.worker_v2.omni_generation_model_runner.OmniGenerationAsyncOutput
• vllm_omni.worker_v2.omni_generation_model_runner.OmniGenerationModelRunner
• vllm_omni.worker_v2.omni_model_runner.OmniGPUModelRunner
• vllm_omni.worker_v2.output_snapshot.PackedOutputSnapshot