Skip to content

vllm_omni.distributed.omni_connectors.utils.config

TRANSFER_ENGINE_CONNECTOR_NAMES module-attribute 

TRANSFER_ENGINE_CONNECTOR_NAMES = frozenset(
    {
        "MooncakeTransferEngineConnector",
        "MoriTransferEngineConnector",
        "YuanrongTransferEngineConnector",
        "NixlConnector",
    }
)

logger module-attribute 

logger = get_connector_logger(__name__)

ConnectorSpec dataclass

Specification for a connector instance.

extra class-attribute instance-attribute 

extra: dict[str, Any] = field(default_factory=dict)

name instance-attribute 

name: str

OmniTransferConfig dataclass

Top-level configuration for OmniConnector system. Members: connectors: A dictionary of connectors, keyed by (from_stage, to_stage). default_connector: The default connector to use if no connector is specified for an edge.

connectors class-attribute instance-attribute 

connectors: dict[tuple[str, str], ConnectorSpec] = field(
    default_factory=dict
)

default_connector class-attribute instance-attribute 

default_connector: ConnectorSpec | None = None

get_connector_for_edge 

get_connector_for_edge(
    from_stage: str, to_stage: str
) -> ConnectorSpec | None

Get connector spec for a specific edge.

has_connector_for_edge 

has_connector_for_edge(
    from_stage: str, to_stage: str
) -> bool

Check if there's a connector configured for the edge.

get_stage_connector_role 

get_stage_connector_role(model_config: Any) -> str | None

Return the configured stage connector direction, if explicit.

stage_receives_chunks 

stage_receives_chunks(model_config: Any) -> bool

Whether connector chunks, rather than the orchestrator, feed a stage.

stage_sends_async_output 

stage_sends_async_output(model_config: Any) -> bool

Whether async output should be partitioned for connector transport.