vllm_omni.diffusion.interaction.registry ¶

Registry of mid-way interaction handlers during a diffusion generation. A pipeline only declares the modalities that it supports. And a pipeline can choose which handler for each modality (there can be more than one handler for a modality).

Outer key: pipeline architecture name (od_config.model_class_name). Inner key: modality name (e.g. "camera").

STRUCTURED_HANDLER_REGISTRY: dict[
    str, dict[str, type[InteractionHandler]]
] = {
    "HeliosPipeline": {"prompt": PromptInteractionHandler},
    "HeliosPyramidPipeline": {
        "prompt": PromptInteractionHandler
    },
    "LingBotWorldCausalDMDPipeline": {
        "camera": SE3DeltaCameraHandler
    },
}