Skip to content

vllm_omni.inputs.data

DIFFUSION_QUALITY_LEVELS module-attribute 

DIFFUSION_QUALITY_LEVELS: tuple[str, ...] = (
    "lossless",
    "high",
)

OmniPromptType module-attribute 

OmniPromptType: TypeAlias = (
    PromptType
    | OmniTextPrompt
    | OmniTokensPrompt
    | OmniEmbedsPrompt
    | OmniCustomPrompt
)

OmniSamplingParams module-attribute 

OmniSamplingParams: TypeAlias = (
    SamplingParams | OmniDiffusionSamplingParams
)

OmniSingletonPrompt module-attribute 

OmniSingletonPrompt: TypeAlias = (
    str
    | list[int]
    | OmniTextPrompt
    | OmniTokensPrompt
    | OmniEmbedsPrompt
)

Omni singleton prompt type extending vLLM's SingletonPrompt.

OmniCustomPrompt

Bases: TypedDict

Custom prompt type for diffusion pipelines with pre-tokenized inputs.

Allows users to pass pre-tokenized prompt IDs, attention masks, and extra arguments directly, bypassing the tokenization stage in the pipeline.

Attributes:

Name Type Description
prompt_ids list[int] | list[list[int]]

Pre-tokenized prompt token IDs (single or batched)
negative_prompt_ids list[int] | list[list[int]]

Pre-tokenized negative prompt token IDs
prompt_mask Tensor

Attention mask tensor for the prompt
negative_prompt_mask Tensor

Attention mask tensor for the negative prompt
extra_args dict[str, Any]

Additional pipeline-specific arguments

extra_args instance-attribute 

extra_args: dict[str, Any]

negative_prompt_ids instance-attribute 

negative_prompt_ids: list[int] | list[list[int]]

negative_prompt_mask instance-attribute 

negative_prompt_mask: Tensor

prompt_ids instance-attribute 

prompt_ids: list[int] | list[list[int]]

prompt_mask instance-attribute 

prompt_mask: Tensor

OmniDiffusionSamplingParams dataclass

The collection of sampling parameters passed to diffusion pipelines.

This dataclass contains all information needed during the diffusion pipeline execution, allowing methods to update specific components without needing to manage numerous individual parameters.

STA_param class-attribute instance-attribute 

STA_param: list | None = None

VSA_sparsity class-attribute instance-attribute 

VSA_sparsity: float = 0.0

audio_latents class-attribute instance-attribute 

audio_latents: Tensor | None = None

batch_size property 

batch_size

boundary_ratio class-attribute instance-attribute 

boundary_ratio: float | None = None

cfg_active_branch class-attribute instance-attribute 

cfg_active_branch: str | None = None

cfg_branch_kv_metadata class-attribute instance-attribute 

cfg_branch_kv_metadata: dict[str, dict[str, Any]] | None = (
    None
)

cfg_branch_past_key_values class-attribute instance-attribute 

cfg_branch_past_key_values: dict[str, Any] | None = None

cfg_branch_roles class-attribute instance-attribute 

cfg_branch_roles: list[str] | None = None

cfg_img_kv_metadata class-attribute instance-attribute 

cfg_img_kv_metadata: dict[str, Any] | None = None

cfg_img_past_key_values class-attribute instance-attribute 

cfg_img_past_key_values: Any | None = None

cfg_kv_request_ids class-attribute instance-attribute 

cfg_kv_request_ids: dict[str, str] | None = None

cfg_normalize class-attribute instance-attribute 

cfg_normalize: bool = False

cfg_text_kv_metadata class-attribute instance-attribute 

cfg_text_kv_metadata: dict[str, Any] | None = None

cfg_text_past_key_values class-attribute instance-attribute 

cfg_text_past_key_values: Any | None = None

debug class-attribute instance-attribute 

debug: bool = False

decode_noise_scale class-attribute instance-attribute 

decode_noise_scale: float | list[float] | None = None

decode_timestep class-attribute instance-attribute 

decode_timestep: float | list[float] | None = None

do_classifier_free_guidance class-attribute instance-attribute 

do_classifier_free_guidance: bool = False

emit_request_lifecycle class-attribute instance-attribute 

emit_request_lifecycle: bool = False

enable_frame_interpolation class-attribute instance-attribute 

enable_frame_interpolation: bool = False

eta class-attribute instance-attribute 

eta: float = 0.0

extra_args class-attribute instance-attribute 

extra_args: dict[str, Any] = field(default_factory=dict)

extra_step_kwargs class-attribute instance-attribute 

extra_step_kwargs: dict[str, Any] = field(
    default_factory=dict
)

fps class-attribute instance-attribute 

fps: float | None = None

frame_interpolation_exp class-attribute instance-attribute 

frame_interpolation_exp: int = 1

frame_interpolation_model_path class-attribute instance-attribute 

frame_interpolation_model_path: str | None = None

frame_interpolation_scale class-attribute instance-attribute 

frame_interpolation_scale: float = 1.0

frame_rate class-attribute instance-attribute 

frame_rate: float | None = None

generator class-attribute instance-attribute 

generator: Generator | list[Generator] | None = None

generator_device class-attribute instance-attribute 

generator_device: str | None = None

guidance_rescale class-attribute instance-attribute 

guidance_rescale: float = 0.0

guidance_scale class-attribute instance-attribute 

guidance_scale: float | None = None

guidance_scale_2 class-attribute instance-attribute 

guidance_scale_2: float | None = None

guidance_scale_2_provided class-attribute instance-attribute 

guidance_scale_2_provided: bool = False

guidance_scale_provided class-attribute instance-attribute 

guidance_scale_provided: bool = False

height class-attribute instance-attribute 

height: int | None = None

height_latents class-attribute instance-attribute 

height_latents: list[int] | int | None = None

height_not_provided class-attribute instance-attribute 

height_not_provided: bool = False

image_latent class-attribute instance-attribute 

image_latent: Tensor | None = None

is_cfg_negative class-attribute instance-attribute 

is_cfg_negative: bool = False

is_prompt_processed class-attribute instance-attribute 

is_prompt_processed: bool = False

kv_metadata class-attribute instance-attribute 

kv_metadata: dict[str, Any] | None = None

latents class-attribute instance-attribute 

latents: Tensor | None = None

layers class-attribute instance-attribute 

layers: int = 4

lora_request class-attribute instance-attribute 

lora_request: LoRARequest | None = None

lora_scale class-attribute instance-attribute 

lora_scale: float = 1.0

mask_search_final_result_neg class-attribute instance-attribute 

mask_search_final_result_neg: list[list] | None = None

mask_search_final_result_pos class-attribute instance-attribute 

mask_search_final_result_pos: list[list] | None = None

max_sequence_length class-attribute instance-attribute 

max_sequence_length: int | None = None

modules class-attribute instance-attribute 

modules: dict[str, Any] = field(default_factory=dict)

n_tokens class-attribute instance-attribute 

n_tokens: int | None = None

need_kv_receive class-attribute instance-attribute 

need_kv_receive: bool = True

noise_pred class-attribute instance-attribute 

noise_pred: Tensor | None = None

num_frames class-attribute instance-attribute 

num_frames: int = 1

num_frames_round_down class-attribute instance-attribute 

num_frames_round_down: bool = False

num_inference_steps class-attribute instance-attribute 

num_inference_steps: int | None = None

num_outputs_per_prompt class-attribute instance-attribute 

num_outputs_per_prompt: int = 1

num_profiled_timesteps class-attribute instance-attribute 

num_profiled_timesteps: int = 8

output class-attribute instance-attribute 

output: Tensor | None = None

output_type class-attribute instance-attribute 

output_type: str | None = None

past_key_values class-attribute instance-attribute 

past_key_values: Any | None = None

profile class-attribute instance-attribute 

profile: bool = False

prompt_template class-attribute instance-attribute 

prompt_template: dict[str, Any] | None = None

quality class-attribute instance-attribute 

quality: str | None = None

raw_latent_shape class-attribute instance-attribute 

raw_latent_shape: Tensor | None = None

resolution class-attribute instance-attribute 

resolution: int = 640

resolved_frame_rate property 

resolved_frame_rate: float | None

Frame rate the model should run at, or None if the user did not provide one.

Precedence is frame_rate (the model-internal rate) over fps (the request/output-encoding alias). Returns None when neither was set, which is the model-agnostic "fps not provided" signal the video serving layer emits when the request omits fps. Consumers MUST treat None as "use your own default" and guard the read (e.g. req.sampling_params.resolved_frame_rate or 24.0); never feed it into arithmetic unguarded.

return_frames class-attribute instance-attribute 

return_frames: bool = False

return_trajectory_decoded class-attribute instance-attribute 

return_trajectory_decoded: bool = False

return_trajectory_latents class-attribute instance-attribute 

return_trajectory_latents: bool = False

save_output class-attribute instance-attribute 

save_output: bool = True

seed class-attribute instance-attribute 

seed: int | None = None

sigmas class-attribute instance-attribute 

sigmas: list[float] | None = None

step_index class-attribute instance-attribute 

step_index: int | None = None

strength class-attribute instance-attribute 

strength: float | None = None

timestep class-attribute instance-attribute 

timestep: Tensor | float | int | None = None

timesteps class-attribute instance-attribute 

timesteps: Tensor | None = None

trajectory_latents class-attribute instance-attribute 

trajectory_latents: Tensor | None = None

trajectory_timesteps class-attribute instance-attribute 

trajectory_timesteps: list[Tensor] | None = None

true_cfg_scale class-attribute instance-attribute 

true_cfg_scale: float | None = None

use_en_prompt class-attribute instance-attribute 

use_en_prompt: bool = False

width class-attribute instance-attribute 

width: int | None = None

width_latents class-attribute instance-attribute 

width_latents: list[int] | int | None = None

width_not_provided class-attribute instance-attribute 

width_not_provided: bool = False

clone 

clone() -> OmniDiffusionSamplingParams

from_params classmethod 

from_params(params: Any) -> OmniDiffusionSamplingParams

Normalize caller params for diffusion stages.

Existing Omni params are returned unchanged. Plain vLLM SamplingParams copy seed and matching extra_args fields; remaining extra_args stay nested. Unsupported types raise TypeError.

OmniEmbedsPrompt

Bases: EmbedsPrompt

Embeddings prompt with optional stage payloads.

Extends EmbedsPrompt to support additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings
additional_information NotRequired[dict[str, Any]]

Legacy optional dictionary containing additional information to pass along with the prompt.
model_intermediate_buffer NotRequired[dict[str, Any]]

Optional runner-owned stage payload.

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

model_intermediate_buffer instance-attribute 

model_intermediate_buffer: NotRequired[dict[str, Any]]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[list[Tensor] | None]

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

OmniInteractionEvent

Bases: OmniTextPrompt

Prompt-event payload for mid-stream interactions.

OmniInteractionPrompt

Bases: TypedDict

Normalized mid-stream interaction payload passed to submit_interaction.

WebSocket clients payload may omit event_id; the API layer assigns a fallback ID in this case.

event instance-attribute 

event: OmniInteractionEvent

event_id instance-attribute 

event_id: str

transition_chunks instance-attribute 

transition_chunks: NotRequired[int]

OmniTextPrompt

Bases: TextPrompt

Text prompt with optional embeddings and stage payloads.

Extends TextPrompt to support prompt embeddings and additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings
additional_information NotRequired[dict[str, Any]]

Legacy optional dictionary containing additional information to pass along with the prompt.
model_intermediate_buffer NotRequired[dict[str, Any]]

Optional runner-owned stage payload.

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

modalities instance-attribute 

modalities: NotRequired[list[str]]

model_intermediate_buffer instance-attribute 

model_intermediate_buffer: NotRequired[dict[str, Any]]

negative_prompt instance-attribute 

negative_prompt: NotRequired[str]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[Tensor]

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

OmniTokenInputs

Bases: TokensInput

Token inputs with optional embeddings and stage payloads.

Extends TokensInput to support prompt embeddings and additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings aligned with token IDs
additional_information NotRequired[dict[str, Any]]

Legacy optional dictionary containing additional information to pass along with the inputs.
model_intermediate_buffer NotRequired[dict[str, Any]]

Optional runner-owned stage payload.

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

model_intermediate_buffer instance-attribute 

model_intermediate_buffer: NotRequired[dict[str, Any]]

negative_prompt instance-attribute 

negative_prompt: NotRequired[str]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[list[Tensor] | None]

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

OmniTokensPrompt

Bases: TokensPrompt

Tokens prompt with optional embeddings and stage payloads.

Extends TokensPrompt to support prompt embeddings and additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings
additional_information NotRequired[dict[str, Any]]

Legacy optional dictionary containing additional information to pass along with the prompt.
model_intermediate_buffer NotRequired[dict[str, Any]]

Optional runner-owned stage payload.

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

model_intermediate_buffer instance-attribute 

model_intermediate_buffer: NotRequired[dict[str, Any]]

negative_prompt instance-attribute 

negative_prompt: NotRequired[str]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[list[Tensor] | None]

The embeddings of the prompt.

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

token_inputs_omni 

token_inputs_omni(
    prompt_token_ids: list[int],
    prompt: str | None = None,
    cache_salt: str | None = None,
    prompt_embeds: Tensor | None = None,
    additional_information: dict[str, Any] | None = None,
    model_intermediate_buffer: dict[str, Any] | None = None,
) -> OmniTokenInputs

Construct token inputs with optional embeddings and metadata.

Creates an OmniTokenInputs object with token IDs and optional embeddings and additional information for pipeline stage transfer.

Parameters:

Name Type Description Default
prompt_token_ids list[int]

List of token IDs for the prompt

 required
prompt str | None

Optional prompt string

 None
cache_salt str | None

Optional cache salt for prefix caching

 None
prompt_embeds Tensor | None

Optional tensor containing prompt embeddings

 None
additional_information dict[str, Any] | None

Legacy optional dictionary containing additional information (tensors or lists).

 None
model_intermediate_buffer dict[str, Any] | None

Optional runner-owned stage payload.

 None

Returns:

Type Description
OmniTokenInputs

OmniTokenInputs instance with the provided data