Skip to content

vllm_omni.diffusion.sched.interface

CachedRequestData dataclass

Cached diffusion requests that only need their request ids resent.

request_ids instance-attribute 

request_ids: list[str]

make_empty classmethod 

make_empty() -> CachedRequestData

DiffusionRequestStatus

Bases: IntEnum

Request status tracked by diffusion scheduler.

FINISHED_ABORTED class-attribute instance-attribute 

FINISHED_ABORTED = enum.auto()

FINISHED_COMPLETED class-attribute instance-attribute 

FINISHED_COMPLETED = enum.auto()

FINISHED_ERROR class-attribute instance-attribute 

FINISHED_ERROR = enum.auto()

PREEMPTED class-attribute instance-attribute 

PREEMPTED = enum.auto()

RUNNING class-attribute instance-attribute 

RUNNING = enum.auto()

WAITING class-attribute instance-attribute 

WAITING = enum.auto()

is_finished staticmethod 

is_finished(status: DiffusionRequestStatus) -> bool

DiffusionSchedulerOutput dataclass

Output of a single scheduling cycle.

finished_req_ids instance-attribute 

finished_req_ids: set[str]

has_sync_kv_loads property 

has_sync_kv_loads: bool

is_empty property 

is_empty: bool

kv_connector_metadata class-attribute instance-attribute 

kv_connector_metadata: KVConnectorMetadata | None = None

kv_finished_request_ids class-attribute instance-attribute 

kv_finished_request_ids: set[str] = field(
    default_factory=set
)

kv_poll_only class-attribute instance-attribute 

kv_poll_only: bool = False

kv_prefetch_connector_metadata class-attribute instance-attribute 

kv_prefetch_connector_metadata: (
    KVConnectorMetadata | None
) = None

kv_prefetch_job class-attribute instance-attribute 

kv_prefetch_job: KVPrefetchJob | None = None

kv_prefetch_request_ids class-attribute instance-attribute 

kv_prefetch_request_ids: set[str] = field(
    default_factory=set
)

kv_required_request_ids class-attribute instance-attribute 

kv_required_request_ids: set[str] | None = None

kv_transfer_request_ids class-attribute instance-attribute 

kv_transfer_request_ids: set[str] = field(
    default_factory=set
)

num_running_reqs instance-attribute 

num_running_reqs: int

num_scheduled_reqs property 

num_scheduled_reqs: int

num_waiting_reqs instance-attribute 

num_waiting_reqs: int

scheduled_cached_reqs instance-attribute 

scheduled_cached_reqs: CachedRequestData

scheduled_new_reqs instance-attribute 

scheduled_new_reqs: list[NewRequestData]

scheduled_request_ids cached property 

scheduled_request_ids: list[str]

All scheduled request ids in this cycle, including both new and cached ones.

step_id instance-attribute 

step_id: int

KVPrefetchJob

Bases: TypedDict

Descriptor for prefetching the next request's received KV cache.

kv_sender_info instance-attribute 

kv_sender_info: dict[str, Any]

request_id instance-attribute 

request_id: str

NewRequestData dataclass

Payload for a newly scheduled diffusion request.

Carries the already-initialized request object so executors and workers do not re-run OmniDiffusionRequest.__post_init__ and mutate sentinel-based fields like guidance_scale_provided.

diffusion_kv_metadata class-attribute instance-attribute 

diffusion_kv_metadata: DiffusionKVMetadata | None = None

req instance-attribute 

req: OmniDiffusionRequest

request_id instance-attribute 

request_id: str

from_state classmethod 

from_state(
    state: SchedulerRequestState,
    *,
    diffusion_kv_metadata: DiffusionKVMetadata
    | None = None,
) -> NewRequestData

RequestBatchSamplingParamsKey dataclass

Request level Batch-compatibility key derived from OmniDiffusionSamplingParams.

Only request-batch-wide fields belong here. Request-local values such as seeds, generators, latent tensors, timesteps, and request-local extra_args are read per request from DiffusionRequestBatch.sampling_params_list. Structural extra_args that affect batching are listed explicitly below.

boundary_ratio class-attribute instance-attribute 

boundary_ratio: float | None = None

cfg_normalize class-attribute instance-attribute 

cfg_normalize: bool = False

condition_key class-attribute instance-attribute 

condition_key: tuple[Any, ...] | None = None

decode_noise_scale class-attribute instance-attribute 

decode_noise_scale: float | list[float] | None = None

decode_timestep class-attribute instance-attribute 

decode_timestep: float | list[float] | None = None

do_classifier_free_guidance class-attribute instance-attribute 

do_classifier_free_guidance: bool = False

eta class-attribute instance-attribute 

eta: float = 0.0

flow_shift class-attribute instance-attribute 

flow_shift: float | None = None

fps class-attribute instance-attribute 

fps: int | None = None

frame_rate class-attribute instance-attribute 

frame_rate: float | None = None

guidance_rescale class-attribute instance-attribute 

guidance_rescale: float = 0.0

guidance_scale class-attribute instance-attribute 

guidance_scale: float = 0.0

guidance_scale_2 class-attribute instance-attribute 

guidance_scale_2: float | None = None

guidance_scale_2_provided class-attribute instance-attribute 

guidance_scale_2_provided: bool = False

guidance_scale_provided class-attribute instance-attribute 

guidance_scale_provided: bool = False

height class-attribute instance-attribute 

height: int | None = None

layers class-attribute instance-attribute 

layers: int = 4

lora_int_id class-attribute instance-attribute 

lora_int_id: int | None = None

lora_scale class-attribute instance-attribute 

lora_scale: float = 1.0

max_sequence_length class-attribute instance-attribute 

max_sequence_length: int | None = None

num_frames class-attribute instance-attribute 

num_frames: int = 1

num_inference_steps class-attribute instance-attribute 

num_inference_steps: int | None = None

num_outputs_per_prompt class-attribute instance-attribute 

num_outputs_per_prompt: int = 1

output_type class-attribute instance-attribute 

output_type: str | None = None

quality class-attribute instance-attribute 

quality: str | None = None

resolution class-attribute instance-attribute 

resolution: int = 640

sample_solver class-attribute instance-attribute 

sample_solver: str | None = None

sigmas class-attribute instance-attribute 

sigmas: list[float] | None = None

strength class-attribute instance-attribute 

strength: float | None = None

true_cfg_scale class-attribute instance-attribute 

true_cfg_scale: float | None = None

use_en_prompt class-attribute instance-attribute 

use_en_prompt: bool = False

width class-attribute instance-attribute 

width: int | None = None

SchedulerRequestState dataclass

Scheduler-owned state for one queued OmniDiffusionRequest.

diffusion_kv_requests class-attribute instance-attribute 

diffusion_kv_requests: tuple[DiffusionKVRequest, ...] = ()

error class-attribute instance-attribute 

error: str | None = None

queued_at class-attribute instance-attribute 

queued_at: float = 0.0

req instance-attribute 

req: OmniDiffusionRequest

request_id instance-attribute 

request_id: str

sampling_params_key class-attribute instance-attribute 

sampling_params_key: (
    StepBatchSamplingParamsKey
    | RequestBatchSamplingParamsKey
    | None
) = None

status class-attribute instance-attribute 

status: DiffusionRequestStatus = (
    DiffusionRequestStatus.WAITING
)

is_finished 

is_finished() -> bool

StepBatchSamplingParamsKey dataclass

Denoise step level Batch-compatibility key derived from OmniDiffusionSamplingParams.

Only requests with the same key can be batched together. Fields not included here are treated as request-local and do not participate in the current homogeneous batching policy.

boundary_ratio class-attribute instance-attribute 

boundary_ratio: float | None = None

cfg_normalize class-attribute instance-attribute 

cfg_normalize: bool = False

condition_key class-attribute instance-attribute 

condition_key: tuple[Any, ...] | None = None

do_classifier_free_guidance class-attribute instance-attribute 

do_classifier_free_guidance: bool = False

fps class-attribute instance-attribute 

fps: int | None = None

frame_rate class-attribute instance-attribute 

frame_rate: float | None = None

guidance_rescale class-attribute instance-attribute 

guidance_rescale: float = 0.0

guidance_scale class-attribute instance-attribute 

guidance_scale: float = 0.0

guidance_scale_2 class-attribute instance-attribute 

guidance_scale_2: float | None = None

guidance_scale_provided class-attribute instance-attribute 

guidance_scale_provided: bool = False

height class-attribute instance-attribute 

height: int | None = None

lora_int_id class-attribute instance-attribute 

lora_int_id: int | None = None

lora_scale class-attribute instance-attribute 

lora_scale: float = 1.0

num_frames class-attribute instance-attribute 

num_frames: int = 1

num_outputs_per_prompt class-attribute instance-attribute 

num_outputs_per_prompt: int = 1

quality class-attribute instance-attribute 

quality: str | None = None

resolution class-attribute instance-attribute 

resolution: int | str | None = None

true_cfg_scale class-attribute instance-attribute 

true_cfg_scale: float | None = None

use_step_execution class-attribute instance-attribute 

use_step_execution: bool = True

width class-attribute instance-attribute 

width: int | None = None

validate_new_request_data_identity 

validate_new_request_data_identity(
    new_req: NewRequestData,
) -> None

Ensure an envelope and its forwarded request describe the same request.