Text-To-Video¶

Source https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/text_to_video.

A unified script for text-to-video generation. Supports multiple models with model-aware defaults.

Supported Models¶

MAGI-2 native Preview setup, four-GPU topology and DLO choices, request constraints, and eight-GPU validation status are documented in the MAGI-2 Preview L20X recipe.

Local CLI Usage¶

Wan2.2 (default)¶

python text_to_video.py \
  --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
  --negative-prompt "<optional quality filter>" \
  --height 480 \
  --width 832 \
  --num-frames 33 \
  --guidance-scale 4.0 \
  --guidance-scale-high 3.0 \
  --flow-shift 12.0 \
  --num-inference-steps 40 \
  --fps 16 \
  --output t2v_out.mp4

Wan2.1 VACE (T2V)¶

VACE text-to-video uses this shared entrypoint. Conditional VACE tasks use the shared image_to_video.py entrypoint, which constructs the pipeline-native conditioning data from the provided media inputs. No explicit mode parameter is required.

python text_to_video.py \
  --model Wan-AI/Wan2.1-VACE-1.3B-diffusers \
  --prompt "A sleek, humanoid robot stands in a vast warehouse filled with neatly stacked cardboard boxes on industrial shelves." \
  --seed 0 \
  --height 480 \
  --width 832 \
  --num-frames 81 \
  --num-inference-steps 30 \
  --guidance-scale 5.0 \
  --flow-shift 5.0 \
  --vae-use-tiling \
  --output vace_t2v_output.mp4

LingBot-Video¶

The shared runner recognizes the official dense and MoE checkpoint IDs, selects LingBotVideoPipeline, and builds the canonical video request envelope:

python text_to_video.py \
  --model robbyant/lingbot-video-dense-1.3b \
  --prompt "a robotic arm picks up a red block" \
  --height 192 --width 320 --num-frames 9 --num-inference-steps 2 \
  --guidance-scale 3.0 --flow-shift 3.0 --fps 24 \
  --output lingbot_t2v.mp4

Use the same command with robbyant/lingbot-video-moe-30b-a3b for the MoE checkpoint. Requested frame counts are rounded upward to the causal VAE's 4n+1 grid. For a local checkpoint path whose name does not contain lingbot, pass --model-class-name LingBotVideoPipeline.

LingBot-only options such as batch_cfg and output_type use the shared model-extra channel, for example:

python text_to_video.py \
  --model robbyant/lingbot-video-dense-1.3b \
  --extra-body '{"batch_cfg": true, "output_type": "np"}'

LTX-2¶

python text_to_video.py \
  --model Lightricks/LTX-2 \
  --prompt "Cherry blossoms swaying gently in the breeze with synchronized ambient sound" \
  --output ltx2_output.mp4

See the LTX-2 recipe for all checkpoints, pipeline selection, I2V, defaults, and advanced options.

HunyuanVideo-1.5 (480p)¶

python text_to_video.py \
  --model hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v \
  --prompt "A cat walks through a sunlit garden, flowers swaying gently in the breeze." \
  --height 480 \
  --width 832 \
  --num-frames 121 \
  --guidance-scale 6.0 \
  --flow-shift 5.0 \
  --num-inference-steps 50 \
  --fps 24 \
  --output hunyuan_video_15_output.mp4

HunyuanVideo-1.5 (720p)¶

python text_to_video.py \
  --model hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v \
  --prompt "A serene lakeside sunrise with mist over the water." \
  --height 720 \
  --width 1280 \
  --num-frames 121 \
  --guidance-scale 6.0 \
  --flow-shift 9.0 \
  --num-inference-steps 50 \
  --fps 24 \
  --output hunyuan_720p.mp4

HunyuanVideo-1.5 with FP8 Quantization¶

python text_to_video.py \
  --model hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v \
  --prompt "A dog running across a field of golden wheat." \
  --quantization fp8 \
  --height 480 --width 832 --num-frames 121 \
  --guidance-scale 6.0 --flow-shift 5.0 \
  --output hunyuan_fp8.mp4

Quick test (smaller resolution, fewer frames):

python text_to_video.py \
  --model hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v \
  --prompt "A serene lakeside sunrise with mist over the water." \
  --height 320 --width 576 --num-frames 17 --num-inference-steps 30 \
  --flow-shift 5.0 \
  --output quick_test.mp4

SANA-Video-2B¶

Select the native T2V pipeline explicitly:

python text_to_video.py \
  --model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
  --model-class-name SanaVideoPipeline \
  --prompt "A cinematic tracking shot of a sailboat crossing the ocean at sunset." \
  --height 480 \
  --width 832 \
  --num-frames 81 \
  --num-inference-steps 50 \
  --guidance-scale 6.0 \
  --extra-body '{"motion_score": 30}' \
  --fps 16 \
  --output sana_video_480p.mp4

For the 720p checkpoint, switch the model to Efficient-Large-Model/SANA-Video_2B_720p_diffusers, use --height 704 --width 1280, and write to a different output path. Both native checkpoint variants are supported. The Diffusers adapter T2V compatibility path is also validated at both resolutions through online serving.

The native 480p path uses vLLM-Omni's DistributedAutoencoderKLWan wrapper; the native 720p path uses DistributedAutoencoderKLLTX2Video. Both wrappers retain their underlying Diffusers autoencoders. The pipeline intentionally uses Diffusers' checkpoint-compatible DPMSolverMultistepScheduler.

For image-to-video, use the shared image_to_video.py example with --model-class-name SanaImageToVideoPipeline and --image <path>.

Cosmos3¶

python text_to_video.py \
  --model nvidia/Cosmos3-Nano \
  --prompt "A robot arm is cleaning a plate in the kitchen." \
  --negative-prompt "blurry, distorted, low quality, jittery, deformed" \
  --height 720 --width 1280 --num-frames 189 --fps 24 \
  --num-inference-steps 35 --guidance-scale 6.0 \
  --extra-body '{"flow_shift": 10.0, "max_sequence_length": 4096, "guardrails": false,
                 "use_resolution_template": false, "use_duration_template": false}' \
  --output cosmos3_t2v.mp4

Helios (T2V)¶

Helios ships three variants. Model-specific knobs (declared in vllm_omni/model_extras/helios.py) are passed via the generic --extra-body JSON flag rather than bespoke per-model flags.

Helios-Base (Stage 1 only):

python text_to_video.py \
  --model BestWishYsh/Helios-Base \
  --prompt "A dynamic time-lapse of scenery rushing past the window of a speeding train." \
  --guidance-scale 5.0 \
  --output helios_t2v_base.mp4

Helios-Mid (Stage 2 pyramid + CFG-Zero*):

python text_to_video.py \
  --model BestWishYsh/Helios-Mid \
  --prompt "A dynamic time-lapse of scenery rushing past the window of a speeding train." \
  --guidance-scale 5.0 \
  --extra-body '{"is_enable_stage2": true, "pyramid_num_inference_steps_list": [20, 20, 20], "use_cfg_zero_star": true, "use_zero_init": true, "zero_steps": 1}' \
  --output helios_t2v_mid.mp4

Helios-Distilled (Stage 2 pyramid + DMD, few-step):

python text_to_video.py \
  --model BestWishYsh/Helios-Distilled \
  --prompt "A dynamic time-lapse of scenery rushing past the window of a speeding train." \
  --num-frames 240 \
  --guidance-scale 1.0 \
  --extra-body '{"is_enable_stage2": true, "pyramid_num_inference_steps_list": [2, 2, 2], "is_amplify_first_chunk": true}' \
  --output helios_t2v_distilled.mp4

Helios image-to-video (I2V) and video-to-video (V2V) require image/video conditioning tensors that cannot be passed through the JSON --extra-body flag; they are out of scope for this text-to-video example.

Key Arguments¶

Common¶

• --model: Diffusers model ID or local path.
• --model-class-name: Optional explicit pipeline override.
• --prompt: text description (string).
• --height/--width: output resolution. Default depends on model.
• --num-frames: number of frames. Default depends on model.
• --guidance-scale: CFG scale. Default depends on model.
• --num-inference-steps: sampling steps. Default depends on model.
• --fps: frames per second for the saved MP4.
• --output: path to save the generated video.
• --extra-body: JSON object of model-specific generation knobs, filtered against the model's declared extra_body_params and merged into sampling extra_args (see the Helios and Cosmos3 examples above and vllm_omni/model_extras).
• --vae-use-slicing: enable VAE slicing for memory optimization.
• --vae-use-tiling: enable VAE tiling for memory optimization.
• --cfg-parallel-size: set it to 2 to enable CFG Parallel. See more examples in user_guide.
• --tensor-parallel-size: tensor parallel size (effective for models that support TP, e.g. LTX2).
• --enable-cpu-offload: enable CPU offloading for diffusion models.
• --enable-layerwise-offload: enable layerwise offloading on DiT modules.
• --diffusion-offload-config: component-selective offload JSON; do not combine it with the legacy flags above.
• --frame-rate: generation FPS for pipelines that require it (e.g., LTX2).
• --audio-sample-rate: fallback audio sample rate when the pipeline returns audio.
• --quantization: quantization method (such as fp8 for FP8).
• --flow-shift: scheduler flow_shift parameter.
• --lora-path: path to PEFT LoRA adapter folder or checkpoint file.
• --lora-scale: scale factor for LoRA weights.
• --lora-backend: backend for loading LoRA adapters. Default: peft. Available options: peft, distill.

Wan2.2-specific¶

• --negative-prompt: artifacts to suppress.
• --guidance-scale-high: separate CFG scale for high-noise stage.
• --boundary-ratio: boundary split for low/high DiT (default 0.875).
• --flow-shift: scheduler flow_shift (5.0 for 720p, 12.0 for 480p).
• --cache-backend: cache_dit for acceleration.

HunyuanVideo-1.5 Optimal Configs¶

If you encounter OOM errors, try --vae-use-slicing, --vae-use-tiling, module/layer offload through --diffusion-offload-config, or --quantization fp8.

Example materials¶