Skip to content

vllm_omni.model_executor.models.gepard.gepard_talker

Gepard-1.0 native-AR talker.

Single-stage AR TTS on a vLLM-native Qwen3.5 backbone. Each step samples one 32-channel FSQ frame; head0 is the vLLM-facing token and the other 31 are side-channel. Generation ends on a learned binary stop head, not an EOS token. The NeMo NanoCodec decodes committed frames to a waveform outside vLLM.

Zero-shot only, and enforce_eager; CUDA graph is a perf follow-up.

CHUNK_FRAMES module-attribute 

CHUNK_FRAMES = int(
    os.environ.get("VLLM_GEPARD_CHUNK_FRAMES", "86")
)

FIRST_CHUNK_FRAMES module-attribute 

FIRST_CHUNK_FRAMES = int(
    os.environ.get("VLLM_GEPARD_FIRST_CHUNK_FRAMES", "10")
)

logger module-attribute 

logger = init_logger(__name__)

GepardTalkerForConditionalGeneration

Bases: Module

Gepard native-AR TTS talker (Qwen3.5 backbone + 32 FSQ heads + stop).

audio_embed_proj instance-attribute 

audio_embed_proj = nn.Sequential(
    nn.Linear(
        self.num_heads * cfg.audio_embed_dim, hidden
    ),
    nn.GELU(),
    nn.Linear(hidden, hidden),
    nn.LayerNorm(hidden, elementwise_affine=False),
)

audio_embeddings instance-attribute 

audio_embeddings = nn.ModuleList(
    [
        nn.Embedding(v, cfg.audio_embed_dim)
        for v in self.vocab_sizes
    ]
)

config instance-attribute 

config = cfg

fused_codebook_head instance-attribute 

fused_codebook_head = nn.Linear(
    hidden, self.total_vocab, bias=True
)

has_postprocess class-attribute instance-attribute 

has_postprocess = False

has_preprocess class-attribute instance-attribute 

has_preprocess = True

have_multimodal_outputs class-attribute instance-attribute 

have_multimodal_outputs = True

head0_vocab instance-attribute 

head0_vocab: int = cfg.head0_vocab_size

model instance-attribute 

model = Qwen3_5ForCausalLM(
    vllm_config=vllm_config,
    prefix=maybe_prefix(prefix, "model"),
)

null_prefix instance-attribute 

null_prefix = nn.Parameter(
    torch.zeros(cfg.num_speaker_prefix, hidden)
)

num_heads instance-attribute 

num_heads: int = cfg.num_audio_heads

stop_head instance-attribute 

stop_head = nn.Linear(hidden, 1, bias=True)

stop_threshold instance-attribute 

stop_threshold: float = cfg.stop_threshold

stop_token instance-attribute 

stop_token: int = cfg.stop_token

temperature instance-attribute 

temperature: float = cfg.temperature

total_vocab instance-attribute 

total_vocab: int = sum(self.vocab_sizes)

vllm_config instance-attribute 

vllm_config = vllm_config

vocab_sizes instance-attribute 

vocab_sizes: list[int] = list(cfg.audio_head_levels)

compute_logits 

compute_logits(
    hidden_states: Tensor | OmniOutput,
    sampling_metadata: Any = None,
)

embed_input_ids 

embed_input_ids(input_ids: Tensor, **_: Any) -> Tensor

forward 

forward(
    input_ids: Tensor,
    positions: Tensor,
    intermediate_tensors: IntermediateTensors | None = None,
    inputs_embeds: Tensor | None = None,
    **kwargs: Any,
) -> Tensor | OmniOutput

load_weights 

load_weights(
    weights: Iterable[tuple[str, Tensor]],
) -> set[str]

Fuse codebook_heads.{0..31} into fused_codebook_head; delegate the rest.

make_omni_output 

make_omni_output(
    model_outputs: Tensor | OmniOutput, **kwargs: Any
) -> OmniOutput

on_requests_finished 

on_requests_finished(
    finished_req_ids: set[str] | list[str],
) -> None

preprocess 

preprocess(
    input_ids: Tensor,
    input_embeds: Tensor | None = None,
    **req_infos: Any,
)