vllm_omni.model_executor.models.personaplex.personaplex_embeddings ¶

PersonaPlex input embeddings (Moshi embed_codes).

The temporal transformer (:class:HeliumForCausalLM) consumes precomputed inputs_embeds rather than token ids. Those embeddings come from Moshi's embed_codes: a per-frame [B, 1 + n_q, S] token stack (row 0 = text, rows 1..n_q = audio codebooks) is mapped through one text embedding plus n_q audio-codebook embeddings and summed into [B, S, hidden].

This module owns exactly those embedding tables (Moshi text_emb and emb.0..n_q-1), which modeling_helium deliberately omits. The talker (Task 2) uses it to build the temporal inputs_embeds for both the system / voice-prompt prefill and the per-frame decode step.

PersonaPlexInputEmbeddings ¶

Bases: Module

Moshi embed_codes: summed text + audio-codebook embeddings.

Parameters:

Name	Type	Description	Default
`config`	`PersonaPlexConfig`	the top-level PersonaPlex config (supplies the temporal hidden size, the audio cardinality / codebook count, and the text table size).	required

audio_emb `instance-attribute` ¶

audio_emb = nn.ModuleList(
    [
        _ScaledEmbedding(config.audio_vocab_size + 1, dim)
        for _ in range(self.num_audio_codebooks)
    ]
)

num_audio_codebooks `instance-attribute` ¶

num_audio_codebooks = config.num_audio_codebooks

text_emb `instance-attribute` ¶

text_emb = _ScaledEmbedding(config.text_embedding_rows, dim)

forward ¶

forward(sequence: Tensor) -> Tensor

Map a [B, 1 + n_q, S] token stack to [B, S, hidden].

Row 0 is the text stream; rows 1..n_q are the audio codebooks (in Moshi's layout the audio rows carry both the agent and user streams).

load_weights ¶

load_weights(
    weights: Iterable[tuple[str, Tensor]]
    | Mapping[str, Tensor],
) -> set[str]

Load Moshi text_emb and emb.0..n_q-1 embedding tables.