Skip to content

vllm_omni.host_weight_runtime.protocols

Provider and consumer protocols for Host Weight Runtime extensions.

ArtifactWriter

Bases: Protocol

open_tensor_file 

open_tensor_file(
    relative_name: str,
    specs: tuple[TensorWriteSpec, ...],
    *,
    ordered: bool = False,
) -> TensorFileWriter

BuildRequest dataclass

identity instance-attribute 

identity: WeightArtifactIdentity

CoordinationScope

Bases: str, Enum

ARTIFACT_BUNDLE class-attribute instance-attribute 

ARTIFACT_BUNDLE = 'artifact_bundle'

GROUP_COHORT class-attribute instance-attribute 

GROUP_COHORT = 'group_cohort'

SINGLE_PROCESS class-attribute instance-attribute 

SINGLE_PROCESS = 'single_process'

HostWeightStore

Bases: Protocol

Exact artifact access whose deadlines bound lock acquisition only.

get_or_build 

get_or_build(
    request: BuildRequest,
    producer: WeightProducer,
    *,
    validation: ValidationLevel,
    deadline: float,
) -> StoreResult

lookup 

lookup(
    identity: WeightArtifactIdentity,
    *,
    validation: ValidationLevel,
    deadline: float | None = None,
) -> StoreResult

LookupPhase

Bases: str, Enum

POST_LOAD_ONLY class-attribute instance-attribute 

POST_LOAD_ONLY = 'post_load_only'

PRE_LOAD_SAFE class-attribute instance-attribute 

PRE_LOAD_SAFE = 'pre_load_safe'

ProductionMetadata dataclass

format_metadata class-attribute instance-attribute 

format_metadata: CanonicalJson = field(
    default_factory=CanonicalJson.empty
)

producer_schema instance-attribute 

producer_schema: str

restorer_schema instance-attribute 

restorer_schema: str

ProductionSourceMode

Bases: str, Enum

CHECKPOINT_STREAM class-attribute instance-attribute 

CHECKPOINT_STREAM = 'checkpoint_stream'

FINALIZED_MODEL class-attribute instance-attribute 

FINALIZED_MODEL = 'finalized_model'

RemoteArtifactDescriptor dataclass

content_sha256 instance-attribute 

content_sha256: str

identity instance-attribute 

identity: WeightArtifactIdentity

manifest_sha256 instance-attribute 

manifest_sha256: str

provider_id instance-attribute 

provider_id: str

RemoteArtifactProvider

Bases: Protocol

provider_id property 

provider_id: str

fetch 

fetch(
    descriptor: RemoteArtifactDescriptor,
    writer: ArtifactWriter,
) -> ProductionMetadata

probe 

probe(
    identity: WeightArtifactIdentity,
) -> RemoteArtifactDescriptor | None

TensorFileWriter

Bases: Protocol

close 

close() -> None

write_rows 

write_rows(
    name: str, start_row: int, tensor: Tensor
) -> None

write_tensor 

write_tensor(name: str, tensor: Tensor) -> None

TensorWriteSpec dataclass

dtype instance-attribute 

dtype: dtype

kind class-attribute instance-attribute 

kind: TensorKind = TensorKind.PARAMETER

name instance-attribute 

name: str

nbytes property 

nbytes: int

numel property 

numel: int

role class-attribute instance-attribute 

role: str = 'weight'

shape instance-attribute 

shape: tuple[int, ...]

WeightProducer

Bases: Protocol

Synchronous V1 producer; callers do not preempt an active build.

spec property 

spec: WeightProductionSpec

produce 

produce(writer: ArtifactWriter) -> ProductionMetadata

Produce one exact artifact synchronously or raise.

WeightProductionSpec dataclass

coordination_scope class-attribute instance-attribute 

coordination_scope: CoordinationScope = (
    CoordinationScope.SINGLE_PROCESS
)

lookup_phase class-attribute instance-attribute 

lookup_phase: LookupPhase = LookupPhase.PRE_LOAD_SAFE

outputs instance-attribute 

outputs: tuple[WeightArtifactIdentity, ...]

producer_id instance-attribute 

producer_id: str

source_mode instance-attribute 

source_mode: ProductionSourceMode

WeightRestorePlan

Bases: Protocol

A validated, one-shot model mutation plan.

commit 

commit() -> None

Mutate the target model exactly once or raise.

WeightRestorer

Bases: Protocol

Validate a lease-to-model restore without mutating either input.

schema property 

schema: str

plan_restore 

plan_restore(
    model: object, lease: HostWeightLease
) -> WeightRestorePlan

Return a validated plan; all model mutation is deferred to commit().