Skip to content

vllm_omni.config.endpoint_policy

Endpoint restriction policy for omni pipelines.

EndpointRestriction dataclass

capability instance-attribute 

capability: OmniServingCapability

reason instance-attribute 

reason: str

OmniServingCapability

Bases: Enum

Serving capabilities that pipelines can shut down.

COMPLETIONS class-attribute instance-attribute 

COMPLETIONS = RouteTarget(
    "/v1/completions", frozenset({"POST"})
)

IMAGE_EDITS class-attribute instance-attribute 

IMAGE_EDITS = RouteTarget(
    "/v1/images/edits", frozenset({"POST"})
)

methods property 

methods: frozenset[str]

path property 

path: str

RouteTarget

Bases: NamedTuple

A server path & supported methods.

methods instance-attribute 

methods: frozenset[str]

path instance-attribute 

path: str

build_rejection_handler 

build_rejection_handler(reason: str)

Build a rejection handler for a given endpoint for the provided reason.

shutdown_unsupported_routes 

shutdown_unsupported_routes(
    app: FastAPI,
    endpoint_restrictions: tuple[EndpointRestriction, ...],
)

Given an initialized FastAPI server instance and a set of model specific endpoint restrictions, remove the restricted routes and patch a handler that returns 400.