Diffusion parallelism¶

Parallelism owns diffusion rank topology, process groups, tensor and sequence sharding, collectives, and distributed execution.

Candidate invariants¶

PARALLEL-INV-001: Topology has one source of truth¶

Rule: Rank coordinates, group membership, and parallel dimensions MUST be derived from validated configuration.

PARALLEL-INV-002: Sharding contracts are explicit¶

Rule: Every distributed boundary MUST define tensor shape, sharded dimension, placement, dtype, producer, and consumer.

PARALLEL-INV-003: Collectives are symmetric¶

Rule: Process-group members MUST invoke compatible collectives in the same logical order.

PARALLEL-INV-004: Single-rank behavior remains valid¶

Rule: Distributed implementations SHOULD preserve a supported single-rank path without distributed initialization.

Safe-change guide¶

Test topology validation, single-rank execution, affected parallel modes, combined modes, invalid shapes, and orderly teardown.