# Compiler data flow

Updated 2026-09-06 for the whole-device mid boundary and staged physical selection.
These curated diagrams supersede the older generated `ipu-stack-package-callgraph*` artifacts.

## Selection and execution

```mermaid
flowchart TD
  G[ComputeGraph: semantic operations and regions] --> C[Catalogue and geometry screening]
  C --> B[mid implementation: distributed tensor primitives]
  B --> CACHE[Compact implementation cache]
  CACHE --> SHORT[Execution-cost shortlist with geometry diversity]
  SHORT --> REGION[Compose compact candidate region]
  REGION --> COST[Geometry prices and coarse live storage]
  COST --> BEAM[Beam ranking]
  BEAM --> FINAL[Selected MidProgram: resolve recipes and deferred movement]
  FINAL --> EXPAND[low expand: enumerate tile calls and transfers]
  EXPAND --> TG[TileGraph: storage, views, calls, copies and exchanges]
  TG --> LOW[LowProgram: per-tile work lists]
  LOW --> PLACE[Provisional physical allocation]
  PLACE --> MODEL[Model mappings and shortlist complete layouts]
  MODEL --> EX[Exact scheduling and finalist selection]
  EX --> SUPPORT[Link and reserve package support storage]
  SUPPORT --> FINALPLACE[Final allocation and bounded SRAM refinement]
  FINALPLACE --> REPLAY[Replay or rebuild schedules with final addresses]
  REPLAY --> IMAGE[Tile images and final timeline cost]
```

Mid selection chooses distributed work. Low expansion realizes that work; it does
not rebuild a GEMM or attention algorithm. Cached fragments contain distributed
tensor values, not tile buffers. Final resolution inserts ordinary mid copies
where selected ownership differs, and maps claimed views directly into consumer
windows. Low projection only builds per-tile references to the expanded arenas.

## Shared prices, different precision

```mermaid
flowchart LR
  MID[Mid primitives and tensor layouts] --> GEO[Maximum local geometry]
  GEO --> PRICE[Shared primitive kernel prices]
  GEO --> COARSE[Approximate traffic and storage liveness]
  PRICE --> SCORE[Beam score]
  COARSE --> SCORE
  TILE[Expanded calls and movement] --> PRICE
  TILE --> TIME[Actual per-tile timelines]
  PRICE --> TIME
  TILE --> ALLOC[Access requirements and allocation lifetimes]
  TILE --> SCHED[Exchange scheduler]
  SCHED --> TIME
```

Beam costing never constructs tile graphs or runs physical allocation analysis.
Its exchange approximation assumes a representative fragment size rather than
walking byte spans or predicting the ready queue. Shared kernel prices avoid
maintaining separate GEMM/attention cost algorithms. Actual timelines and
scheduled phase prices remain available after expansion. Coarse memory feasibility
does not guarantee placement, particularly with disjoint ownership groups and
fragmented exchange tables.

Sources: [mid decomposition](../crates/ipu-codegen/src/mid/implementation/mod.rs),
[mid primitives](../crates/ipu-codegen/src/mid/primitive.rs),
[compact costing](../crates/ipu-codegen/src/estimate/mid.rs),
[shared kernel prices](../crates/ipu-codegen/src/estimate/primitive.rs),
[tile expansion](../crates/ipu-codegen/src/low/expand/primitive.rs),
[final timelines](../crates/ipu-codegen/src/estimate/program.rs).

## Explicit decomposition

```mermaid
flowchart LR
  L[Left materialization] --> GEMM[Distributed partial GEMM]
  R[Right materialization] --> GEMM
  GEMM --> P[Tensor with independent-partials axis]
  P --> SUM[Sum: complete or streamed contributors]
  SUM --> O[Output tensor]
```

Output-stationary GEMM instead exposes staged K panels and accumulating output
versions. Attention exposes Q/K/V copies, products, softmax and merge. Key/value packing
on a small owner grid and broadcast to the compute grid are separate mid copies. Selected
view slices are mapped copies whose output is an ordinary mid value. Tile
expansion shares physical copy realization across all these uses, including
padding, direct resident views and destination packing.

```mermaid
flowchart LR
  K[Logical keys and values] --> PACK[Copy to distributed packed tensors]
  PACK --> WINDOW[Copy each key-block window to the compute grid]
  Q[Materialized queries] --> QK[QK product]
  WINDOW --> QK
  QK --> SOFTMAX[Softmax and row state]
  SOFTMAX --> PV[Probability/value product]
  WINDOW --> PV
  PV --> MERGE[Merge output version]
```

Packing happens once before the key-block sequence. Flash key/value broadcasts
for a block are adjacent independent copies; generic low transfer consolidation
can combine them. Materialized attention delays the resident value copy until
after softmax, keeping the large resident key/value matrices in disjoint
lifetimes. This ordering is explicit in mid; low has no attention strategy builder.

**General copy/view chain composition (#4) remains deferred at the user's
request.** Discuss its overlap with planning before implementing it. Resolving
already selected deferred views is part of the current boundary rewrite; it is
not an arbitrary producer/consumer layout optimization pass.

Candidate attention dispatches retain geometry and the materialization choice once.
The mid implementation derives QK/PV kernel specifications from that geometry.
Pointwise operand selection follows tensor shapes and ownership; it has no separate
mapping-policy field. Region cost analysis returns cycles and memory directly,
while only reusable operator implementations retain `Arc<MidProgram>` values.