# Research Status This checkpoint records what has actually been demonstrated on the C600. A tool producing an expected file or an HSP mark reaching zero is not classified as hardware execution unless a result depends on the tested tile instructions. ## Hardware-Proven - `tools/test_host_exchange.sh` generates fresh data after engine load, uploads it with a host write, transfers it with independently assembled plans, and reads it back. Hardware runs passed for 257 and 4,096 words across two tile pairs, including source and destination-guard integrity checks. - The independent IPU21 exchange-plan assembler does not use JDL headers or its code blob. Point-to-point transfers preserve nonuniform data for counts 1 through 4148 across varied source and receiver tiles. - Proper arbitrary multicast sends one payload packet and arms all receivers. Serialized and relay-tree fanout remain available as fallback strategies. - `parallel_sum` reduces values from all 1,472 tiles in 11 rounds using custom plans. `multi_tile_matmul` performs eight custom exchange passes and checks a distributed 4x4 integer matrix product. - A saved matrix executable runs in a separate process without graph creation or codelet compilation. Poplar remains the loader, command runtime, and host stream implementation on this path. - The SDK-free loader attaches through the public kernel ABI, replays the C600 initialization capture, installs the secondary bootloader, and transfers 23 batches containing all 1,472 physical tile images. Every batch and the final startup handshake complete. - Direct application execution and result readback are verified. `BareResult` computes `1234 + 5678`, stores `6912` at `0x52000`, and the loader reads it through TDI. A second image then stores and returns `3333` at the same address, excluding retained SRAM as the source of either result. - Direct SDK-style host tensor exchange is verified without Poplar at runtime. The loader boots a serialized `createHostWrite`/`createHostRead` oracle, uploads 64 host-generated bytes through attached exchange page 0, executes a tile-0 to tile-1 copy, and reads the bytes back through the same page. The device-side host-exchange schedule is SDK-generated; page management and GS1/GS2 synchronization are implemented by `direct_ipu_loader`. - The D2H payload source field is independently hardware-verified on logical tile 100. A structural image patch redirects the source to an adjacent initialized region, and poisoned direct readback returns that region exactly. Controller and host-hierarchy code remain SDK-generated in this test. - The compact IPUIMG format reconstructs all per-tile linked images with CRC32 verification. A roughly 299 MiB uncompressed matrix image set is represented by a roughly 1.9 MiB file. ## Structurally Verified Only - `ipu_image inject-exchange-plan` patches two aligned nine-word slots in the `BareHostExchange` text independently for every physical tile. The plan and operand slots reconstruct correctly, but direct supervisor execution enters `TCTXT_STATUS_EXCEPTED_NDBG` (raw TDI context state 3) at `0x4c220`. `ERERR=6` identifies an exchange-configuration exception. This failed custom sync experiment is not an execution proof. - `ipu_image inject-physical`, `inject-tile-id`, and `inject-constant` patch a relocated global independently in every physical image. Output images pass reconstruction checks; the corresponding predicates do not prove execution. - Exchange-BAR writes make `TILEUPDR` report the selected logical tile and end address for every window. A tile load has not observed fresh values, so this is not a verified H2D path. ## Disproved Assumptions - The final HSP startup handshake is not an execution oracle. With final mark 100, both a valid and a deliberately invalid redirected predicate leave mark 76. Startup and worker synchronization account for 24 events without a visible supervisor-success event. - An early constant H2D result read stale SRAM that survived reset. Fresh poison/value trials time out. Raw tile-update BAR stores are still not a verified H2D transport; the separate HSP host-tensor path is verified. - Commit `7610e58` overstated its result at the time. The later arithmetic readback test is the first proof that the redirected supervisor ran. ## Current Boundary For useful programs, the reliable path is: ```text custom exchange assembler and tile code -> Poplar build/link/relocation -> saved executable -> Poplar device loader/runtime and host streams ``` The direct path now loads, starts, exchanges host tensors, and reads results: ```text Poplar-linked per-tile images -> IPUIMG -> kernel ABI/MMIO loader -> attached host pages + generated exchange schedule + GS1/GS2 -> supervisor execution -> host-tensor readback or TDI diagnostics ``` The current host-tensor replay uses command IDs and HSP phase counts recovered from serialized executables. A live SDK oracle now verifies host write, two dependent device copies, and host read with exact data. Its supervisor program dispatches command 2 (write), command 0 (run), and command 1 (read) as distinct handlers; the run handler owns both device copies. Do not model those handlers as one linear list of control-read, payload, D2D, and D2H rows driven by a total HSP event count. Hardware testing showed that a nominal host call can consume events produced by the following row. That made the local H2D/D2H case pass while the composed H2D/D2D/D2H case returned zeros. The Rust hardware suite keeps those as separate acceptance gates. A correct runtime must reproduce command-scoped dispatch and a real device completion barrier instead of adjusting per-call phase counts until adjacent tests pass. The SDK runtime remains useful as an oracle and recovery mechanism. After a failed direct experiment, the known recovery check is: ```bash source /srv/home/gc-sdk/poplar_sdk-ubuntu_20_04-3.4.0+1507-69d9d03fd8/poplar-ubuntu_20_04-3.4.0+73-aa67dd6164/enable.sh ./run_matmul_executable /tmp/custom-matmul-final.popef ``` ## Different Approach Stop using HSP event counts to infer control flow. The target-access archive contains a direct debug-memory implementation. Disassembly shows that `IPUDebugLLD::readTileMemoryWord` stops one context, writes an SRAM address to M1, injects an `ld32`, executes it, and reads M0. The next work item is to recover its TDI register primitives and expose a read-word diagnostic in the independent loader. TDI registers are configuration-space words at `0x30000 + physicalTile * 0x40 + register * 4`. Context status is register 0, debug data is register 7, real TDI status is register 8, and status clear is register 9. `ipu21_arch_constants` extracts the fields and debugger instruction encodings from the SDK architecture object; the direct loader uses fixed recovered IPU21 values and no Graphcore library. It now implements context stop/restore, instruction injection, and stopped-context SRAM word reads. The independent loader now reaches application code. POPEF conversion omits the `.secondaryLower` reservation from transmitted payload but preserves the first module segment's linked address (`0x4c010` in the acceptance image). Loading sends 23 data batches plus the SDK's 24th all-`0xff` sentinel transaction before the mark-23 execute signal. Independent CSR injection also programs IPU21 hardware instruction breakpoints using actual architecture indices `0x73`, `0x80`, and `0x81`. The loader traps after reading the final descriptor with `$m0=0xffffffff` and at its `0x4c21c` remaining-exchange loop. Later breakpoint tracing is intrusive and can stop normal sync progress. The nonintrusive acceptance test releases startup with proxy-0 mark 1 and checks context state: the redirected self-branch leaves supervisor context 0 active on all 1,472 tiles, while the unmodified image leaves all of them inactive. Application entry is verified. The acceptance test deliberately uses two independently compiled arithmetic results at one SRAM address. This resolves entry redirection and SRAM readback without depending on Poplar host streams or ambiguous HSP event counts.