# SDK Bypass Notes ## Current Boundary The matrix data path no longer uses Poplar exchange code generation. The standalone loader no longer links or invokes a Graphcore userspace library. Poplar is still used at build time for: - tile SRAM allocation and relocation - compiling `matmul_codelets.cpp` - startup/runtime scaffolding and POPEF construction - construction of the original host stream metadata `multi_tile_matmul --save FILE` serializes that result. The separate `run_matmul_executable` process deserializes and runs it without constructing a graph, compiling a codelet, or invoking the graph compiler. This is the most useful short-term deployment boundary: compile once, then depend only on the Poplar device runtime. The POPEF still contains SDK-generated host-read exchange code. It does not contain an SDK-generated matrix copy: all matrix operands move through plan rows generated by `assembleMulticast` and executed by `MultiTileMatMul`. A Poplar vertex is tile-local, so "one vertex" cannot literally span the IPU. The demo presents one vertex type and one compute-set execution to Poplar, with one instance on every tile. Instances are required on inactive tiles because each tile must explicitly participate or emit the recovered nonparticipation sequence for every exchange pass. ## Below Poplar The installed kernel module is `ipu_driver` 1.4.0. Its source is installed at `/usr/src/ipu_driver-1.4.0` and exposes these character devices: ```text /dev/ipu0 configuration BAR and control ioctls /dev/ipu0_ex exchange BAR /dev/ipu0_mem reserved host memory /dev/ipu0_p2p peer-to-peer mappings ``` `tools/ipu_driver_probe.cpp` uses that ioctl ABI directly and does not link Poplar. It currently performs the safe `IPU_GET_DRIVER_VERSION` operation. The SDK also ships `libgraphcore_target_access.a`. Its exported C++ symbols include: ```text GraphcoreDeviceAccessInstance::attach GraphcoreDeviceMultiIPU::loadBinaryDuplicate GraphcoreDeviceMultiIPU::startIPUBootloadedApplications IPULoader::loadBinary IPULoader::bootloaderSignalExecute ``` However, the corresponding C++ headers are absent. Calling this library directly therefore requires reconstructing class layouts and enums, or reimplementing its bootloader and host-sync protocol over the documented kernel driver. POPEF extraction alone is insufficient: an independent runtime must also install startup state, load per-tile ELFs, start all tiles coherently, handle exceptions, and provide a result transport. ## Practical Next Step The least fragile route to a fully independent runner is: 1. Build a bare-metal executable with fixed SRAM placement and no Poplar host stream. 2. Recover the small `GraphcoreDeviceAccess`/`IPULoader` ABI needed to attach, load one duplicate ELF, start it, and inspect tile memory. 3. Replace that recovered ABI with direct `/dev/ipu0*` operations once the bootloader message and synchronization sequence is understood. The fixed-placement executable avoids implementing the full POPEF allocator and relocation machinery before basic loading and execution work. ## Compact Physical Image `popef_to_ipu_image` is the last build-time tool that reads a Poplar object. It uses the public `poplar::Module::forEachLoadableSegment` interface to obtain the fully linked SRAM contents, then writes an independent `IPUIMG1` file. The loader side only needs `ipu_image.hpp` and has no Graphcore library dependency. The format contains: ```text 40-byte header 16-byte record for each physical tile one complete template tile image per-tile sparse patches: (offset u32, size u32, bytes...) ``` Header and record integers are little endian. The header carries the load base, image size, entry point, template tile, and template CRC32. Each tile record carries its patch extent and reconstructed-image CRC32. The reader rejects invalid extents, patch ranges, truncation, and checksum failures. Poplar enumerates module segments in logical tile order, but the device loader addresses physical tile IDs. For IPU21 the recovered permutation is: ```text pair = logical / 2 lane = logical & 1 block = pair / 23 row = pair % 23; reverse row when block is odd column = (block / 2) * 4 + (block & 1) physical = row * 64 + column + lane * 2 ``` For the distributed matrix multiply, 1,472 images totaling about 299 MiB when stored separately become a 1,885,475-byte IPUIMG. Reconstruction checks passed for every tile. `ipu_image pack` can also construct the format from extracted physical `t_N.elf` files, and `ipu_image extract` emits one raw load image. This solves storage and per-tile fixups. `direct_ipu_loader` now consumes the format, installs the secondary tile bootloader, and feeds all physical images. An unmodified linked image enters Poplar's resident command interpreter after the final loader handshake; that handshake alone does not execute the graph's matrix program. `ipu_image redirect` preserves the initial `nop; sync3` rendezvous and replaces the following two words with `setzi $m0, target; br $m0`. The target is recovered from a unique two-word marker emitted inside a bare supervisor function and skips that function's repeated marker and `sync3` prefix. This is an experimental entry patch; execution at the target has not yet been observed through an independent result channel. ## Direct Attach And Autoloader `direct_ipu.hpp` opens `/dev/ipu0`, maps the 512 KiB configuration BAR, issues `IPU_USER_ATTACH` and `IPU_SET_IPU_ID`, and provides bounds-checked MMIO. The `direct_ipu_loader inspect` command validates an IPUIMG before attaching and reports the hardware-autoloader state without linking a Graphcore library. The stripped runtime's configuration writes were traced at the common PCIe write function. Register numbers are byte offsets into the mapped BAR. For the 948-byte IPU2 bootloader padded to 1 KiB, the hardware-autoloader sequence is: ```text AUTOLDCSR (0x2068) = 0 AUTOLDDATAR (0x2070) = 256 bootloader/padding words AUTOLDTRGTR (0x206c) = 0x02000400 AUTOLDCSR (0x2068) = 0x0010013f; wait for GO bit 8 to clear AUTOLDTRGTR (0x206c) = 0x0209c001 AUTOLDCSR (0x2068) = 0x000b7f3f; wait for GO bit 8 to clear ``` `direct_ipu_loader bootloader TILE_BOOTLOADER_ELF` parses and checks the ELF, derives the corresponding range fields for a 1-4 KiB image, blocks the driver's real-time interrupt signals, and performs that sequence. It completed on the C600 for all 1,472 physical tiles. A normal precompiled Poplar matrix-multiply run passed immediately afterward, confirming that the sequence completed and left the device recoverable. The `load` command owns initialization from a driver-idle device. It replays the recovered ICU mailbox/reset/clock sequence, then applies the ordered 2,003 MMIO writes in `artifacts/c600-init.ipucfg`. The artifact has this independent little-endian format: ```text offset size meaning 0 8 "IPUCFG1\0" 8 4 record count 12 8*N ordered (configuration offset u32, value u32) records ``` The checked-in image was captured from Poplar 3.4 on the C600 with firmware 2.7.1 and has SHA-256 `1838395ba3b291696ae9d2b5656725e590ecef419f55eb23cac4aad336b76df0`. It should be treated as target-specific until the fields have been reduced to a generated configuration model. `tools/gdb_config_capture.py` reproduces it; `ipu_ioctl_trace.so` records the five-word ICU mailbox operations. ## Secondary Loader Frames The secondary bootloader reads a pinned host buffer containing a descriptor table followed by fixed 1 KiB frames. The first 512 bytes contain 64 entries: ```text offset size meaning 0 4 byte offset of this tile's first frame in the pinned buffer 4 4 number of frames for this tile ``` Unused bytes through offset `0x1000` are zero. Frame storage starts at `0x1000`; each descriptor points at one contiguous framed tile image. A frame has this layout: ```text offset size meaning 0 4 zero-based chunk index 4 4 physical tile ID 8 4 flags; final frame has byte 11 set to 8 12 4 reserved, zero 16 992 consecutive raw tile-image bytes, zero padded at the end 1008 16 0xff trailer ``` For the current 212,980-byte load image this is 215 frames per tile. The `ipu_bootloader_frames` tool reconstructs physical images from IPUIMG and emits that stream for any contiguous physical-tile range. Its complete 220,160-byte output for physical tile 1409 compared byte-for-byte equal to a stopped SDK loader's pinned buffer before execution. The SDK uses HSP general-sync register 1 at configuration offset `0x302c` for the transfer handshake. Its `MARK` field is bits 0-15: every 64-tile batch sets mark 1 and waits for the bootloaders' collective response to return it to zero. After 23 batches, setting mark 23 starts all 1,472 images and again waits for a collective zero. Both the eight primary and eight secondary exchange-buffer controls must be armed with `0x07fffffa`; omitting the secondary controls or the descriptor table produces incomplete synchronization rather than a clean transport error. The direct loader passed the complete 23-batch load and final startup handshake on the C600. A minimal redirected `signal` image also reaches zero from the existing 23-domain host-sync mark, but this is not proof that its supervisor body executes. A deliberately unsatisfiable redirected predicate consumes the same startup events. With an initial mark of 100 both variants stop at 76, showing that 24 startup/worker events are visible independently of predicate success. ## Experimental Tile SRAM Writes IPU21 tile-update completion requires both PCI address translation and a logical tile window. The current experimental sequence is: ```text IPU_SET_HEXOPT_IDENTITY_TABLE ioctl EXCHWINDOWBASER (0x3044) = logical tile; HEXOPT bit 11 clear read EXCHWINDOWBASER back to flush the posted config write store at exchange BAR + 0x80000 + (tile address - 0x4c000) poll TILEUPDR (0x3054) LASTTILEID[27:17] == logical tile LASTADDR[16:0] == ((BAR-relative address + size - 1) >> 3) ``` The exchange window index is logical. Bootloader frame destinations and the tile `TILE_ID` CSR are physical, using the serpentine permutation documented above. `--tile-id-input ADDRESS BIAS` converts each logical window to its physical ID and writes `physical + BIAS`; `--constant-input ADDRESS VALUE` writes one strict control pattern to every tile. `TILEUPDR` reaches the expected logical tile and end address for all 1,472 windows. This is not yet a verified H2D channel. A fresh poison/value test at free SRAM `0x52000` does not satisfy a running vertex load, for either raw or `+0x80000` aperture variants; a pass seen without poisoning was caused by stale SRAM surviving reset. Writes before final release may also be replaced by bootloader finalization. The `--constant-input` and `--tile-id-input` options are retained as diagnostics and currently time out under the strict bare checks. Direct SRAM readback or recovery of the debug-interface sequence is needed to resolve the remaining address/coherency issue. The post-start test waited a fixed 100 ms before writing, not for a tile-side "ready for input" acknowledgement. It ordered `EXCHWINDOWBASER` with a PCI configuration readback, used host sequential-consistency fences around BAR stores, and polled `TILEUPDR` through the final beat. It did not run a confirmed tile-side synchronization barrier after the write. A proper test must have the application acknowledge readiness, accept the host update, cross a documented tile/exchange synchronization point, and only then consume and acknowledge the new SRAM value. The earlier apparent bare-execution result was therefore a false positive. The same applies to an early H2D pass: stale SRAM survived reset, while a fresh poison/value test times out. Neither result should be used as evidence for direct execution or host data transport. ## Debug Readback Pivot The next direct-runtime milestone is an unambiguous observation channel rather than another HSP-count experiment. `libgraphcore_target_access.a` contains `IPUDebug.cpp.o` and `IPUDebugLLD.cpp.o`. Disassembly of `IPUDebugLLD::readTileMemoryWord` recovers this sequence: ```text validate tile/thread/address stop the selected worker thread write requested address to M1 generate and inject an ld32 instruction using M1 as the address execute that instruction on the stopped thread read the result from M0 restore the thread state ``` This does not depend on a readable exchange BAR. The remaining primitives are `stopThread`, `writeMRegister`, `executeInstruction`, `readMRegister`, and the underlying `readTDIRegister`/`writeTDIRegister` operations. Recovering those register accesses provides one oracle for all three uncertain boundaries: whether the entry patch ran, whether an exchange-BAR store reached SRAM, and whether a result word was produced. The first test must write distinct success and failure sentinels into a known image global and read that word back; HSP state is only a liveness diagnostic in that test. The first layer is recovered and exposed read-only by `tdi-dump`. For IPU21, the target-access architecture constants resolve the configuration address to: ```text TDI register address = 0x30000 + physicalTile * 0x40 + register * 4 TDI_DBG_DATA register index = 7 ``` `ipu21_arch_constants` extracts these values from `libipu_arch_info.so`; the direct loader uses the resolved IPU21 constants and no Graphcore library. Read-only dumps succeeded on physical tiles 0 and 1471. The recovered register indices are: ```text 0 CTXT_STS 5 DBG_ECSR 9 TDI_CLR 1 DBG_RBRK 6 DBG_ECLR 10 INCOMING_DCOUNT 2 DBG_RBRK_VERT 7 DBG_DATA 11 EXCHANGE_CTL 3 DBG_IEXEC 8 TDI_STS 12 TDI_CTL 4 DBG_IOWNER ``` `DBG_RBRK` uses bits 0-6 for context selection, bit 30 for ATOV, and bit 31 for VM enable. `TDI_STS` uses bit 0 for context-not-quiescent, bit 1 for invalid instruction, bit 2 for double issue, and bit 3 for busy. Reset-idle tiles have all seven contexts inactive and `TDI_STS=0`; an inactive context cannot execute an injected instruction. The direct implementation now stops an active context, writes the instruction owner and instruction, waits for true TDI completion, and restores the prior run-break state. The SDK debugger's injectable NOP is `0x44f0f3f1` (`zero $m15`), not the compiler's ordinary `nop` encoding. Stopped-context SRAM readback is also independent. It writes an address to `DBG_DATA` and injects these IPU21 instructions: ```text 0x41010070 get $m1, $C_DBG_DATA 0x01f01000 ld32 $m0, $m15, $m1, 0 0x43008070 put $C_DBG_DATA, $m0 ``` An earlier `load-bare-trace` diagnostic armed a supervisor run break before final release. It reliably entered debug-excepted state on tile 0, proving that the secondary loader context is activated. At that stop, later image data reads back correctly, while the low SRAM region containing the patched entry and target still contains bootloader instructions. The protected low region is therefore replaced during the final loader handoff, after this first observable stop. Two loader details were missing from the first independent implementation: - The POPEF module API omits the zero-fill `.secondaryLower` PT_LOAD segment. This reservation is not transmitted, but it still occupies address space. SDK frame payload starts with the first segment byte and IPU21 installs it at its linked address (`0x4c010` here). Labelling that byte as `0x4c000` made redirect and branch targets 16 bytes too low. - After the 23 64-tile data batches, the SDK fills the 40 MiB transport buffer with `0xff` and performs one more mark-1/wait-zero transaction. Only then does it signal execution with mark 23. The direct loader now reproduces all 24 mark-1 transactions. The IPU21 hardware instruction breakpoint is independently programmable while the supervisor is stopped. The recovered supervisor CSR indices are `DBG_ECSR=0x73`, `DBG_IBRK0_CTL=0x80`, and `DBG_IBRK0_PC=0x81`; notably, the generated MAX header reports `DBG_ECSR=0x74`, which is not the instantiated IPU21 value. A breakpoint at `0x4c118` confirmed that the final descriptor word arrives as `0xffffffff`, and one at `0x4c21c` confirmed entry into the loader's 22-exchange wait loop. Breakpoint exceptions alter observable supervisor state and can stall this synchronization sequence. The invasive diagnostic was removed after application startup was verified; it remains in git history. The supported-runtime oracle for the same device-target `BareSignal` POPEF resolved the image-layout ambiguity. Both the first batch (physical tiles 0-63) and final batch (1408-1471), including complete 40 MiB transport buffers, compare byte-for-byte equal to independently generated buffers. The startup sequence rewrites `EXCHWINDOWBASER` and sets mark 1 on proxy 0; the SDK's following wait targets proxy 1, not proxy 0. Reproducing that sequence starts the redirected self-branch on all 1,472 tiles. Supervisor context 0 is active on every tile, while the byte-identical unmodified image has no active supervisor under the same host sequence. The loader's former exchange-buffer constants are `XB.DCXCR`, not `XB.CSR`. The SDK saves all 16 controls, programs `DCXHWM=2` and each `DCXnXREQID=63` during secondary loading, and restores request IDs 0 for the primary PCI interface and 1 for the secondary interface. It also clears `PCI_COMPLEX.CCSR.NOSNOOP` and `ELINKECRCEN` while loading. The direct loader now constructs those words from named fields and generated instance addresses. Application execution is result-verified rather than inferred from context state. `BareResult` stores `6912` at `0x52000`, and `BareResultAlternate` subsequently stores `3333` at the same address. Both values are read through the independent TDI path with all supervisor contexts active. `ipu_ioctl_trace.so` can optionally dump a pinned buffer just before detach by setting `IPU_IOCTL_TRACE_DUMP` to a directory. This is diagnostic only: by detach time the secondary bootloader has replaced consumed frames with `0xff`, so pre-execution debugger snapshots remain the framing oracle. ## Host Tensor Exchange `createHostWrite` and `createHostRead` do not use TDI or direct tile-update stores. The SDK compiles host-exchange packets into the resident device schedule, pins two 4 KiB host pages, and uses HSP general-sync group 2 to hand ownership of each phase between host and device. In the minimal oracle: ```text attached page 0 + 0x40 64-byte tensor payload attached page 1 + 0x00 callable ID (0=run, 1=read, 2=write) GS1 control 0x302c startup release GS2 control 0x3030 host-exchange phase handoff ``` The load transition is order-sensitive. After mark 23 reaches zero, the SDK restores operational `XB.DCXCR` and `PCI_COMPLEX.CCSR`, selects HEXOPT, detaches the 40 MiB loader buffer, posts GS1 mark 1, waits for the preloaded GS2 mark to reach zero, attaches page 1 and then page 0 (reselecting HEXOPT before each), and posts one GS2 load-completion mark. The first host call waits for that mark to return to zero. For `sdk_host_tensor_oracle`, write, run, and read contain four, two, and four GS2 phases respectively. These executable-specific values now live in `artifacts/sdk_host_tensor.ipuhsp`. `direct_ipu_loader host-exchange` consumes that manifest and an arbitrary sequence of named calls without loading any Graphcore host library. The hardware test supplies randomized input and checks the returned 64-byte file after the tile-0 to tile-1 copy. The runtime supports arbitrary page sizes, attach order, command location, phase counts, and multiple input/output slices. Static extraction from the serialized Cap'n Proto `CoreHostBufferInfo` and `CoreHostSyncInfo` remains build-time work.