2026-09-06T16:49:05.703277Z  INFO retained operator-plan finalist finalist=0 values=11 operations=7 estimated_cycles=609851 estimated_exchange_cycles=188600 exchange_row_bytes=41780 peak_standard_bytes=131892 peak_interleaved_bytes=155648 peak_total_bytes=238388 maximum_standard_allocation_bytes=32768 active_tile_counts={12, 64} padding_group_counts={} conversion_sources=[]
2026-09-06T16:49:05.709286Z  INFO ipu_codegen.package.phase{phase="lower_mid"}: package build phase finished phase="lower_mid" elapsed_ms=448 success=true
2026-09-06T16:49:06.157107Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: screened tile mappings by exchange resource load candidates=5 baseline_cycles=73536 candidate_cycles=73536 baseline_pressure=66310 candidate_pressure=66310
2026-09-06T16:49:06.159888Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: modelled expanded operator plan finalist=0 modelled_cycles=445899 estimated_cycles=677323 estimated_exchange_cycles=307960
2026-09-06T16:49:06.533109Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: prepared large physical exchange phase phase=1 transfers=1148 maximum_identical_destinations=1
2026-09-06T16:49:06.533151Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: optimized physical exchange schedule phase=1 initial_horizon=6419 selected_horizon=6419 endpoint_lower_bound=5832 lower_bound_gap=587 selected_kind="full-duplex" neighborhood_improvements=0
2026-09-06T16:49:06.533953Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: finished large physical exchange phase phase=1 tile=4 row_words=1503 horizon=6419
2026-09-06T16:49:06.843746Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: scheduled operator-plan finalist finalist=0 mapped=false analytical_cycles=677323 analytical_exchange_cycles=307960 scheduled_exchange_cycles=77613 refined_cycles=446976
2026-09-06T16:49:06.844247Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: selected physically scheduled operator-plan finalist selected=0 mapped=false
2026-09-06T16:49:06.844260Z  INFO ipu_codegen.package.phase{phase="select_finalist"}: package build phase finished phase="select_finalist" elapsed_ms=1134 success=true
2026-09-06T16:49:06.844281Z  INFO lowered graph for package construction logical_shards=911 exchange_phases=5
2026-09-06T16:49:06.844514Z  INFO ipu_codegen.package.phase{phase="compile_runtime"}: package build phase finished phase="compile_runtime" elapsed_ms=0 success=true
2026-09-06T16:49:06.844671Z  INFO ipu_codegen.package.phase{phase="plan_kernels"}: package build phase finished phase="plan_kernels" elapsed_ms=0 success=true
2026-09-06T16:49:06.844907Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: compiling kernel source=/srv/home/gc-sdk/ipu-stack/crates/ipu-tests/../../device/gemm_f16_amp.S name="gemm_f16_interleaved_k80_c768_r11_r12" target=ipu21
2026-09-06T16:49:06.913388Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: kernel artifact created object=/srv/home/.cache/ipu-stack/kernels/5bbfc8f106587b6c506e049bc4c4215fefe31bd85e454c8c80a4feb3350a126e/kernel.o metadata=/srv/home/.cache/ipu-stack/kernels/5bbfc8f106587b6c506e049bc4c4215fefe31bd85e454c8c80a4feb3350a126e/kernel.json
2026-09-06T16:49:06.913789Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: compiling kernel source=/srv/home/gc-sdk/ipu-stack/crates/ipu-tests/../../device/gemm_f16_amp.S name="gemm_f16_interleaved_k768_c80_r11_r12" target=ipu21
2026-09-06T16:49:06.982735Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: kernel artifact created object=/srv/home/.cache/ipu-stack/kernels/2cd4b9666ca46dcf26405b60541a5674d1969cd71e8324f3f862756c5a27f492/kernel.o metadata=/srv/home/.cache/ipu-stack/kernels/2cd4b9666ca46dcf26405b60541a5674d1969cd71e8324f3f862756c5a27f492/kernel.json
2026-09-06T16:49:06.983582Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: compiling kernel source=/srv/home/gc-sdk/ipu-stack/crates/ipu-tests/../../device/attention_softmax_f16.S name="attention_softmax_d72_p768_tail" target=ipu21
2026-09-06T16:49:07.041007Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: kernel artifact created object=/srv/home/.cache/ipu-stack/kernels/2c491947a5db2cf9c8c5acabdca65da5317e10f948021ed4a665cbde220359fb/kernel.o metadata=/srv/home/.cache/ipu-stack/kernels/2c491947a5db2cf9c8c5acabdca65da5317e10f948021ed4a665cbde220359fb/kernel.json
2026-09-06T16:49:07.041451Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: compiling kernel source=/srv/home/gc-sdk/ipu-stack/crates/ipu-tests/../../device/attention_merge_f16.S name="attention_merge_v72_p80_k768" target=ipu21
2026-09-06T16:49:07.096474Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: kernel artifact created object=/srv/home/.cache/ipu-stack/kernels/8d1aae7d28b9a0d8a69ee6d42e45590b9b487e77feedf9945c78f65930d91863/kernel.o metadata=/srv/home/.cache/ipu-stack/kernels/8d1aae7d28b9a0d8a69ee6d42e45590b9b487e77feedf9945c78f65930d91863/kernel.json
2026-09-06T16:49:07.096594Z  INFO ipu_codegen.package.phase{phase="compile_kernels"}: package build phase finished phase="compile_kernels" elapsed_ms=251 success=true
2026-09-06T16:49:07.097120Z  INFO ipu_codegen.package.phase{phase="link_runtime"}: package build phase finished phase="link_runtime" elapsed_ms=0 success=true
2026-09-06T16:49:07.121980Z  INFO ipu_codegen.package.phase{phase="size_tile_code"}: package build phase finished phase="size_tile_code" elapsed_ms=2 success=true
2026-09-06T16:49:07.122022Z  INFO allocated package support memory linked_end=320796 profile_bytes=0 exchange_table_bytes=14124 host_code_bytes=1076 generated_code_bytes=1764 code_address=321880 standard_ranges=[(393216, 524288)]
2026-09-06T16:49:07.124592Z  INFO ipu_codegen.package.phase{phase="place_storage"}: package build phase finished phase="place_storage" elapsed_ms=2 success=true
2026-09-06T16:49:07.131414Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: reused exchange optimization after validating relocated rows phase=0
2026-09-06T16:49:07.263935Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: reused exchange optimization after validating relocated rows phase=1
2026-09-06T16:49:07.264584Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: prepared large physical exchange phase phase=1 transfers=1148 maximum_identical_destinations=1
2026-09-06T16:49:07.264599Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: optimized physical exchange schedule phase=1 initial_horizon=6419 selected_horizon=6419 endpoint_lower_bound=5832 lower_bound_gap=587 selected_kind="reused" neighborhood_improvements=0
2026-09-06T16:49:07.265410Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: finished large physical exchange phase phase=1 tile=4 row_words=1503 horizon=6419
2026-09-06T16:49:07.275822Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: reused exchange optimization after validating relocated rows phase=2
2026-09-06T16:49:07.342101Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: reused exchange optimization after validating relocated rows phase=3
2026-09-06T16:49:07.357558Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: reused exchange optimization after validating relocated rows phase=4
2026-09-06T16:49:07.357974Z  INFO ipu_codegen.package.phase{phase="lower_exchanges"}: package build phase finished phase="lower_exchanges" elapsed_ms=233 success=true
2026-09-06T16:49:07.388758Z  INFO ipu_codegen.package.phase{phase="optimize_exchange_placement"}: package build phase finished phase="optimize_exchange_placement" elapsed_ms=30 success=true
2026-09-06T16:49:07.388944Z  INFO costed final placed program final_cycles=446976 final_exchange=77613
2026-09-06T16:49:07.398463Z  INFO ipu_codegen.package.phase{phase="prepare_tile_code"}: package build phase finished phase="prepare_tile_code" elapsed_ms=3 success=true
2026-09-06T16:49:07.401968Z  INFO ipu_codegen.package.phase{phase="emit_tile_code"}: package build phase finished phase="emit_tile_code" elapsed_ms=3 success=true
2026-09-06T16:49:07.531957Z  INFO ipu_codegen.package.phase{phase="build_tile_images"}: package build phase finished phase="build_tile_images" elapsed_ms=128 success=true
2026-09-06T16:49:07.538000Z  INFO writing application package tiles=1472
2026-09-06T16:49:07.551805Z  INFO application package written
2026-09-06T16:49:07.552767Z  INFO reading application package
2026-09-06T16:49:07.570145Z  INFO application package read tiles=1472 compiler=0.1.0
2026-09-06T16:49:07.570429Z  INFO waiting for hardware ownership path=artifacts/layout-sweep/device.lock
2026-09-06T16:49:16.767192Z  INFO acquired hardware ownership path=artifacts/layout-sweep/device.lock
2026-09-06T16:49:16.767249Z  INFO opening IPU device path="/dev/ipu0"
2026-09-06T16:49:16.767304Z  INFO IPU device attached path=/dev/ipu0
2026-09-06T16:49:16.767308Z  INFO initializing IPU device
2026-09-06T16:49:16.776915Z  INFO IPU device initialized
2026-09-06T16:49:16.776934Z  INFO replaying device configuration register_writes=2003
2026-09-06T16:49:16.784811Z  INFO loading application tile_count=1472 final_mark=23
2026-09-06T16:49:16.977011Z  INFO application loaded and initial synchronization completed tile_count=1472
2026-09-06T16:49:17.147527Z  INFO starting host exchange session startup_mark=23
2026-09-06T16:49:17.147557Z  INFO attaching host exchange pages pages=2
2026-09-06T16:49:17.148276Z  INFO host exchange session started
2026-09-06T16:49:17.148284Z  INFO invoking streaming host exchange call call="initialize" command=0 phases=24
2026-09-06T16:49:17.152491Z  INFO invoking streaming host exchange call call="run" command=0 phases=5
checkpoint=0 operation=0 tensors=1 samples=256 maximumAbsoluteError=0.000244
checkpoint=1 operation=1 tensors=1 samples=256 maximumAbsoluteError=0.000244
checkpoint=2 operation=2 tensors=1 samples=256 maximumAbsoluteError=0.000244
2026-09-06T16:49:30.359986Z ERROR diagnostic run failed error=device timeout: host call run phase 4/5: device timeout: HSP register 0x3030: expected 0, observed 1 completed=3 device=counts=[1408, 64, 0, 0] firstUnexpected=[(0, 1), (2, 1), (64, 1), (66, 1), (128, 1), (130, 1), (192, 1), (194, 1), (256, 1), (258, 1), (320, 1), (322, 1), (384, 1), (386, 1), (448, 1), (450, 1)] contexts=[(0, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 23)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (2, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 23)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (64, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 23)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (66, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 23)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (128, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 23)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (130, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (192, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (194, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (256, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (258, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (320, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (322, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (384, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (386, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (448, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))]), (450, 1, Err(Timeout("stopping tile context")), None, None, None, None, Err(Timeout("stopping tile context")), Ok((0, 19)), Ok(None), Ok([3, 3, 3, 3, 3, 3]), [(1, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (2, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (3, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (4, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (5, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74")))), (6, Ok((319228, Some("ipu_stack_attention_softmax_d72_p768_tail_f16+0x74"))))])]
Error: device timeout: host call run phase 4/5: device timeout: HSP register 0x3030: expected 0, observed 1