# Error code: E2001

**Category:** Compile Time: Unsupported RHS DataType on Hardware

This error occurs when the data type used for the **Right-Hand Side (RHS)**
operand in a matrix multiplication (e.g., `jax.lax.dot_general`, `jax.lax.conv`,
`jax.numpy.matmul`, or the `@` operator) is not natively supported by the
specific TPU generation being used.

**Sample error messages:**

```
INTERNAL: Mosaic failed to compile TPU kernel: Unsupported matmul RHS type on target: 'vector<256x256xi8>'
...

The MLIR operation involved:
%13440 = "tpu.matmul"(%13435, %13437, %13439) <dimension_numbers = #tpu.dot_dimension_numbers<...>
```

**XLA backends:** TPU

## Overview

The TPU's Matrix Multiply Unit (MXU) natively supports `Float32` operations on
all hardware generations.

However, native support for `BFloat16` and other quantized data types
(e.g., Int4, Int8, or Float8) varies by hardware generation. This error is
triggered when your kernel attempts to map a matrix multiplication to the MXU
using a data type that your specific TPU generation does not have the physical
circuitry to execute.

This error typically indicates that the compiler's **Canonicalization** pass—
which attempts to automatically convert unsupported types into supported ones
(e.g., via software emulation)—was unable to find a valid conversion rule or
was prevented from doing so because **Compatibility Mode** was disabled.

## Debugging

To resolve this error, you must align your data types with the capabilities of
your hardware. You have following options:

### 1. Cast to native types

The most reliable fix is to manually cast your operands to a hardware-supported
datatype (like `Float32` or `BFloat16` on TPU v4+) inside your kernel before the
matmul operation.

* **Why:** `Float32` is the universal data type supported natively by the
MXU on all TPU generations.
* **Trade-off:** This comes with a VPU (Vector Processing Unit) cost - the
cycles required to perform the cast, but it guarantees your kernel
will run on the current hardware.

### 2. Check Compatibility Mode
Typically the compiler can automatically handle these type mismatch issues in
**Compatibility Mode** which is enabled by default. Double check XLA configs
to make sure `--xla_mosaic_compat_mode` is not set to false.

This acts as a "polyfill," injecting software emulation sequences for operations
your hardware does not natively support.

**What Compatibility Mode enables:**

*   **Mixed-precision MatMuls:** Allows mixing Integer operands with Float
    accumulators by automatically inserting cast operations (e.g., extending
    integers to `Float32` before the matmul).
*   **Low-precision emulation:** On certain hardware generations, emulates
    unsupported types like `4-bit` floating point (`4E2M1FN`) or `8-bit`
    floating point (`8E4M3FN`) by extending them to supported types like
    `BFloat16` or `Float32` before execution.

Note that this mode prioritizes compatibility over peak performance as emulation
requires additional instructions to convert data formats before the MXU can
operate on them.

### 3. Upgrade hardware or request support

If your algorithm strictly requires native performance for types like `Int4` or
`Float8` without the overhead of casting or emulation, you will need to run on
a newer TPU generation with native support.

**Feature request:** If you believe your hardware supports this operation, or if
the compiler is missing a valid emulation path even in Compatibility Mode,
please file a feature request. We usually guarantee that operations are forward
compatible. So if your kernel runs on a TPU generation then it should run on all
future generations, but it is not guaranteed to have emulation for older
generations (for some of which the casts would be very expensive).