Graphcore Linux PCIe Driver
===========================

When the Graphcore IPUs are directly attached to the host you will need to
install a PCIe driver. This enables the host to communicate with the IPU hardware.

To check whether the Graphcore hardware is present on the host execute
the following command:

```
$ lspci | grep -i Graphcore
```

You should see a list of devices similar to this one:

```
1a:00.0 Processing accelerators: Graphcore Ltd Device 0001
```

Note: If you do not see Graphcore hardware (the list is empty), you might be
using an IPU-POD system. For guidance on how to configure and use an IPU-POD
system please browse: https://docs.graphcore.ai/

The steps described below require `sudo` privileges.

Installing DKMS
---------------




Installing the driver
---------------------

To install the driver execute the following commands:

```
$ cd {{host_runtime_path}}/pkg
$ sudo ./driver_load.sh
```

where `{host_runtime_path}` is the path to where the driver package is located.

To confirm that the driver has installed correctly, run:

```
$ modinfo ipu_driver
```

If installation was successful, this command will display the attributes of the
driver kernel module as a list of field names and values. The following is
an example of the output and there can be differences based on Linux OS version:

```
filename:       /lib/modules/5.4.0-64-generic/updates/dkms/ipu_driver.ko
version:        1.1.6
description:    IPU PCI Driver
author:         Graphcore Limited
license:        GPL
srcversion:     43D4C6A60A3D8E2E85DCD72
alias:          pci:v00001D95d00002001sv*sd*bc*sc*i*
alias:          pci:v00001D95d00002000sv*sd*bc*sc*i*
alias:          pci:v00001D95d00000600sv*sd*bc*sc*i*
alias:          pci:v00001D95d00000003sv*sd*bc*sc*i*
alias:          pci:v00001D95d00000002sv*sd*bc*sc*i*
alias:          pci:v00001D95d00000001sv*sd*bc*sc*i*
depends:
retpoline:      Y
name:           ipu_driver
vermagic:       5.4.0-64-generic SMP mod_unload
parm:           memmap_start:array of ulong
parm:           memmap_size:array of ulong
```

Enabling/disabling driver logging
---------------------------------

By default, the Linux IPU PCIe driver does not output log information. To enable verbose
logging, write `1` to the file `/sys/bus/pci/drivers/ipu/global_log_enable`:

```
$ sudo su -
[sudo] password for user:
root:~# echo 1 > /sys/bus/pci/drivers/ipu/global_log_enable
```

An example of the output log is:

```
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.342414] ipu 0000:1a:00.0: ipu0: open(config)
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.342419] ipu 0000:1a:00.0: ipu0: IPU mmap address 0xaad00000, size 524288
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.342431] ipu 0000:1a:00.0: ipu0: user PID attached
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.342436] ipu 0000:1a:00.0: ipu0: open(exchange)
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.342438] ipu 0000:1a:00.0: ipu0: IPU mmap address 0x37fe0000000, size 536870912
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.343273] ipu 0000:1a:00.0: ipu0: open(memory)
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.343276] ipu 0000:1a:00.0: ipu0: IPU mmap address 0x400000000, size 17179869184
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.390471] ipu 0000:1a:00.0: ipu0: ICU Message response
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.576454] ipu 0000:1a:00.0: ipu0: ICU Message response
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.581451] ipu 0000:1a:00.0: ipu0: ICU Message response
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.586450] ipu 0000:1a:00.0: ipu0: ICU Message response
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.591458] ipu 0000:1a:00.0: ipu0: ICU Message response
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.597481] ipu 0000:1a:00.0: ipu0: IPU reset (3)
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.597501] ipu 0000:1a:00.0: ipu0: session's NLC correctable error count reset
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.597670] ipu 0000:1a:00.0: ipu0: attach_buffer using DMA
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.641311] ipu 0000:1a:00.0: ipu0: primary exchange addr at 0xffffc90460000000
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.641928] ipu 0000:1a:00.0: ipu0: HEXOPT: page 0 -- DMA 0xb80656a000
Dec  2 09:27:06 axbycz-0001 kernel: [9253380.641929] ipu 0000:1a:00.0: ipu0: HEXOPT: page 1 -- DMA 0xbc328fc000
```

To disable verbose logging, write `0` to the file `/sys/bus/pci/drivers/ipu/global_log_enable`:

```
$ sudo su -
[sudo] password for user:
root:~# echo 0 > /sys/bus/pci/drivers/ipu/global_log_enable
```