# Include what you use `include-what-you-use` (`IWYU`) is a program used to remove superfluous includes, add missing includes and suggest forward declarations where appropriate in C++ files. ## Setup `IWYU` needs to know where the build directory is, so that it can access the include directories of the view. This information is taken from the `POPLAR_VIEW_BUILD_DIR` environment variable which needs to be set by the user. No additional setup should be necessary. The easiest way to run the linter is through the `scripts/lint/linters/iwyu/standalone_iwyu.sh` script that you can call directly. This will run IWYU on every `cpp` and `hpp` file in the repository other than a few files that either shouldn't be touched or depend on generated files during compilation. `IWYU` is also set up as a `pre-commit` hook for PopART, though disabled by default. To run it as a `pre-commit` hook, uncomment the relevant lines in `.pre-commit-config.yaml`, and can be directly called in one of the following ways from the home directory: ```sh # Runs on all of PopART pre-commit run iwyu --all-files # Runs on specific files pre-commit run iwyu --files ... # Runs on staged files pre-commit run iwyu ``` If IWYU identifies issues, you should not trust it blindly, as it can often have false positives. By default IWYU only prints a list of issues, but doesn't attempt to fix them. Once you look over the suggested fixes, you can make the linter fix them automatically in the next run by specifying the `AUTOFIX_IWYU=1` environment variable. While this approach should just work most of the time, sometimes IWYU suggestions can break compilation / functionality and you might need to make sure that certain headers are kept by using IWYU pragmas / mappings files as described below. Using `IWYU` with `popsdk` is not supported at the moment; you can instead skip the hook by setting the environment variable `SKIP=iwyu`. ## Fixing suggestions As `IWYU` has no way to know for sure which headers belong where, it doesn't automatically fix it's suggestions, instead requiring them to be applied manually. The user can then decide whether the changes are acceptable, or whether they should update the mappings to disable the suggestion (as explained below) In most cases there won't be many changes needed. `IWYU` provides line numbers for removal suggestions, and reasons for including files in a very nice human readable format, hence making changes is relatively simple. Alternatively if you agree with all IWYU suggestions, you can specify the `AUTOFIX_IWYU=1` environment variable to have the script fix the issues for you. Be careful when fixing autogenerated files (ones that take the form `*.gen.[hc]pp`). You should change the includes that are written when generating them (most likely in `gen_operators.hpp`), and run the generating script instead of directly changing the includes; this way you make sure that the IWYU fixes persist after regenerating the files. ## Dealing with undesirable suggestions If `IWYU` makes undesirable suggestions for a particular file you have several options to deal with them. ### Exclude file from being linted If you think that `IWYU` should not run on the file (for example if the file is just a collection of header includes for legacy reasons), you can exclude it in `.pre-commit-config.yaml` by adding it to the `excludes` list under the `iwyu` hook. Please also add the file to the `excluded_files` array in the `standalone_iwyu.sh` script. ### Use mappings You can also add a new mapping in one of the `*.imp` files in `scripts/lint/linters/iwyu/` to specify where certain symbols should / could be included from. The following mapping rules are available to use: * `{ include: ["a.hpp", "public", "b.hpp", "public"] }` specifies that if `b.hpp` is present, `a.hpp` should not be included. But `a.hpp` can be included when `b.hpp` is not included. * `{ include: ["a.hpp", "private", "b.hpp", "public"] }` specifies that `a.hpp` should always be included through `b.hpp`. * `{ symbol: ["some::symbol", "private", "b.hpp", "public"] }` specifies that `some::symbol` should always be included through `b.hpp` * `{ ref: "c.imp" }` allows us to include rules from `c.imp` Additionally regex patterns are supported for the include case when the string starts with an `@` symbol: ```text # Matches any header that starts with '", "public", "", "public"] }, # Unfortunately it seems that doing regex on brackets / quotes part of the include is not supported # This doesn't work: #{ include: ["@[\"<]popart/onnxoperators.gen.hpp[\">]", "private", "\"popart/operators.hpp\"", "public"]} ``` You can find plenty of examples in the `*.imp` files in `scripts/lint/linters/iwyu`. ### Use IWYU pragmas You can also use `IWYU` pragmas as described in the [`IWYU` documentation](https://github.com/include-what-you-use/include-what-you-use/blob/master/docs/IWYUPragmas.md). The most useful IWYU pragma is the `// IWYU pragma: keep` comment that can be added at the end of a header to make sure IWYU does not remove it. ## IWYU false positives IWYU is not perfect and does produce a non-trivial amount of false positives (especially when templated variables are concerned). Here are some of the common issues and how to fix them. ### Removing pybind11 headers from bindings files. Some pybind11 headers, while not necessary for compilation, introduce additional functionality that IWYU doesn't know about. The following headers are therefore often suggested to be removed when they shouldn't be: * * * * There can be others as well. The safest thing to do is to keep any pybind11 headers unless there is good reason to believe they are unnecessary. To keep these headers add a `// IWYU pragma: keep` comment at the end of each include. ### `` and `` These are 2 of the most commonly suggested headers to remove, and they often turn out to be needed. To keep them simply add a `// IWYU pragma: keep` comment at the end of the include statement. ### Repeatedly suggesting to add and then remove a header This usually happens when you run IWYU and then forget to `git add` the changes. Running `git add` therefore should fix it. This should never be an issue when running IWYU as part of `git commit`, since pre-commit stashes unstaged changes.