// Copyright 2016 The Bazel Authors. All rights reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package com.google.devtools.build.lib.runtime.commands; import com.google.common.collect.ImmutableList; import com.google.devtools.build.lib.packages.TestTimeout; import com.google.devtools.build.lib.runtime.Command; import com.google.devtools.common.options.OptionPriority.PriorityCategory; import com.google.devtools.common.options.OptionsParser; import com.google.devtools.common.options.OptionsParsingException; /** * Handles the 'coverage' command on the Bazel command line. * *

Here follows a brief, partial and probably wrong description of how coverage collection works * in Bazel. * *

Coverage is reported by the tests in LCOV format in the files {@code * testlogs/PACKAGE/TARGET/coverage.dat} and {@code testlogs/PACKAGE/TARGET/coverage.micro.dat}. * *

To collect coverage, each test execution is wrapped in a script called {@code * collect_coverage.sh}. This script sets up the environment of the test to enable coverage * collection and determine where the coverage files are written by the coverage runtime(s). It then * runs the test. A test may itself run multiple subprocesses and consist of modules written in * multiple different languages (with separate coverage runtimes). As such, the wrapper script * converts the resulting files to lcov format if necessary, and merges them into a single file. * *

The interposition itself is done by the test strategies, which requires {@code * collect_coverage.sh} to be on the inputs of the test. This is accomplished by an implicit * attribute {@code :coverage_support} which is resolved to the value of the configuration flag * {@code --coverage_support} (see {@link * com.google.devtools.build.lib.analysis.test.TestConfiguration.TestOptions#coverageSupport}). * *

There are languages for which we do offline instrumentation, meaning that the coverage * instrumentation is added at compile time, e.g. for C++, and for others, we do online * instrumentation, meaning that coverage instrumentation is added at execution time, e.g. for * Javascript. * *

Another core concept is that of baseline coverage. This is essentially the coverage of * library, binary, or test if no code in it was run. The problem it solves is that if you want to * compute the test coverage for a binary, it is not enough to merge the coverage of all of the * tests, because there may be code in the binary that is not linked into any test. Therefore, what * we do is to emit a coverage file for every binary, which contains only the files we collect * coverage for with no covered lines. The baseline coverage file for a target is at {@code * testlogs/PACKAGE/TARGET/baseline_coverage.dat}. Note that it is also generated for binaries and * libraries in addition to tests if you pass the {@code --nobuild_tests_only} flag to Bazel. * *

Baseline coverage collection is currently broken. * *

We track two groups of files for coverage collection for each rule: the set of instrumented * files and the set of instrumentation metadata files. * *

The set of instrumented files is just that, a set of files to instrument. For online coverage * runtimes, this can be used at runtime to decide which files to instrument. It is also used to * implement baseline coverage. * *

The set of instrumentation metadata files is the set of extra files a test needs to generate * the LCOV files Bazel requires from it. In practice, this consists of runtime-specific files; for * example, the gcc compiler emits {@code .gcno} files during compilation. These are added to the * set of inputs of test actions if coverage mode is enabled (otherwise the set of metadata files is * empty). * *

Whether or not coverage is being collected is stored in the {@code BuildConfigurationValue}. * This is handy because then we have an easy way to change the test action and the action graph * depending on this bit, but it also means that if this bit is flipped, all targets need to be * re-analyzed (note that some languages, e.g. C++ require different compiler options to emit code * that can collect coverage, which dominates the time required for analysis). * *

The coverage support files are depended on through labels an implicit dependency so that they * can be overridden by the invocation policy, which allows them to differ between the different * versions of Bazel. Ideally, these differences will be removed, and we standardize on * * @bazel_tools//tools/coverage. *

A partial set of file types that can be encountered in the coverage world: *

*

OPEN QUESTIONS: *

*/ @Command( name = "coverage", builds = true, inherits = {TestCommand.class}, shortDescription = "Generates code coverage report for specified test targets.", completion = "label-test", help = "resource:coverage.txt", allowResidue = true) public class CoverageCommand extends TestCommand { @Override protected String commandName() { return "coverage"; } @Override public void editOptions(OptionsParser optionsParser) { super.editOptions(optionsParser); try { optionsParser.parse( PriorityCategory.SOFTWARE_REQUIREMENT, "Options required by the coverage command", ImmutableList.of("--collect_code_coverage")); optionsParser.parse( PriorityCategory.COMPUTED_DEFAULT, "Options suggested for the coverage command", ImmutableList.of(TestTimeout.COVERAGE_CMD_TIMEOUT)); } catch (OptionsParsingException e) { // Should never happen. throw new IllegalStateException("Unexpected exception", e); } } }