Merge pull request RobotLocomotion#4701 from rpoyner-tri/kcov

bazel tools: integrate kcov for test coverage measurement.

drake/doc/bazel.rst

@@ -85,6 +85,7 @@ Cheat sheet for operating on specific portions of the project::
   bazel test --config=memcheck common:polynomial_test  # Run one test under memcheck (valgrind).
   bazel test --config=fastmemcheck common:*            # Run common's tests under memcheck, with minimal recompiling.
   bazel test --config=asan common:polynomial_test      # Run one test under AddressSanitizer.
+  bazel test --config=kcov common:polynomial_test      # Run one test under kcov (see instructions below).
   bazel build -c dbg common:polynomial_test && \
     gdb ../bazel-bin/drake/common/polynomial_test      # Run one test under gdb.
 
@@ -128,6 +129,34 @@ Note that this config includes *only* the tests that require Gurobi::
 
   ``bazel test --config gurobi ...``
 
+Optional Tools
+==============
+
+The Drake Bazel build system has integration support for some optional
+development tools:
+
+ * kcov -- test coverage analysis
+
+kcov
+----
+
+``kcov`` can analyze coverage for any binary that contains DWARF format
+debuggging symbols, and produce nicely formatted browse-able coverage
+reports. It is supported on Ubuntu and OSX only. Install ``kcov`` from source
+following the instructions here: :ref:`Building kcov <building-kcov>`.
+
+To analyze test coverage, run the tests under ``kcov``::
+
+  bazel test --config kcov //...
+
+The coverage report is written to the ``drake-distro/bazel-kcov`` directory. To
+view it, browse to ``drake-distro/bazel-kcov/index.html``.
+
+.. toctree::
+   :hidden:
+
+   building_kcov
+
 FAQ
 ===
 
@@ -136,4 +165,3 @@ Q. What does ``ccache: error: Could not find compiler "gcc" in PATH`` mean?
    A. Your ``$PATH`` still has the magic ``ccache`` directory on it somewhere.
       Update your dotfiles so that something like ``/usr/lib/ccache`` is not on
       your ``$PATH``.
-

drake/doc/building_kcov.rst

@@ -0,0 +1,40 @@
+.. _building-kcov:
+
+*****************************
+Building ``kcov`` From Source
+*****************************
+
+As of this writing, the recommended way to get ``kcov`` is to build from
+upstream sources from GitHub. Ubuntu Xenial (for example) has packaged
+versions, but they have known problems not evident when using upstream.
+
+Ubuntu-only Preparation
+=======================
+
+On Ubuntu, install the build dependencies::
+
+  sudo apt install binutils-dev libdw-dev libelf-dev libiberty-dev
+
+OSX and Ubuntu Build Steps
+==========================
+
+To build from source, follow these steps::
+
+  git clone [email protected]:SimonKagstrom/kcov.git
+  cd kcov
+  mkdir build
+  cd build
+  cmake ..
+
+The resulting binary will be in ``kcov/build/src/kcov``. Create a symbolic link
+to the binary from some directory on your ``$PATH``.
+
+OSX Run-time Requirements
+=========================
+
+On OSX, be sure that your account has developer mode enabled, which gives you
+the privileges necessary to run debuggers and similar tools. If you are an
+administrator, use this command::
+
+  sudo /usr/sbin/DevToolsSecurity --enable
+

tools/BUILD

@@ -8,6 +8,11 @@ sh_binary(
     data = ["valgrind.supp"],
 )
 
+sh_binary(
+    name = "kcov",
+    srcs = ["kcov.sh"],
+)
+
 py_binary(
     name = "cmake_configure_file",
     srcs = ["cmake_configure_file.py"],

tools/bazel.rc

@@ -53,10 +53,13 @@ test:cpplint --test_tag_filters=cpplint
 build:clang --compiler=clang-3.9
 build:clang --crosstool_top=//tools:default-toolchain
 
-### Coverage build. ###
-build:coverage --copt -g
-build:coverage --copt -fprofile-arcs -ftest-coverage
-build:coverage --linkopt -fprofile-arcs -ftest-coverage
+### Kcov coverage build. ###
+build:kcov --copt -g
+test:kcov --spawn_strategy=standalone
+test:kcov --run_under //tools:kcov
+test:kcov --local_test_jobs=1
+test:kcov --test_tag_filters=-cpplint,-gurobi
+test:kcov --nocache_test_results
 
 ### ASan build. ###
 build:asan --action_env=ASAN_OPTIONS

tools/kcov.sh

@@ -0,0 +1,17 @@
+#!/bin/bash -vx
+
+
+me=$(python -c 'import os; print(os.path.realpath("'"$0"'"))')
+WORKSPACE=$(dirname $(dirname "$me"))
+
+# Just in case, don't collect data from cpplint tests.
+if echo "$@" | grep -q _cpplint ; then
+    "$@"
+    exit $?
+fi
+
+kcov \
+    --include-path=$WORKSPACE \
+    --exclude-pattern=thirdParty,externals \
+    $WORKSPACE/bazel-kcov \
+    "$@"