Documentation and scripts of CodeChecker

integration in the Gerrit Review process
through Jenkins is added.

docs/jenkins_gerrit_integration.md

@@ -0,0 +1,232 @@
+Integrate CodeChecker with Gerrit review
+========================================
+
+This guide covers a method to integrate CodeChecker with the Gerrit review
+process.
+
+To achieve this, a Jenkins job is configured to clone the changes from Gerrit,
+analyze the project with CodeChecker and publish (as comments in Gerrit) the
+issues found in the changed files. Based on the result of the analysis, the job
+also sets the Code-Review and Verify labels in Gerrit Review. For reporting,
+[Gerrit's REST API](https://gerrit-review.googlesource.com/Documentation/rest-api.html)
+is used. In addition to the inline comments in Gerrit, the full result of the
+analysis is stored and visualised in Jenkins.
+
+Table of Contents
+=================
+* [Gerrit requirements](#gerrit-requirements)
+* [Jenkins requirements](#jenkins-requirements)
+* [Gerrit Trigger plugin setup](#gerrit-plugin-setup)
+* [Configuring the Jenkins job](#configuring-the-jenkins-job)
+  * [Source Code Managament](#jenkins-source-code-management)
+  * [Build Triggers](#jenkins-build-triggers)
+  * [Gerrit Trigger](#jenkins-gerrit-trigger)
+  * [Build Environment](#jenkins-build-environment)
+  * [Build](#jenkins-build)
+
+## <a name="gerrit-requirements"></a> Gerrit requirements
+On Gerrit, a user with the following privileges must be present:
+* Read
+* Label Code-Review
+* Label Verified
+
+If the user is about to clone from Gerrit through SSH, its SSH Public Key
+(generated on the CI machine) has to be added to Gerrit.
+
+## <a name="jenkins-requirements"></a> Jenkins requirements
+Required plugins
+* Git plugin
+* Gerrit Trigger plugin
+
+Optional plugins for reporting (could be replaced by `curl`)
+* HTTP Request Plugin
+* Environment Injector Plugin
+
+Requirements of the Jenkins machine
+* CodeChecker >= 6.4
+* Python >= 2.7
+* Access to Gerrit (i.e. proper firewall settings)
+
+## <a name="gerrit-plugin-setup"></a> Gerrit Trigger plugin setup
+If Gerrit Trigger plugin is installed correctly, its settings
+should be available under Manage Jenkins.
+At Gerrit Connection Setting a working connection must be defined
+to the Gerrit server (can be tested with Test Connection). Note that 
+the SSH private key's public pair should have been already added to Gerrit.
+
+In the Advanced settings, at Gerrit Reporting Values, the default
+reporting values can be configured. These can be overwritten later in the job's
+configuration. In this guide the Gerrit review message is crafted by hand,
+therefore it is strongly advised to remove all the default values from the list
+(otherwise once the review message is sent, the job will send another
+reporting value based on this default configuration that can overwrite 
+the first review and change the final result shown in Gerrit).
+
+In the REST API section the user performing the Gerrit review must be added,
+with `Enable Code-Review` and `Enable Verified` set to enabled.
+
+Here is the config that worked best for us:
+![Reporting values](/docs/images/gerrit_jenkins/gerrit_plugin_values.png)
+![REST API](/docs/images/gerrit_jenkins/gerrit_plugin_restapi.png)
+
+## <a name="configuring-the-jenkins-job"></a> Configuring the Jenkins job
+The job is configured as a freestyle project.
+### <a name="jenkins-source-code-management"></a> Source Code Management
+In the job's configuration menu set `Source Code Management` to `Git`.  The
+`Repository URL` is the link of the repo that has to be checked. Add new or set
+from the list the credentials of the user who can clone and review the changes.
+In the Repositories' advanced settings, set refspec to `${GERRIT_REFSPEC}`.
+`Branch Specifier` should be set to `**`.
+
+In the `Additional Behaviour` section, add
+`Strategy for choosing what to build`, then select the `Gerrit Trigger`
+from the `Choosing strategy` list.
+
+If needed, the subdirectory used to check out the repo could be also defined
+as an `Additional Behaviour`.
+
+![Source Code Management configuration](/docs/images/gerrit_jenkins/source_code_management.png)
+
+### <a name="jenkins-gerrit-trigger"></a> Gerrit Trigger
+`Build Triggers` should be set to `Gerrit event`. This opens the Gerrit Trigger
+configuration form.
+In the `Choose a Server` field, the server that has been already configured in 
+[Gerrit Trigger plugin setup](#gerrit-plugin-setup)
+should be selected from the list.
+In the server's advanced section, the `Gerrit reporting values`
+(e.g. Succesful - verified +1) and `Build Messages` can be configured.
+If left empty, values configured in the [plugin's setup](#gerrit-plugin-setup)
+are used. One possible setting is to leave all fields empty, 
+except Verify - Failed - -1. This ensures that the values reported at the 
+end of the analysis (crafted manually) will not be overwritten by Jenkins.
+
+Add a `Trigger` on `Patchset Created` and tick `Exclude No Code Change`
+(there is no point in reanalyzing the project if the code has not changed).
+At Gerrit Project, set the project that should trigger the build. 
+Type should be Path and Pattern should be the location of your project
+(relative to the Gerrit server's address).
+
+`Branches` should be set to Type - Path and Pattern - `**`.
+
+![Gerrit Trigger configuration](/docs/images/gerrit_jenkins/build_trigger.png)
+
+### <a name="jenkins-build-environment"></a> Build Environment
+It is advised to delete the workspace before starting a new build.
+It is also possible to delete only part of it, by applying patterns
+for the files to be deleted. This is useful if the workspace contains 
+dependencies which are not cloned with the project (e.g. CodeChecker binary).
+
+### <a name="jenkins-build"></a> Build
+All of the followings are added as separate build steps. Note that the two HTTP
+Requests and the Environment Injection could be replaced by `curl` in the
+Execute Shell part. We decided to use these plugins in order to prevent
+storing the reporting Gerrit user's password in the script.
+
+#### HTTP Request (get the changes)
+To query the list of the changed files from Gerrit, add a `HTTP Request` build
+step.
+The `URL` should be the Gerrit [revision files endpoint](https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#list-files),
+filled with Jenkins' Gerrit environment variables
+(`$GERRIT_CHANGE_ID` and `$GERRIT_PATCHSET_REVISION`).
+In the `Advanced` section, `Authorization` should be set to the user with access
+to the project. `Headers` are `APPLICATION_JSON_UTF8` for both accept
+and content-type.
+
+In the `Response` section, set `Output response` to the file
+that is parsed in the following build step.
+
+![HTTP Request configuration (get the changes)](/docs/images/gerrit_jenkins/http_request_changed_files.png)
+
+#### Execute shell
+In this shell, the project is logged and analyzed by CodeChecker.
+A CodeChecker skipfile is generated from the file containing the changes 
+(see previous section). This is done by the `create_skipfile.py` script. 
+Based on this skipfile, CodeChecker parses the analysis results. After this,
+the Gerrit review message is crafted (with `parse_output_parser.py`).
+Both Python scripts used here are available in CodeChecker's GitHub repository,
+under `scripts/gerrit_jenkins`.
+
+In addition, this shell can run tests. This could be used to
+set Verify -1 or +1, depending on success or failure of the tests
+(failing test causes the Jenkins job to fail).
+
+It could happen that a change in a file causes an error that is found
+in another file. This means, having only the inline comments in Gerrit
+is not satisfactory to identify all new bugs introduced by the change.
+To ensure that all issues related to the new code are found, a full analysis
+has to be performed. The results of this check can be compared to a baseline
+analysis and all new findings can be saved as HTML. Then the URL of
+`index.html` can be sent as part of the review, shown as a normal
+(non-inline) comment.
+
+Our recommendation for having an up-to-date baseline is to
+create a Jenkins job which is triggered on every merge to the master branch.
+This job can perform a full analysis and store the analysis results on a
+CodeChecker server. After this, `CodeChecker cmd diff` can be used to
+compare the two analysis.
+
+The following script is an example of what could be added 
+to an Execute shell build step, to achieve the above.
+~~~~~~{.sh}
+# Make sure all requirements of the build is fulfilled by this stage!
+# Log the build.
+CodeChecker log -b "build command" -o compile_cmd.json
+
+# Run the tests. If this fails, the result will be verified -1.
+make tests
+
+# Check the project.
+# NOTE: It is possible to use the skipfile of changed files in the analysis,
+# but it is discouraged, since changes can cause errors in other files.
+CodeChecker analyze \
+  -e sensitive \
+  -j 16 \
+  compile_cmd.json \
+  -o cc_reports
+
+# Generate skipfile of changed files from Gerrit.
+python create_skipfile.py files-changed skipfile
+
+# Diff to baseline and filter for new findings, i.e. intorudced
+# by the changes. Save the output as HTML in Jenkins' userContent
+# directory, so it is available on a webserver.
+CodeChecker cmd diff \
+  -b clangsa_checkers-merge \
+  --url your_baseline_url \
+  -n cc_reports \
+  --new \
+  -o html \
+  -e $JENKINS_HOME/userContent/$JOB_NAME/$BUILD_NUMBER
+
+# Craft the review message and write it into a file,
+# that is later going to be loaded as an Environment Variable.
+echo DATA_TO_POST=\\ > review
+CodeChecker parse -i skipfile cc_reports |
+  python parse_output_parser.py \
+    $WORKSPACE/repo_dir \
+    --report_url http://your_jenkins_address/userContent/$JOB_NAME/$BUILD_NUMBER/index.html \
+      >> review
+
+~~~~~~
+
+#### Inject environment variable
+To load the previously crafted review message as an environment variable,
+add an Inject environment variables build step. The file path is the 
+path of the file written at the end of the execute shell step, 
+while the content field can remain empty.
+
+#### HTTP Request (send the review)
+To send the crafted Gerrit review, add a `HTTP Request` build step.
+The `URL` should be the Gerrit
+[revision set review endpoint](https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#set-review),
+filled with Jenkins' Gerrit environment variables
+(`$GERRIT_CHANGE_ID` and `$GERRIT_PATCHSET_REVISION`). 
+The `HTTP method` to use is `POST`.
+In the `Advanced` section, `Authorization` should be set to a user with access
+to the project in Gerrit. `Headers` are `APPLICATION_JSON_UTF8` for both accept
+and content-type.
+
+The request's `body` should be set to the environment variable that was
+injected before, which contains the review message.
+
+![HTTP Request configuration (send the review)](/docs/images/gerrit_jenkins/http_request_post_review.png)

scripts/gerrit_jenkins/create_skipfile.py

@@ -0,0 +1,50 @@
+# -------------------------------------------------------------------------
+#                     The CodeChecker Infrastructure
+#   This file is distributed under the University of Illinois Open Source
+#   License. See LICENSE.TXT for details.
+# -------------------------------------------------------------------------
+"""
+Converts Gerrit Review changed files list to CodeChecker skipfile.
+"""
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
+
+import argparse
+import json
+import re
+
+
+def create_skipfile(files_changed, skipfile):
+    # File is likely to contain some garbage values at start,
+    # only the corresponding json should be parsed.
+    json_pattern = re.compile(r"^\{.*\}")
+    for line in files_changed.readlines():
+        if re.match(json_pattern, line):
+            for filename in json.loads(line):
+                if "/COMMIT_MSG" in filename:
+                    continue
+                skipfile.write(u"+*/%s\n" % filename)
+
+    skipfile.write(u"-*\n")
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Converts Gerrit Review changed files '
+                    'json to CodeChecker skipfile.')
+    parser.add_argument(
+        'files_changed',
+        type=argparse.FileType('r'),
+        help="Path of changed files json from Gerrit.")
+    parser.add_argument(
+        'skipfile', nargs='?', default='skipfile',
+        type=argparse.FileType('w'),
+        help="Path of the skipfile output. Default is ./skipfile.")
+    args = parser.parse_args()
+
+    create_skipfile(args.files_changed, args.skipfile)
+
+
+if __name__ == '__main__':
+    main()

scripts/gerrit_jenkins/parse_output_parser.py

@@ -0,0 +1,87 @@
+# -------------------------------------------------------------------------
+#                     The CodeChecker Infrastructure
+#   This file is distributed under the University of Illinois Open Source
+#   License. See LICENSE.TXT for details.
+# -------------------------------------------------------------------------
+"""
+Generates Gerrit Review json from CodeChecker parse's output.
+"""
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
+
+import argparse
+import json
+import os
+import re
+import sys
+
+
+def parse_issue(issue, repo_dir, review):
+    line = issue[0]
+    abs_file_path = line[line.find(' ') + 1: line.find(':')]
+    rel_file_path = os.path.relpath(abs_file_path, repo_dir)
+    splitted = line.split(':')
+    line_num = splitted[1]
+    column_num = splitted[2]
+    if rel_file_path not in review['comments']:
+        review['comments'][rel_file_path] = []
+    review['comments'][rel_file_path].append(
+        {"range": {"start_line": line_num,
+                   "start_character": column_num,
+                   "end_line": line_num,
+                   "end_character": column_num}, "message": ''.join(issue)})
+
+
+def process_reports(repo_dir, reports, report_url):
+    read = False
+    issues = []
+    # Matches the severity of the bug. If doesn't match, there is no report.
+    severity = re.compile(r"^\[.*\]")
+    for line in reports.readlines():
+        if re.match(severity, line):
+            read = True
+            issues.append([line])
+        elif line == '\n':
+            read = False
+        elif read:
+            issues[-1].append(line)
+
+    result = 1 if issues == 0 else -1
+    report = ""
+    if report_url:
+        report = " Html report of the run is available at %s." % report_url
+    review = {"tag": "jenkins",
+              "message":
+                  "CodeChecker found %i issue(s) in the code.%s"
+                  % (len(issues), report),
+              "labels": {"Code-Review": result, "Verified": 1}, "comments": {}}
+    for issue in issues:
+        parse_issue(issue, repo_dir, review)
+
+    json.dump(review, sys.stdout)
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Generates Gerrit Review json'
+                    ' from CodeChecker parse\'s output.')
+    parser.add_argument(
+        'repo_dir',
+        help="Root directory of the sources, \
+              i.e. the directory where the repository was cloned.")
+    parser.add_argument(
+        '--report_url', nargs='?', default="",
+        help="URL where the report can be found.")
+    parser.add_argument(
+        'parse_output', nargs='?', default=sys.stdin,
+        type=argparse.FileType('r'),
+        help="The output of CodeChecker parse. \
+              If no file is provided, stdin is used.")
+    args = parser.parse_args()
+
+    process_reports(args.repo_dir, args.parse_output, args.report_url)
+
+
+if __name__ == '__main__':
+    main()