Translate annotation line numbers from merge to head (pytorch#55569)

Summary:
This PR

- adds a `tools/translate_annotations.py` script that
  - parses annotations into JSON using the regexes that we were previously passing to [`pytorch/add-annotations-github-action`](https://github.com/pytorch/add-annotations-github-action) and
  - uses `git diff-index` to translate the line numbers for those annotations from the PR `merge` onto the PR `head`, since (as of pytorch#54967) we now run CI on the former instead of the latter;
- modifies the `flake8-py3` and `clang-tidy` jobs to use that script and thus upload JSON in their artifacts instead of raw text; and
- modifies the "Add annotations" workflow to specify `mode: json` to allow it to use those preprocessed annotations.

Depends on pytorch/add-annotations-github-action#18.

Pull Request resolved: pytorch#55569

Test Plan:
You can run the unit tests with this command:
```
python tools/test/test_translate_annotations.py
```
I also tested the entire system together in my personal sandbox repo.

Reviewed By: malfet

Differential Revision: D27662161

Pulled By: samestep

fbshipit-source-id: ecca51b79b9cf00c90fd89f0d41d0c7b89d69c63

.github/workflows/add_annotations.yml

@@ -12,11 +12,9 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        include:
-          - name: flake8-py3
-            regex: '^(?<filename>.*?):(?<lineNumber>\d+):(?<columnNumber>\d+): (?<errorCode>\w+\d+) (?<errorDesc>.*)'
-          - name: clang-tidy
-            regex: '^(?<filename>.*?):(?<lineNumber>\d+):(?<columnNumber>\d+): (?<errorDesc>.*?) \[(?<errorCode>.*)\]'
+        name:
+          - flake8-py3
+          - clang-tidy
     if: github.event.workflow_run.event == 'pull_request'
     runs-on: ubuntu-18.04
     steps:
@@ -62,8 +60,8 @@ jobs:
         uses: pytorch/add-annotations-github-action@master
         with:
           check_name: ${{ matrix.name }}
-          linter_output_path: warnings.txt
+          linter_output_path: annotations.json
           commit_sha: ${{ steps.unzip.outputs.commit-sha }}
-          regex: ${{ matrix.regex }}
+          mode: json
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/lint.yml

@@ -115,6 +115,8 @@ jobs:
           architecture: x64
       - name: Fetch PyTorch
         uses: actions/checkout@v2
+        with:
+          fetch-depth: 2 # to allow us to use github.event.pull_request.head.sha
       - name: Prepare output dir with HEAD commit SHA
         run: |
           mkdir flake8-output
@@ -125,17 +127,30 @@ jobs:
       - name: Run flake8
         run: |
           set -eux
+          pip install typing-extensions # for tools/translate_annotations.py
           pip install -r requirements-flake8.txt
           flake8 --version
-          flake8 | tee "${GITHUB_WORKSPACE}"/flake8-output/warnings.txt
+          flake8 | tee "${GITHUB_WORKSPACE}"/flake8-output.txt
+          cp flake8-output.txt flake8-output/annotations.json
+      - name: Translate annotations
+        if: github.event_name == 'pull_request'
+        run: |
+          tools/translate_annotations.py \
+            --file=flake8-output.txt \
+            --regex='^(?P<filename>.*?):(?P<lineNumber>\d+):(?P<columnNumber>\d+): (?P<errorCode>\w+\d+) (?P<errorDesc>.*)' \
+            --commit="$HEAD_SHA" \
+            > flake8-output/annotations.json
+        env:
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with:
           name: flake8-py3
           path: flake8-output/
       - name: Fail if there were any warnings
         run: |
-          [ ! -s flake8-output/warnings.txt ]
+          cat flake8-output.txt
+          [ ! -s flake8-output.txt ]
 
   clang-tidy:
     if: github.event_name == 'pull_request'
@@ -148,6 +163,8 @@ jobs:
           architecture: x64
       - name: Checkout PyTorch
         uses: actions/checkout@v2
+        with:
+          fetch-depth: 0 # to allow tools/clang_tidy.py to do its thing
       - name: Prepare output dir with HEAD commit SHA
         run: |
           mkdir clang-tidy-output
@@ -225,11 +242,18 @@ jobs:
             -g"-torch/csrc/deploy/interpreter/interpreter.h"  \
             -g"-torch/csrc/deploy/interpreter/interpreter_impl.h"  \
             -g"-torch/csrc/deploy/interpreter/test_main.cpp"  \
-            "$@" > "${GITHUB_WORKSPACE}"/clang-tidy-output/warnings.txt
+            "$@" > "${GITHUB_WORKSPACE}"/clang-tidy-output.txt
+
+          cat "${GITHUB_WORKSPACE}"/clang-tidy-output.txt
 
-          cat "${GITHUB_WORKSPACE}"/clang-tidy-output/warnings.txt
+          tools/translate_annotations.py \
+            --file=clang-tidy-output.txt \
+            --regex='^(?P<filename>.*?):(?P<lineNumber>\d+):(?P<columnNumber>\d+): (?P<errorDesc>.*?) \[(?P<errorCode>.*)\]' \
+            --commit="$HEAD_SHA" \
+            > clang-tidy-output/annotations.json
         env:
           BASE_SHA: ${{ github.event.pull_request.base.sha }}
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with:

mypy-strict.ini

@@ -49,7 +49,9 @@ files =
     tools/test_history.py,
     tools/test/test_test_history.py,
     tools/test/test_trailing_newlines.py,
+    tools/test/test_translate_annotations.py,
     tools/trailing_newlines.py,
+    tools/translate_annotations.py,
     torch/testing/_internal/framework_utils.py,
     torch/utils/benchmark/utils/common.py,
     torch/utils/benchmark/utils/timer.py,

tools/README.md

@@ -60,6 +60,11 @@ Developer tools which you might find useful:
   stdin, print names of nonempty files whose contents don't end in exactly one
   trailing newline, exit with status 1 if no output printed or 0 if some
   filenames were printed.
+* [translate_annotations.py](translate_annotations.py) - Read [Flake8][] or
+   [clang-tidy][] warnings (according to a `--regex`) from a `--file`, convert
+   to the JSON format accepted by [pytorch/add-annotations-github-action], and
+   translate line numbers from `HEAD` back in time to the given `--commit` by
+   running `git diff-index --unified=0` appropriately.
 
 Important if you want to run on AMD GPU:
 
@@ -80,5 +85,8 @@ Tools which are only situationally useful:
 * [run-clang-tidy-in-ci.sh](run-clang-tidy-in-ci.sh) - Responsible
   for checking that C++ code is clang-tidy clean in CI on Travis
 
+[clang-tidy]: https://clang.llvm.org/extra/clang-tidy/
+[flake8]: https://flake8.pycqa.org/en/latest/
 [github actions expressions]: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#about-contexts-and-expressions
+[pytorch/add-annotations-github-action]: https://github.com/pytorch/add-annotations-github-action
 [shellcheck]: https://github.com/koalaman/shellcheck