CI: run pre-commit hooks in CI

This also means flake8 will output to stdout in CI now. It didn't
before because it output to an XML format for publishing.

Pre-commit creates its own environment in which it installs hooks. To
speed up runs, the pipeline will cache this for use with future jobs.
The cache will update if .pre-commit-config.yaml changes.

The flake8 pre-commit hook invokes flake8 via `pipenv run flake8`. It's
normally useful to use pipenv here cause it ensures flake8 is invoked
within the context of the venv. However, in CI, there is no venv -
dependencies are installed directly to the system site-packages.
`pipenv run` does not work in such case because it tries to create a new
venv if one doesn't exist (it doesn't consider the system interpreter to
be a venv).

This workaround (okay, it's a hack) creates an executable shell script
which replaces the original pipenv binary. The shell script simply
ignores the first argument (i.e. ignores `run` in `pipenv run`) and
executes the rest of the arguments as a command. It essentially makes
`pipenv run flake8` equivalent to just having ran `flake8`. When
pre-commit executes pipenv, the aforementioned script is what will run.

Author: MarkKoz

azure-pipelines.yml

@@ -8,10 +8,12 @@ jobs:
 
     variables:
       PIP_CACHE_DIR: .cache/pip
+      PRE_COMMIT_HOME: $(Pipeline.Workspace)/pre-commit-cache
 
     steps:
       - task: UsePythonVersion@0
         displayName: 'Set Python Version'
+        name: PythonVersion
         inputs:
           versionSpec: '3.7.x'
           addToPath: true
@@ -31,8 +33,28 @@ jobs:
           pip install flake8-formatter-junit-xml
         displayName: 'Install Project Environment'
 
-      - script: flake8 --format junit-xml --output-file TEST-lint.xml
-        displayName: 'Run Linter'
+      # Create an executable shell script which replaces the original pipenv binary.
+      # The shell script ignores the first argument and executes the rest of the args as a command.
+      # It makes the `pipenv run flake8` command in the pre-commit hook work by circumventing
+      # pipenv entirely, which is too dumb to know it should use the system interpreter rather than
+      # creating a new venv.
+      - script: |
+          printf '%s\n%s' '#!/bin/bash' '"${@:2}"' > $(PythonVersion.pythonLocation)/bin/pipenv \
+          && chmod +x $(PythonVersion.pythonLocation)/bin/pipenv
+        displayName: 'Mock pipenv binary'
+
+      - task: Cache@2
+        displayName: 'Restore pre-commit environment'
+        inputs:
+          key: pre-commit | "$(PythonVersion.pythonLocation)" | .pre-commit-config.yaml
+          restoreKeys: |
+            pre-commit | "$(PythonVersion.pythonLocation)"
+          path: $(PRE_COMMIT_HOME)
+
+      # flake8 runs so it can generate the XML output. pre-commit will run it again to show stdout.
+      # flake8 standalone runs first to avoid any fixes pre-commit hooks may make.
+      - script: flake8 --format junit-xml --output-file TEST-lint.xml; pre-commit run --all-files
+        displayName: 'Run pre-commit hooks'
 
       - script: |
           python3 manage.py makemigrations --check