Add validator to ensure runner determinator script is kept in sync (p…

…ytorch#134800)

We keep two copies of the runner-determinator script:
1. In runner_determinator.py, for ease of testing.  This however is not actually executed during CI
2. Embedded in _runner-determinator.yml.  This is what CI uses.

Why the duplication? Short version: Because of how github CI works, during a given CI run the workflow yml files could actually come from the main branch, while the remaining files get read from the local commit.
This can lead to a newer version of _runner-determinator.yml trying to invoke an older version of runner_determintor.py than it was actually designed for. Chaos ensues.

We mitigate this by embedding the script into the yml file.  But we still keep the script around because it's much easier to run tests against.

This workflow's job is to ensure that if one edits the script in one of those two locations then they remember to update it in the other location as well
Pull Request resolved: pytorch#134800
Approved by: https://github.com/zxiiro, https://github.com/PaliC
ghstack dependencies: pytorch#134796

.github/workflows/_runner-determinator.yml

@@ -320,7 +320,7 @@ jobs:
               """
               gh = get_gh_client(github_token)
               issue = get_issue(gh, repo, issue_num)
-              return issue.get_comments()[0].body.strip("\n\t ")
+              return str(issue.get_comments()[0].body.strip("\n\t "))
 
 
           def main() -> None:

.github/workflows/runner-determinator-validator.yml

@@ -0,0 +1,40 @@
+name: Validate Runner Determinator Script is in Sync
+
+on:
+  # Run on PRs when the runner-determinator script is updated to ensure it's copies are kept in sync
+  pull_request:
+    paths:
+      - .github/workflows/_runner-determinator.yml
+      - .github/workflows/runner-determinator-validator.yml
+      - .github/scripts/runner_determinator.py
+  workflow_dispatch:
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}-${{ github.event_name == 'workflow_dispatch' }}-${{ github.event_name == 'schedule' }}
+  cancel-in-progress: true
+
+jobs:
+  check-runner-determinator:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+
+    - name: Run Hardcode runner-determinator script
+      id: hardcode-script
+      run: |
+        # Extract the script content from _runner-determinator.yml and skip the first 10 spaces of each line
+        script_content=$(awk '/cat <<EOF > runner_determinator.py/{flag=1;next}/EOF$/{flag=0}flag{print substr($0, 11)}' .github/workflows/_runner-determinator.yml)
+
+        # Write the extracted script content to runner_determinator.py
+        echo "$script_content" > runner_determinator_workflow.py
+
+    - name: Compare runner-determinator script embedded in workflow with checked in script
+      run: |
+        # Compare the extracted runner_determinator script with the existing one
+        # If this check fails, then make sure the contents of .github/scripts/runner_determinator.py is in sync with the
+        # version embedded into .github/workflows/_runner-determinator.yml
+        diff runner_determinator_workflow.py .github/scripts/runner_determinator.py
+      # Fail the job if the scripts are not identical
+      continue-on-error: false