Skip to content

Commit

Permalink
Migrate binary builds to use Amazon2023 runners (pytorch#131826)
Browse files Browse the repository at this point in the history
A continuation of the migration started in
- pytorch#131250

Migrates all linux binary builds.

The failures are windows jobs which aren't touched by this PR

prev runs (for tracking):
- https://hud.pytorch.org/pytorch/pytorch/pull/131826?sha=e1ee074b1e7b17008e3f3774e4842b5e1d4c1355
- https://hud.pytorch.org/pytorch/pytorch/pull/131826?sha=50a3488ae776f86bd6bead8b048b051c49a25ec7
Pull Request resolved: pytorch#131826
Approved by: https://github.com/malfet
  • Loading branch information
ZainRizvi authored and pytorchmergebot committed Aug 12, 2024
1 parent b06959e commit 1371c42
Show file tree
Hide file tree
Showing 11 changed files with 194 additions and 3 deletions.
7 changes: 7 additions & 0 deletions .github/templates/linux_binary_build_workflow.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,17 @@ jobs:
uses: ./.github/workflows/_binary-build-linux.yml
with:!{{ upload.binary_env_as_input(config) }}
{%- if "aarch64" in build_environment %}
runner_prefix: amz2023.
runs_on: linux.arm64.m7g.4xlarge
ALPINE_IMAGE: "arm64v8/alpine"
{%- elif "s390x" in build_environment %}
runs_on: linux.s390x
ALPINE_IMAGE: "docker.io/s390x/alpine"
{%- elif "conda" in build_environment and config["gpu_arch_type"] == "cuda" %}
runner_prefix: amz2023.
runs_on: linux.24xlarge
{%- else %}
runner_prefix: amz2023.
{%- endif %}
build_name: !{{ config["build_name"] }}
build_environment: !{{ build_environment }}
Expand All @@ -87,6 +91,7 @@ jobs:
build_name: !{{ config["build_name"] }}
build_environment: !{{ build_environment }}
{%- if "aarch64" in build_environment %}
runner_prefix: amz2023.
runs_on: linux.arm64.2xlarge
ALPINE_IMAGE: "arm64v8/alpine"
{%- elif "s390x" in build_environment %}
Expand All @@ -95,8 +100,10 @@ jobs:
{%- elif config["gpu_arch_type"] == "rocm" %}
runs_on: linux.rocm.gpu
{%- elif config["gpu_arch_type"] == "cuda" %}
runner_prefix: amz2023.
runs_on: linux.4xlarge.nvidia.gpu
{%- else %}
runner_prefix: amz2023.
runs_on: linux.4xlarge
{%- endif %}
secrets:
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/_binary-build-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,16 @@ on:
required: true
type: string
description: The build environment
runner_prefix:
required: false
default: ""
type: string
description: prefix for runner label
runs_on:
required: false
default: linux.12xlarge
type: string
description: Hardware to run this "build"job on, linux.12xlarge or linux.arm64.2xlarge.
description: Hardware to run this "build" job on, linux.12xlarge or linux.arm64.2xlarge.
timeout-minutes:
required: false
default: 210
Expand Down Expand Up @@ -89,7 +94,7 @@ on:

jobs:
build:
runs-on: ${{ inputs.runs_on }}
runs-on: ${{ inputs.runner_prefix}}${{ inputs.runs_on }}
timeout-minutes: ${{ inputs.timeout-minutes }}
env:
PYTORCH_ROOT: ${{ inputs.PYTORCH_ROOT }}
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/_binary-test-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,11 @@ on:
required: false
type: string
description: Desired python version
runner_prefix:
required: false
default: ""
type: string
description: prefix for runner label
runs_on:
required: true
type: string
Expand All @@ -77,7 +82,7 @@ on:

jobs:
test:
runs-on: ${{ inputs.runs_on }}
runs-on: ${{ inputs.runner_prefix}}${{ inputs.runs_on }}
timeout-minutes: 240
env:
PYTORCH_ROOT: ${{ inputs.PYTORCH_ROOT }}
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 1371c42

Please sign in to comment.