Skip to content

Commit

Permalink
Reuse managed test components across all *nix flavors (dotnet/coreclr…
Browse files Browse the repository at this point in the history
…#26581)

Today CoreCLR pipelines build tests on the same OS flavors they are
supposed to run on. This is wasteful as we produce identical
managed test artifacts on OSX, Linux, Linux_musl and Linux_rhel6.

With this change, the pipeline construction analyzes the set of
OS / architecture combos to run tests on and picks the "best"
OS (or rather the OS that runs on the fastest lab HW) for build
of the managed test components.

Native test components are newly built as part of the product build
as these do need to be built separately for each targeting OS.
Product build zips them up and publishes them to Azure as an artifact
that gets subsequently downloaded and stitched together with
managed test components in the test run job.

Implementation-wise the change basically proceeds in two steps.
In platform-matrix.yml we first identify the exact set of OS / arch
combos to run. Using this information we subsequently identify
the OS to build managed components on for each combo; this information
is passed to test-job.yml which skips managed test build for those
OS-es that are expecting the managed components to be built on a
different OS.

Product build (build-job.yml) contains the additional logic to
build native test artifacts and publish the to Azure for perusal
by the test run job that downloads and unzips both managed and native
artifacts into the final test folder before publishing to Helix.

Thanks

Tomas


Commit migrated from dotnet/coreclr@2a22180
  • Loading branch information
trylek authored Sep 26, 2019

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
1 parent 0366bc7 commit 1448270
Showing 34 changed files with 764 additions and 352 deletions.
74 changes: 74 additions & 0 deletions eng/build-job.yml
Original file line number Diff line number Diff line change
@@ -4,6 +4,7 @@ parameters:
osGroup: ''
osSubgroup: ''
container: ''
testGroup: ''
crossrootfsDir: ''
timeoutInMinutes: ''
signBinaries: false
@@ -88,6 +89,49 @@ jobs:
- name: enforcePgoArg
value: '-enforcepgo'

- name: artifactRootName
value: $(osGroup)$(osSubgroup)_$(archType)_$(buildConfig)

- name: testNativeArtifactName
value: 'NativeTestComponents_$(artifactRootName)'

- name: testNativeRootFolderPath
value: '$(binTestsPath)/obj/$(osGroup).$(archType).$(buildConfigUpper)'

- name: testBuildArtifactName
value: 'TestBuild_$(artifactRootName)'

- name: testBuildRootFolderPath
value: '$(binTestsPath)/$(osGroup).$(archType).$(buildConfigUpper)'

- name: binTestsPath
value: '$(Build.SourcesDirectory)/bin/tests'

- name: archiveExtension
value: '.tar.gz'
- name: archiveType
value: tar
- name: tarCompression
value: gz

- ${{ if eq(parameters.osGroup, 'Windows_NT') }}:
- name: archiveExtension
value: '.zip'
- name: archiveType
value: zip
- name: tarCompression
value: ''

- name: priorityArg
value: ''
- ${{ if ne(parameters.testGroup, 'innerloop') }}:
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- name: priorityArg
value: 'priority1'
- ${{ if eq(parameters.osGroup, 'Windows_NT') }}:
- name: priorityArg
value: '-priority=1'

steps:

# Install native dependencies
@@ -130,6 +174,14 @@ jobs:
- script: set __TestIntermediateDir=int&&build.cmd $(buildConfig) $(archType) -ci -skiptests -skipbuildpackages $(officialBuildIdArg) $(ibcOptimizeArg) $(enforcePgoArg)
displayName: Build product

# Build native test components
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- script: ./build-test.sh skipmanaged $(buildConfig) $(archType) $(crossArg) $(priorityArg) $(clangArg)
displayName: Build native test components
- ${{ if eq(parameters.osGroup, 'Windows_NT') }}:
- script: build-test.cmd skipmanaged $(buildConfig) $(archType) $(priorityArg)
displayName: Build native test components

# Sign on Windows
- ${{ if and(eq(parameters.osGroup, 'Windows_NT'), eq(parameters.signBinaries, 'true')) }}:
- powershell: eng\common\build.ps1 -ci -sign -restore -configuration:$(buildConfig) -warnaserror:0 /p:ArcadeBuild=true /p:OfficialBuild=true /p:BuildOS=$(osGroup) /p:BuildArch=$(archType) /p:BuildType=$(_BuildConfig) /p:DotNetSignType=$env:_SignType -projects $(Build.SourcesDirectory)\eng\empty.csproj
@@ -151,6 +203,28 @@ jobs:
pathtoPublish: $(Build.SourcesDirectory)/bin/Product/$(osGroup).$(archType).$(_BuildConfig)
artifactName: ${{ format('BinDir_{0}{1}_{2}_{3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}

# Publish test native components for consumption by test execution.
- template: /eng/upload-artifact-step.yml
parameters:
rootFolder: $(testNativeRootFolderPath)
includeRootFolder: false
archiveFile: $(Build.StagingDirectory)/$(testNativeArtifactName)$(archiveExtension)
archiveType: $(archiveType)
tarCompression: $(tarCompression)
artifactName: $(testNativeArtifactName)
displayName: 'native test components'

# Publish test build root for consumption by test execution.
- template: /eng/upload-artifact-step.yml
parameters:
rootFolder: $(testBuildRootFolderPath)
includeRootFolder: false
archiveFile: $(Build.StagingDirectory)/$(testBuildArtifactName)$(archiveExtension)
archiveType: $(archiveType)
tarCompression: $(tarCompression)
artifactName: $(testBuildArtifactName)
displayName: 'test build tree'

# Get key vault secrets for publishing
- ${{ if and(ne(variables['System.TeamProject'], 'public'), ne(variables['Build.Reason'], 'PullRequest')) }}:
- task: AzureKeyVault@1
60 changes: 35 additions & 25 deletions eng/build-test-job.yml
Original file line number Diff line number Diff line change
@@ -13,10 +13,13 @@ parameters:
condition: true
stagedBuild: false

### Test build job
### Build managed test components (native components are getting built as part
### of the the product build job).

### Each test build job depends on a corresponding build job with the same
### buildConfig and archType.
### TODO: As of today, build of managed test components requires the product build
### as a prerequisite due to dependency on System.Private.Corelib. After switching
### over to its reference assembly we should be able to remove this dependency and
### run managed test builds in parallel with the product build job.

jobs:
- template: xplat-test-job.yml
@@ -25,6 +28,10 @@ jobs:
archType: ${{ parameters.archType }}
osGroup: ${{ parameters.osGroup }}
osSubgroup: ${{ parameters.osSubgroup }}
managedTestBuildOsGroup: ${{ parameters.osGroup }}
managedTestBuildOsSubgroup: ${{ parameters.osSubgroup }}
container: ${{ parameters.container }}
testGroup: ${{ parameters.testGroup }}
readyToRun: ${{ parameters.readyToRun }}
corefxTests: ${{ parameters.corefxTests }}
stagedBuild: ${{ parameters.stagedBuild }}
@@ -54,16 +61,6 @@ jobs:

variables:
# Map template parameters to command line arguments
- name: priorityArg
value: ''
- ${{ if ne(parameters.testGroup, 'innerloop') }}:
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- name: priorityArg
value: 'priority1'
- ${{ if eq(parameters.osGroup, 'Windows_NT') }}:
- name: priorityArg
value: '-priority=1'

- name: crossgenArg
value: ''
- ${{ if eq(parameters.readyToRun, true) }}:
@@ -94,14 +91,12 @@ jobs:
# See https://docs.microsoft.com/azure/devops/pipelines/process/conditions
condition: and(succeeded(), ${{ parameters.condition }})

# Test job depends on the corresponding build job
# TODO: Build of managed test components currently depends on the corresponding build job
# because it needs System.Private.Corelib; we should be able to remove this dependency
# by switching over to using reference assembly.
${{ if ne(parameters.stagedBuild, true) }}:
dependsOn: ${{ format('build_{0}{1}_{2}_{3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}

# Run all steps in the container.
# Note that the containers are defined in platform-matrix.yml
container: ${{ parameters.container }}

${{ if eq(parameters.testGroup, 'innerloop') }}:
timeoutInMinutes: 90

@@ -135,24 +130,39 @@ jobs:
targetFolder: $(Build.SourcesDirectory)/bin/Product/$(osGroup).$(archType).$(buildConfigUpper)


# Build tests
# Build managed test components
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- script: ./build-test.sh $(buildConfig) $(archType) $(crossArg) $(priorityArg) $(crossgenArg) $(clangArg) $(testhostArg) ci
displayName: Build tests
- script: ./build-test.sh skipnative skipgeneratelayout $(buildConfig) $(archType) $(crossArg) $(priorityArg) $(crossgenArg) $(clangArg) $(testhostArg) ci
displayName: Build managed test components
- ${{ if eq(parameters.osGroup, 'Windows_NT') }}:
- script: build-test.cmd $(buildConfig) $(archType) $(priorityArg) $(crossgenArg) $(testhostArg) ci
displayName: Build tests
- script: build-test.cmd skipnative skipgeneratelayout $(buildConfig) $(archType) $(priorityArg) $(crossgenArg) $(testhostArg) ci
displayName: Build managed test components


# Zip and publish tests
# Zip and publish managed test components
- template: /eng/upload-artifact-step.yml
parameters:
rootFolder: $(testRootFolderPath)
includeRootFolder: false
archiveFile: $(testRootFolderPath)$(archiveExtension)
archiveFile: '$(Build.StagingDirectory)/$(testArtifactName)$(archiveExtension)'
archiveType: $(archiveType)
tarCompression: $(tarCompression)
artifactName: $(testArtifactName)
displayName: 'managed test components'


# Publish .packages/microsoft.net.sdk.il needed for traversing
# test projects during the copynativeonly command in run test job.
- ${{ if ne(parameters.corefxTests, true) }}:
- template: /eng/upload-artifact-step.yml
parameters:
rootFolder: $(microsoftNetSdkIlFolderPath)
includeRootFolder: false
archiveFile: $(Build.StagingDirectory)/$(microsoftNetSdkIlArtifactName)$(archiveExtension)
archiveType: $(archiveType)
tarCompression: $(tarCompression)
artifactName: $(microsoftNetSdkIlArtifactName)
displayName: 'Microsoft.NET.Sdk.IL package'


# Publish Logs
6 changes: 6 additions & 0 deletions eng/pipelines/coreclr/ci.yml
Original file line number Diff line number Diff line change
@@ -26,6 +26,8 @@ jobs:
- OSX_x64
- Windows_NT_arm
- Windows_NT_arm64
jobParameters:
testGroup: outerloop

#
# Checked builds
@@ -35,6 +37,8 @@ jobs:
jobTemplate: build-job.yml
buildConfig: checked
platformGroup: all
jobParameters:
testGroup: outerloop

#
# Release builds
@@ -49,6 +53,8 @@ jobs:
- Linux_x64
- OSX_x64
- Windows_NT_x86
jobParameters:
testGroup: outerloop

#
# Checked test builds
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/corefx-jitstress.yml
Original file line number Diff line number Diff line change
@@ -28,6 +28,8 @@ jobs:
# TODO: re-enable Windows_NT_arm when https://github.com/dotnet/corefx/issues/38863 is resolved
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: jitstress

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/corefx-jitstress2-jitstressregs.yml
Original file line number Diff line number Diff line change
@@ -28,6 +28,8 @@ jobs:
# TODO: re-enable Windows_NT_arm when https://github.com/dotnet/corefx/issues/38863 is resolved
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: jitstress

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/corefx-jitstressregs.yml
Original file line number Diff line number Diff line change
@@ -28,6 +28,8 @@ jobs:
# TODO: re-enable Windows_NT_arm when https://github.com/dotnet/corefx/issues/38863 is resolved
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: jitstressregs

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/corefx.yml
Original file line number Diff line number Diff line change
@@ -28,6 +28,8 @@ jobs:
# TODO: re-enable Windows_NT_arm when https://github.com/dotnet/corefx/issues/38863 is resolved
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: outerloop

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/gc-longrunning.yml
Original file line number Diff line number Diff line change
@@ -25,6 +25,8 @@ jobs:
# disable ARM for now - Linux_arm64
- Windows_NT_x64
# disable ARM for now - Windows_NT_arm64
jobParameters:
testGroup: gc-longrunning

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/gcstress-extra.yml
Original file line number Diff line number Diff line change
@@ -21,6 +21,8 @@ jobs:
jobTemplate: build-job.yml
buildConfig: checked
platformGroup: gcstress
jobParameters:
testGroup: gcstress-extra

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/gcstress0x3-gcstress0xc.yml
Original file line number Diff line number Diff line change
@@ -21,6 +21,8 @@ jobs:
jobTemplate: build-job.yml
buildConfig: checked
platformGroup: gcstress
jobParameters:
testGroup: gcstress0x3-gcstress0xc

- template: /eng/platform-matrix.yml
parameters:
1 change: 1 addition & 0 deletions eng/pipelines/coreclr/internal.yml
Original file line number Diff line number Diff line change
@@ -54,6 +54,7 @@ stages:
timeoutInMinutes: 120
signBinaries: true
publishToBlobFeed: true
testGroup: outerloop

#
# Publish build information to Build Assets Registry
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/jitstress-isas-arm.yml
Original file line number Diff line number Diff line change
@@ -23,6 +23,8 @@ jobs:
platforms:
- Linux_arm64
- Windows_NT_arm64
jobParameters:
testGroup: jitstress-isas-arm

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/jitstress-isas-x86.yml
Original file line number Diff line number Diff line change
@@ -25,6 +25,8 @@ jobs:
- OSX_x64
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: jitstress-isas-x86

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/jitstress.yml
Original file line number Diff line number Diff line change
@@ -21,6 +21,8 @@ jobs:
jobTemplate: build-job.yml
buildConfig: checked
platformGroup: all
jobParameters:
testGroup: jitstress

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/jitstress2-jitstressregs.yml
Original file line number Diff line number Diff line change
@@ -21,6 +21,8 @@ jobs:
jobTemplate: build-job.yml
buildConfig: checked
platformGroup: all
jobParameters:
testGroup: jitstress2-jitstressregs

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/jitstressregs-x86.yml
Original file line number Diff line number Diff line change
@@ -24,6 +24,8 @@ jobs:
- Linux_x64
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: jitstressregs-x86

- template: /eng/platform-matrix.yml
parameters:
2 changes: 2 additions & 0 deletions eng/pipelines/coreclr/jitstressregs.yml
Original file line number Diff line number Diff line change
@@ -21,6 +21,8 @@ jobs:
jobTemplate: build-job.yml
buildConfig: checked
platformGroup: all
jobParameters:
testGroup: jitstressregs

- template: /eng/platform-matrix.yml
parameters:
6 changes: 5 additions & 1 deletion eng/pipelines/coreclr/perf.yml
Original file line number Diff line number Diff line change
@@ -20,6 +20,8 @@ jobs:
- Linux_x64
- Windows_NT_x64
- Windows_NT_x86
jobParameters:
testGroup: perf

- template: /eng/platform-matrix.yml
parameters:
@@ -28,4 +30,6 @@ jobs:
platforms:
- Linux_x64
- Windows_NT_x64
- Windows_NT_x86
- Windows_NT_x86
jobParameters:
testGroup: perf
Loading
Oops, something went wrong.

0 comments on commit 1448270

Please sign in to comment.