Skip to content

Commit

Permalink
Update VMs for Patch Tuesday May 2022 (microsoft#24695)
Browse files Browse the repository at this point in the history
* Install Ubuntu nasm package rather than building from source; 2.14 which is available in Ubuntu 20.04 is sufficient for intel-ipsec.

* Add cmake and ninja.

* Update CUDA signing key.

* Update pwsh to 7.2.3

* Remove clean downloads step obsoleted since we ripped out pacman.

* Note that haskell-stack is for bond.

* Cherry pick from microsoft#23569 : add cuda-nvml-dev-11-6

* Update pools.

* Baseline updates

PASSING, REMOVE FROM FAIL LIST: mesa:x64-windows-static-md (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

:)

```
REGRESSION: cppgraphqlgen:arm64-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows-static failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows-static-md failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x86-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```

This is a compiler behavior change or bug in VS 2022 16.2:

```
C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1432~1.313\bin\Hostx64\x86\cl.exe   /TP -DGRAPHQL_DLLEXPORTS -DIMPL_GRAPHQLSERVICE_DLL -Dgraphqlservice_EXPORTS -ID:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\..\include -ID:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\..\PEGTL\include -external:ID:\installed\x86-windows\include -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP  /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /W4 /WX /permissive- -std:c++20 /showIncludes /Fosrc\CMakeFiles\graphqlservice.dir\GraphQLService.cpp.obj /Fdsrc\CMakeFiles\graphqlservice.dir\ /FS -c D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\GraphQLService.cpp
cl : Command line warning D9025 : overriding '/W3' with '/W4'
D:\installed\x86-windows\include\tao\pegtl/demangle.hpp(147): error C2338: static_assert failed: 'internal::dependent_true< T > && ( begin != std::string_view::npos )'
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\include\graphqlservice/internal/SyntaxTree.h(120): note: see reference to function template instantiation 'std::string_view tao::graphqlpeg::demangle<graphql::peg::variable_value>(void) noexcept' being compiled
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\include\graphqlservice/internal/SyntaxTree.h(53): note: see reference to function template instantiation 'std::string_view graphql::peg::ast_node::type_name<graphql::peg::variable_value>(void) noexcept' being compiled
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\GraphQLService.cpp(310): note: see reference to function template instantiation 'bool graphql::peg::ast_node::is_type<graphql::peg::variable_value>(void) noexcept const' being compiled
ninja: build stopped: subcommand failed.
```

This is a compiler behavior change in 17.2. @wravery do you have comments?

```
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```

qtwebengine doesn't report logs, but I'm pretty sure it's this ICE: https://developercommunity.visualstudio.com/t/Visual-Studio-2022-v1720-reports-fata/10039296 @Neumann-A

```
REGRESSION: rsocket:x64-windows failed with BUILD_FAILED. If expected, add rsocket:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: rsocket:x64-windows-static failed with BUILD_FAILED. If expected, add rsocket:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: rsocket:x64-windows-static-md failed with BUILD_FAILED. If expected, add rsocket:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```

ICE :( https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/1490389

Other changes:
* Removed qt5-webengine skips because they are now skipped by a "supports" clause.
* Removed ctp skips because asset caching exists.

* [stxxl] Guard definition of log2 for current MSVCs.
  • Loading branch information
BillyONeal authored May 17, 2022
1 parent d132d12 commit 22b4033
Show file tree
Hide file tree
Showing 10 changed files with 304 additions and 299 deletions.
529 changes: 273 additions & 256 deletions ports/stxxl/0001-fix-visual-studio.patch

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions ports/stxxl/vcpkg.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "stxxl",
"version-string": "2018-11-15",
"port-version": 4,
"version-date": "2018-11-15",
"port-version": 5,
"description": "Standard Template Library for Extra Large Data Sets"
}
4 changes: 2 additions & 2 deletions scripts/azure-pipelines/azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
# SPDX-License-Identifier: MIT
#
variables:
windows-pool: 'PrWin-2022-04-13'
linux-pool: 'PrLin-2022-04-13'
windows-pool: 'PrWin-2022-05-16'
linux-pool: 'PrLin-2022-05-16'
osx-pool: 'PrOsx-2022-02-04'

jobs:
Expand Down
18 changes: 4 additions & 14 deletions scripts/azure-pipelines/linux/provision-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ APT_PACKAGES="at curl unzip tar libxt-dev gperf libxaw7-dev cifs-utils \
libxcursor-dev yasm libnuma1 libnuma-dev python-six python3-six python-yaml \
flex libbison-dev autoconf libudev-dev libncurses5-dev libtool libxrandr-dev \
xutils-dev dh-autoreconf autoconf-archive libgles2-mesa-dev ruby-full \
pkg-config meson"
pkg-config meson nasm cmake ninja-build"

# Additionally required by qt5-base
APT_PACKAGES="$APT_PACKAGES libxext-dev libxfixes-dev libxrender-dev \
Expand Down Expand Up @@ -58,27 +58,17 @@ APT_PACKAGES="$APT_PACKAGES liblttng-ust0 libkrb5-3 zlib1g libicu66"

apt-get -y install $APT_PACKAGES

# Install newer version of nasm than the apt package, required by intel-ipsec
mkdir /tmp/nasm
cd /tmp/nasm
curl -O https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/nasm-2.15.05.tar.gz
tar -xf nasm-2.15.05.tar.gz
cd nasm-2.15.05/
./configure --prefix=/usr && make -j
make install
cd ~

# Install the latest Haskell stack
# Install the latest Haskell stack for bond
curl -sSL https://get.haskellstack.org/ | sh

# Install CUDA
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
apt-get -y update
apt-get install -y --no-install-recommends cuda-compiler-11-6 cuda-libraries-dev-11-6 cuda-driver-dev-11-6 \
cuda-cudart-dev-11-6 libcublas-11-6 libcurand-dev-11-6 libcudnn8-dev libnccl2 libnccl-dev
cuda-cudart-dev-11-6 libcublas-11-6 libcurand-dev-11-6 cuda-nvml-dev-11-6 libcudnn8-dev libnccl2 libnccl-dev

# Install PowerShell
wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
Expand Down
3 changes: 0 additions & 3 deletions scripts/azure-pipelines/windows/azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,6 @@ jobs:
pwsh: true
- script: .\bootstrap-vcpkg.bat
displayName: 'Bootstrap vcpkg'
- script: |
if exist ${{ variables.VCPKG_DOWNLOADS }} rmdir /S /Q ${{ variables.VCPKG_DOWNLOADS }} 2>&1
displayName: 'Clean downloads'
# Note that we run docs checks before PR checks because they are likely to invalidate a whole run anyway
- task: Powershell@2
displayName: 'Generate Documentation'
Expand Down
2 changes: 1 addition & 1 deletion scripts/azure-pipelines/windows/deploy-pwsh.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@

# REPLACE WITH UTILITY-PREFIX.ps1

$PwshUrl = 'https://github.com/PowerShell/PowerShell/releases/download/v7.2.2/PowerShell-7.2.2-win-x64.msi'
$PwshUrl = 'https://github.com/PowerShell/PowerShell/releases/download/v7.2.3/PowerShell-7.2.3-win-x64.msi'
InstallMSI -Url $PwshUrl -Name 'PowerShell Core'
28 changes: 12 additions & 16 deletions scripts/ci.baseline.txt
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,12 @@ cpp-netlib:x64-uwp=fail
cppcoro:x64-linux=fail
cppcoro:arm-uwp=fail
cppcoro:x64-uwp=fail
# VS 2022 17.2 miscompiles this:
cppgraphqlgen:arm64-windows=fail
cppgraphqlgen:x64-windows=fail
cppgraphqlgen:x64-windows-static=fail
cppgraphqlgen:x64-windows-static-md=fail
cppgraphqlgen:x86-windows=fail
# The x64-linux pipeline uses gcc 9.3.0, which lacks C++20 coroutine support.
# This is known to work on x64-linux as of gcc 10.3.0.
cppgraphqlgen:x64-linux=fail
Expand Down Expand Up @@ -648,7 +654,6 @@ mecab:x64-windows-static-md=skip
mecab:x86-windows = skip
# Due to static crt.
mesa:x64-windows-static=fail
mesa:x64-windows-static-md=fail
# Missing dependent libraries.
mesa:x64-linux=fail
mesa:x64-osx=fail
Expand Down Expand Up @@ -933,9 +938,8 @@ qt5-webengine:x64-osx=fail
# Succesful built requires protobuf to be installed after qt5-webengine not before. Otherwise the build picks up the wrong headers from inside vcpkg.
qt5-webengine:x64-windows=skip
qt5-webengine:x86-windows=skip
# Static builds of qt5-webengine are not supported by the port itself
qt5-webengine:x64-windows-static=skip
qt5-webengine:x64-windows-static-md=skip
# VS2022 17.2 ICE https://developercommunity.visualstudio.com/t/Visual-Studio-2022-v1720-reports-fata/10039296
qtwebengine:x64-windows=fail
# Missing system libraries
qt5-x11extras:x64-osx=fail
# Missing libraries
Expand Down Expand Up @@ -1019,6 +1023,10 @@ rsasynccpp:x64-osx=fail
rsm-binary-io:x64-linux=fail
# Requires g++10 but CI compiler only has g++9
rsm-bsa:x64-linux=fail
# VS2022 17.2 ICE https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/1490389
rsocket:x64-windows=fail
rsocket:x64-windows-static=fail
rsocket:x64-windows-static-md=fail
rtlsdr:x64-uwp=fail
rtlsdr:arm64-windows=fail
rtlsdr:arm-uwp=fail
Expand Down Expand Up @@ -1247,18 +1255,6 @@ yara:x64-uwp=fail
z3:arm-uwp=fail
z3:x64-uwp=fail

# Official downloading server of CTP library is only guaranteed to be available during trading hours of China futures market
# Skip CI to avoid random failures
ctp:arm64-windows=skip
ctp:arm-uwp=skip
ctp:x64-linux=skip
ctp:x64-osx=skip
ctp:x64-uwp=skip
ctp:x64-windows=skip
ctp:x64-windows-static=skip
ctp:x64-windows-static-md=skip
ctp:x86-windows=skip

# clapack is replaced by lapack-reference on the platforms lapack-reference supports
clapack:x64-linux=skip
clapack:x64-osx=skip
Expand Down
8 changes: 4 additions & 4 deletions scripts/vcpkgTools.xml
Original file line number Diff line number Diff line change
Expand Up @@ -187,11 +187,11 @@
<archiveName>ninja-freebsd-1.8.2.zip</archiveName>
</tool>
<tool name="powershell-core" os="windows">
<version>7.2.1</version>
<version>7.2.3</version>
<exeRelativePath>pwsh.exe</exeRelativePath>
<url>https://github.com/PowerShell/PowerShell/releases/download/v7.2.1/PowerShell-7.2.1-win-x86.zip</url>
<sha512>f7a1d7de59c9400ee7d4f988a35daf6f52aef3206a9de9948e5f8d81b7e838301e07491b9cab417fc58f9fd2eb28773175e86a06158ae868c7ef076cc4be7882</sha512>
<archiveName>PowerShell-7.2.1-win-x86.zip</archiveName>
<url>https://github.com/PowerShell/PowerShell/releases/download/v7.2.3/PowerShell-7.2.3-win-x86.zip</url>
<sha512>4b275fd1dc6430e0d7b77d4c34ec54ad47502602592ebc1ec99c88610ec24d3adca27a41896dc4767ac820ad7e000f9c2b1d678b05e9dd76086ff52a11976f80</sha512>
<archiveName>PowerShell-7.2.3-win-x86.zip</archiveName>
</tool>
<tool name="node" os="windows">
<version>16.12.0</version>
Expand Down
2 changes: 1 addition & 1 deletion versions/baseline.json
Original file line number Diff line number Diff line change
Expand Up @@ -6830,7 +6830,7 @@
},
"stxxl": {
"baseline": "2018-11-15",
"port-version": 4
"port-version": 5
},
"suitesparse": {
"baseline": "5.8.0",
Expand Down
5 changes: 5 additions & 0 deletions versions/s-/stxxl.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
{
"versions": [
{
"git-tree": "769f42922523b172d890b697a65096ca47e06ff8",
"version-date": "2018-11-15",
"port-version": 5
},
{
"git-tree": "081a4c6de6290ce05297cc873fa0fde245133a8a",
"version-string": "2018-11-15",
Expand Down

0 comments on commit 22b4033

Please sign in to comment.