Skip to content

Commit

Permalink
[SPARK-50300][BUILD] Use mirror host instead of archive.apache.org
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to use `mirror host` instead of `archive.apache.org`.

### Why are the changes needed?

Currently, Apache Spark CI is flaky due to the checksum download failure like the following. It took over 9 minutes and failed eventually.

- https://github.com/apache/spark/actions/runs/11818847971/job/32927380452
- https://github.com/apache/spark/actions/runs/11818847971/job/32927382179
```
exec: curl --retry 3 --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz?action=download
exec: curl --retry 3 --silent --show-error -L https://archive.apache.org/dist/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz.sha512
curl: (28) Failed to connect to archive.apache.org port 443 after 135199 ms: Connection timed out
curl: (28) Failed to connect to archive.apache.org port 443 after 134166 ms: Connection timed out
curl: (28) Failed to connect to archive.apache.org port 443 after 135213 ms: Connection timed out
curl: (28) Failed to connect to archive.apache.org port 443 after 135260 ms: Connection timed out
Verifying checksum from /home/runner/work/spark/spark/build/apache-maven-3.9.9-bin.tar.gz.sha512
shasum: /home/runner/work/spark/spark/build/apache-maven-3.9.9-bin.tar.gz.sha512: no properly formatted SHA checksum lines found
Bad checksum from https://archive.apache.org/dist/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz.sha512
Error: Process completed with exit code 2.
```

**BEFORE**
```
$ build/mvn clean
exec: curl --retry 3 --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz?action=download
exec: curl --retry 3 --silent --show-error -L https://archive.apache.org/dist/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz.sha512
```

**AFTER**
```
$ build/mvn clean
exec: curl --retry 3 --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz?action=download
exec: curl --retry 3 --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz.sha512?action=download
```

### Does this PR introduce _any_ user-facing change?

No, this is a dev-only change.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#48836 from dongjoon-hyun/SPARK-50300.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Nov 13, 2024
1 parent 05508cf commit 5cc60f4
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion build/mvn
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ install_app() {
local binary="${_DIR}/$6"
local remote_tarball="${mirror_host}/${url_path}${url_query}"
local local_checksum="${local_tarball}.${checksum_suffix}"
local remote_checksum="https://archive.apache.org/dist/${url_path}.${checksum_suffix}"
local remote_checksum="${mirror_host}/${url_path}.${checksum_suffix}${url_query}"

local curl_opts="--retry 3 --silent --show-error -L"
local wget_opts="--no-verbose"
Expand Down

0 comments on commit 5cc60f4

Please sign in to comment.