forked from apache/pulsar
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Build] Do a git shallow clone with fetch-depth of 25 in checkout (ap…
…ache#8394) ### Motivation There's a significant different in performance when using a shallow clone. 125 seconds vs. 10 seconds to clone Pulsar on a 100Mbps internet connection. 568MB vs. 16MB was transferred. In Github Actions, the speed difference with fetch-depth: 25 compared to fetch-depth: 0 for Pulsar repository is 2-4 seconds compared to 70-75 seconds. about **15-35x faster.** Besides a fast checkout, 552MB less data was transferred and less disk space will be consumed. example benchmark of cloning full vs. shallow (--depth=25): ``` $ time git clone https://github.com/apache/pulsar.git pp Cloning into 'pp'... remote: Enumerating objects: 601, done. remote: Counting objects: 100% (601/601), done. remote: Compressing objects: 100% (340/340), done. remote: Total 1207163 (delta 473), reused 296 (delta 226), pack-reused 1206562 Receiving objects: 100% (1207163/1207163), 568.24 MiB | 5.66 MiB/s, done. Resolving deltas: 100% (862029/862029), done. git clone https://github.com/apache/pulsar.git pp 122,95s user 9,60s system 105% cpu 2:05,23 total $ time git clone --depth=25 https://github.com/apache/pulsar.git pp2 Cloning into 'pp2'... remote: Enumerating objects: 7699, done. remote: Counting objects: 100% (7699/7699), done. remote: Compressing objects: 100% (5239/5239), done. remote: Total 7699 (delta 3120), reused 3707 (delta 1177), pack-reused 0 Receiving objects: 100% (7699/7699), 16.16 MiB | 2.85 MiB/s, done. Resolving deltas: 100% (3120/3120), done. git clone --depth=25 https://github.com/apache/pulsar.git pp2 2,24s user 0,64s system 31% cpu 9,152 total ``` Currently for the Pulsar Github workflows, it's only the [diff-only action](https://github.com/apache/pulsar-test-infra/blob/master/diff-only/entrypoint.sh) that depends on having all PR commits in the checked out local git repository. Therefore setting the value to 25 is reasonable. The 25th oldest commit would contain all changes so beyond that point, the diff-only action produce the correct result. That's a non critical critical issue since diff-only is used to detect if the PR contains only documentation changes. It simply wouldn't be able to do the detection if the PR has more than 25 commits. (the fetch-depth limit could be increase if 25 isn't enough) ### Modifications Change the fetch-depth of check action to 25 instead of 0 which does a full clone.
- Loading branch information
Showing
23 changed files
with
23 additions
and
23 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters