Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

debs.tar.gz packages take up more space than necessary #440

Open
jgallag88 opened this issue Mar 4, 2020 · 0 comments
Open

debs.tar.gz packages take up more space than necessary #440

jgallag88 opened this issue Mar 4, 2020 · 0 comments

Comments

@jgallag88
Copy link
Contributor

In order to be able to build the upgrade image, a debs.tar.gz is created for each VM artifact, containing all of the debian packages that were installed in the process of building that VM artifact. This deb.tar.gz averages a bit over 3GB in size, so for a nightly build containing all platforms and variants, these files consume about 150GB of space.

To save space, and probably some amount of time, we should consider changing debs.tar.gz from a collection of packages to a list of package names and versions. When we build the upgrade image, we would then fetch the packages we need from the apt mirrors instead of from debs.tar.gz. In addition to saving space in S3, this would also likely be faster, since it would eliminate most of the duplicated downloading of packages, as well as all of uploading, tarring, compressing, untarring, and decompressing of the debs.tar.gz files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant