Install size is heavily BLOATED

[srcspider]

Before I go on let me just clarify where this is very serious issue: Docker containers. Anywhere else the "disk space is cheap" argument applies, probably.

If you install/compile in a Docker container (and you have to at the moment, since there's no prepackaged version), then you will end up with mcrouter layer in docker of 2.339 GB !! A bit of cleanup later (removing git repo, yum clean all, etc) and you go down to 1.358 GB (I'm gonna guess there's just a lot of packages that get pulled in as deps).

1.35 GB is still in the realm of nonsensical, since this has to get pushed and pulled everywhere the docker is required (resulting in considerable downtime, slowdown, etc etc). Consider that the centos:7 image is only 200 MB.

From debugging by installing to a "mcrouter-install" prefix I've noticed the the source of the disk usage to be (mainly) the following,

778M /build/mcrouter-install/pkgs
175M /build/mcrouter-install/install/lib

All these seem to be build artifacts. The lib directory there gets its sized from .a and .so files for folly and others. But I assume mcrouter is statically linked and doesn't care for any of them? I'm basing this on how the mcrouter binary is 125M itself, which seems a bit bloated to be honest.

Please either optimize the install scripts or make a new install script that's specifically designed to produce the smallest result size (cleanup after itself, no install of folly libs that were only used for building, etc).

Finally, for the sake of fixing it in the meantime, can I have some clarification on this if possible:

1. is there a way to make mcrouter compile to be smaller using the install scripts? Even if it's linked to folly it can't possibly use that much of folly--it's like it's starting it's own operating system to run.
2. is it safe to only keep /build/mcrouter-install/install/bin/mcrouter & mcpiper ? I'm only uncertain if you make use of the .so files in some strange hacky way

[CallMeFoxie]

Hi,
a bit late and unofficial reply but:
you can check dependencies with ldd.
Libfolly is NOT linked in statically. The whole mcrouter binary is not! Just as well as the libfolly has a lot of dependencies.
You can safely delete the .a files.
strip --strip-unneeded on the .so folly lib files and strip the mcrouter and mcpiper binaries. You will get down to about 85MB including dependencies for libfolly/mcrouter/mcpiper on top of base debian:jessie image (totalling only at 227MB) :)
Sadly unsure whether I can post the Dockerfile that builds the mcrouter binary :/ (work stuff)

Basically -- install all compile -dev packages, compile, uninstall all -dev packages, purge apt from unneeded pkgs, reinstall required libraries, strip libs+binaries from debug symbols, done :). Of course the install & uninstall & purge HAVE to be in one RUN layer ;)

[srcspider]

@CallMeFoxie Thanks. I'll have to look into doing that.

For anyone interested who accidentally finds this thread, the Docker setup for generating binaries posted by @netshade in #187 is the closest thing to "easy setup," when you get desperate due to dependency hell.