Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile for reproducible build may have an issue #1218

Open
77corn opened this issue May 16, 2024 · 6 comments
Open

Dockerfile for reproducible build may have an issue #1218

77corn opened this issue May 16, 2024 · 6 comments

Comments

@77corn
Copy link

77corn commented May 16, 2024

[+] Building 66.6s (9/38)                                                                                                                                                                                 docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                                                      0.0s
 => => transferring dockerfile: 5.00kB                                                                                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                                                                                         0.0s
 => => transferring context: 74B                                                                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                                                           2.3s
 => [internal] load build context                                                                                                                                                                                         0.0s
 => => transferring context: 285.32kB                                                                                                                                                                                     0.0s
 => CACHED [ 1/34] FROM docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2                                                                                            0.0s
 => [ 2/34] RUN apt-get update && apt-get upgrade -y && apt-get install -y wget nano rsync curl gnupg2 jq unzip bzip2                                                                                                    29.7s
 => [ 3/34] RUN echo "deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-15 main" >> /etc/apt/sources.list &&     echo "deb-src http://apt.llvm.org/jammy/ llvm-toolchain-jammy-15 main" >> /etc/apt/sources.list &&     0.5s
 => [ 4/34] RUN mkdir ~/Downloads &&    cd ~/Downloads &&    wget -O gcc.tar.bz2 https://developer.arm.com/-/media/Files/downloads/gnu-rm/8-2018q4/gcc-arm-none-eabi-8-2018-q4-major-linux.tar.bz2?revision=d830f9dd-cd  27.5s 
 => ERROR [ 5/34] RUN apt-get update && apt-get install -y     build-essential     llvm-15     gcc-10     binutils     valgrind     cmake     git     autotools-dev     automake     autoconf     libtool     pkg-config  6.5s 
------                                                                                                                                                                                                                         
 > [ 5/34] RUN apt-get update && apt-get install -y     build-essential     llvm-15     gcc-10     binutils     valgrind     cmake     git     autotools-dev     automake     autoconf     libtool     pkg-config     libcmocka-dev     libc6-i386     lib32stdc++6     lib32z1     libusb-1.0-0-dev     libudev-dev     libhidapi-dev:                                                                                                                       
4.794 Hit:2 http://ports.ubuntu.com/ubuntu-ports jammy InRelease                                                                                                                                                               
4.825 Hit:3 http://ports.ubuntu.com/ubuntu-ports jammy-updates InRelease                                                                                                                                                       
4.875 Hit:4 http://ports.ubuntu.com/ubuntu-ports jammy-backports InRelease                                                                                                                                                     
4.933 Get:1 https://apt.llvm.org/jammy llvm-toolchain-jammy-15 InRelease [6833 B]
4.942 Hit:5 http://ports.ubuntu.com/ubuntu-ports jammy-security InRelease
5.034 Get:6 https://apt.llvm.org/jammy llvm-toolchain-jammy-15/main Sources [1997 B]
5.071 Get:7 https://apt.llvm.org/jammy llvm-toolchain-jammy-15/main arm64 Packages [12.1 kB]
5.077 Fetched 20.9 kB in 5s (4400 B/s)
5.077 Reading package lists...
5.538 W: http://apt.llvm.org/jammy/dists/llvm-toolchain-jammy-15/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
5.561 Reading package lists...
6.192 Building dependency tree...
6.365 Reading state information...
6.414 E: Unable to locate package libc6-i386
6.414 E: Unable to locate package lib32stdc++6
6.414 E: Couldn't find any package by regex 'lib32stdc++6'
6.414 E: Unable to locate package lib32z1
------
Dockerfile:38
--------------------
  37 |     # Tools for building
  38 | >>> RUN apt-get update && apt-get install -y \
  39 | >>>     build-essential \
  40 | >>>     llvm-15 \
  41 | >>>     gcc-10 \
  42 | >>>     binutils \
  43 | >>>     valgrind \
  44 | >>>     cmake \
  45 | >>>     git \
  46 | >>>     autotools-dev \
  47 | >>>     automake \
  48 | >>>     autoconf \
  49 | >>>     libtool \
  50 | >>>     pkg-config \
  51 | >>>     libcmocka-dev \
  52 | >>>     libc6-i386 \
  53 | >>>     lib32stdc++6 \
  54 | >>>     lib32z1 \
  55 | >>>     libusb-1.0-0-dev \
  56 | >>>     libudev-dev \
  57 | >>>     libhidapi-dev
  58 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install -y     build-essential     llvm-15     gcc-10     binutils     valgrind     cmake     git     autotools-dev     automake     autoconf     libtool     pkg-config     libcmocka-dev     libc6-i386     lib32stdc++6     lib32z1     libusb-1.0-0-dev     libudev-dev     libhidapi-dev" did not complete successfully: exit code: 100

Seems like that some packages are not in the standard repository and therefore cannot be found.

@benma
Copy link
Collaborator

benma commented May 20, 2024

Maybe a transient issue? The package should be available: https://packages.ubuntu.com/jammy/lib32z1

@NickeZ
Copy link
Collaborator

NickeZ commented May 21, 2024

Could it be that you are using an arm-based mac? In that case you will have to specify the amd64 platform since the way the docker image is constructed it requires that.

docker build --platform linux/amd64 .

@sutterseba
Copy link
Contributor

--platform linux/amd64

Were you able to build this way? Simply specifying the platform never worked for me. I have been trying to build on my M1 device multiple times in the past, but always ended up with a segfault a few minutes into the build.

I never pursued it further and just used a different machine, but it would be great to know if there is a working and straight-forward solution.

@NickeZ
Copy link
Collaborator

NickeZ commented Jun 11, 2024

I didn't actually try to build. I guess the Apple emulation of x86 is only so good. Probably better to see if it is possible to find a compiler that can build on the arm platform.

@schmitmd
Copy link

schmitmd commented Oct 26, 2024

Sorta piggybacking on this issue (and mine is unrelated to the above), but I found a slight "unexpected behavior" situation introduced recently. Figured I'd put it in here in case others run into it (and maybe ask for a README tweak on https://github.com/BitBoxSwiss/bitbox02-firmware/blob/master/releases/README.md ?).

I was running into an issue trying to build firmware/v.9.21.0 and it turned out to be a Docker problem. Building firmware/v9.19.0 still worked just fine, but as of 2e38134 , The build process now requires BuildKit in order to inject the necessary ARG TARGETARCH properly and succeed in downloading Go for the build (see 2e38134#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R134 ).

Yes, I probably should've upgraded my Docker setup before now, but I think it's worth noting that the change forces a switch.
Running on Archlinux and using docker 1:27.3.1-1 from the extra repo, I was getting this error (because TARGETARCH was empty and thus the URL was invalid):

 ---> 08a5e7953c52
Step 27/47 : ENV GOPATH /opt/go
 ---> Running in 8ce5e302bf2a
 ---> Removed intermediate container 8ce5e302bf2a
 ---> e7ce73c9beb1
Step 28/47 : ENV GOROOT /opt/go_dist/go
 ---> Running in 86407b14a03e
 ---> Removed intermediate container 86407b14a03e
 ---> 9ba8d8463718
Step 29/47 : ENV PATH $GOROOT/bin:$GOPATH/bin:$PATH
 ---> Running in 41318af7cf48
 ---> Removed intermediate container 41318af7cf48
 ---> b09eb87f6431
Step 30/47 : RUN mkdir -p /opt/go_dist &&     curl https://dl.google.com/go/go1.19.3.linux-${TARGETARCH}.tar.gz | tar -xz -C /opt/go_dist
 ---> Running in a6ac226ea23f
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1449  100  1449    0     0   1473      0 --:--:-- --:--:-- --:--:--  1472

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
 ---> Removed intermediate container a6ac226ea23f
The command '/bin/sh -c mkdir -p /opt/go_dist &&     curl https://dl.google.com/go/go1.19.3.linux-${TARGETARCH}.tar.gz | tar -xz -C /opt/go_dist' returned a non-zero code: 2

After installing the docker-buildx package and restarting the Docker daemon, I was able to ./build.sh firmware/v9.21.0 "make firmware" and everything worked just fine (and most importantly, the checksums match).

To test if you've got things where you need them, you can explicitly export DOCKER_BUILDKIT=1 before you run build.sh and Docker will exit instead of attempting to build, screaming about not having buildx.

@NickeZ
Copy link
Collaborator

NickeZ commented Oct 26, 2024

Sorta piggybacking on this issue (and mine is unrelated to the above), but I found a slight "unexpected behavior" situation introduced recently. Figured I'd put it in here in case others run into it (and maybe ask for a README tweak on https://github.com/BitBoxSwiss/bitbox02-firmware/blob/master/releases/README.md ?).

Yes, we should probably document the minimal version of docker that is required. I think on most platforms buildx is shipped as the default builder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants