Merge pull request #49 from DeiC-HPC/singularity_fix_perms

Create singularity sandbox with '--fix-perms' to correctly handle file permissions

cotainr/container.py

@@ -99,6 +99,7 @@ def __enter__(self):
                     "build",
                     "--force",  # sandbox_dir.mkdir() checks for existing sandbox image
                     "--sandbox",
+                    "--fix-perms",
                     self.sandbox_dir,
                     self.base_image,
                 ]

cotainr/tests/container/test_singularity_sandbox.py

@@ -107,6 +107,20 @@ def test_add_verbosity_arg(self, capsys, patch_disable_stream_subprocess):
         stdout_lines = capsys.readouterr().out.rstrip("\n").split("\n")
         assert "args=['singularity', '-q', " in stdout_lines[-1]
 
+    def test_fix_perms_on_oci_docker_images(self, tmp_path):
+        # Tests correct permission handling in relation to the error:
+        #   FATAL:   While performing build: packer failed to pack: copy Failed:
+        #   symlink GlobalSign_Root_R46.pem
+        #   .../var/lib/ca-certificates/openssl/002c0b4f.0: permission denied
+        # which seems to be a problem with all SUSE based docker images.
+        # https://github.com/DeiC-HPC/cotainr/issues/48
+        image_path = tmp_path / "image_6021"
+        assert not image_path.exists()
+        with SingularitySandbox(base_image="docker://opensuse/leap:15.4") as sandbox:
+            sandbox.build_image(path=image_path)
+
+        assert image_path.exists()
+
     def test_overwrite_existing_image(self, data_cached_alpine_sif, tmp_path):
         existing_singularity_image_path = tmp_path / "existing_image_6021"
         existing_singularity_image_path.write_bytes(data_cached_alpine_sif.read_bytes())

doc/development/technical_motivation.rst

@@ -17,10 +17,10 @@ However, this comes at the cost of both a much more manual and tedious build pro
 
 Container sandbox design
 ------------------------
-When using :code:`cotainr build`, containers are built using a sandbox, for now a `Singularity`_/`Apptainer`_ sandbox, i.e. a temporary folder is created containing the base container content. The requested software and its configuration, e.g. a :ref:`conda environment <conda_environments>` is then packed into this sandbox using `Singularity`_/`Apptainer`_  as a chroot bootstrapper. Once everything is in place in the sandbox, it is converted to a SIF image file. Finally, everything is cleaned-up and the sandbox directory is removed.
+When using :code:`cotainr build`, containers are built using a sandbox, for now a `Singularity`_/`Apptainer`_ sandbox, i.e. a temporary folder is created containing the base container content. The sandbox is created using the `--fix-perms` option to ensure owner rwX permissions for all files in the container. The requested software and its configuration, e.g. a :ref:`conda environment <conda_environments>` is then packed into this sandbox using `Singularity`_/`Apptainer`_  as a chroot bootstrapper. Once everything is in place in the sandbox, it is converted to a SIF image file. Finally, everything is cleaned-up and the sandbox directory is removed.
 
 
-`cotainr` spefific implementation details
+`cotainr` specific implementation details
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The container sandbox is implemented in the :mod:`cotainr.container` module, specifically in the :class:`cotainr.container.SingularitySandbox` class which is used as a `context manager <https://docs.python.org/3/reference/datamodel.html#context-managers>`_. Running a command in the sandbox context is wrapped as a :mod:`subprocess` call to :code:`singularity exec`.
 
@@ -31,7 +31,7 @@ Limitations
 -----------
 Building containers in user space comes with the following limitations:
 
-- We are unable to correctly handle file permissions that should be set with root privileges.
+- We are unable to correctly handle file permissions that should be set with root privileges. We are forcing owner rwX permission on all files using the `--fix-perms` option to `singularity build`, `as is also implied in the most basic Apptainer fakeroot builds <https://apptainer.org/docs/user/latest/fakeroot.html#build>`_.
 - You can only install software in user space in the container, i.e. there is no :code:`sudo apt install` or the like.