Impossible to share enroot container between users

[arnaudfroidmont]

Right now, on a cluster with many users, each user would need to run his own container from the shared image. When trying to run the container from another user, all kinds of permissions are missing. Would there be a way to enable permission on those files for a specific group rather than for a specific user? This would allow to avoid creating the same container multiple times.

[3XX0]

I guess you could do that by setting ENROOT_DATA_PATH to a well known directory accessible by your users and with the SGID bit set to a common group

[pvaldria]

We have a common group: groupa and all users are part of that group. The ENROOT_DATA_PATH is set to /mnt/localdisk/enroot and the "groupa" has rwx permissions to the folder.

How is enroot configured?

[opc@compute-permanent-node-137 ~]$ cat /etc/enroot/enroot.conf | grep -v "^#"
ENROOT_RUNTIME_PATH        /mnt/localdisk/enroot_runtime
ENROOT_CACHE_PATH          /mnt/localdisk/enroot_cache
ENROOT_DATA_PATH           /mnt/localdisk/enroot
ENROOT_TEMP_PATH           /mnt/localdisk/enroot_tmp

[opc@compute-permanent-node-137 ~]$ ls -l  /mnt/localdisk/
total 4
drwxrwx--- 4 opc groupa   52 May 17 19:30 enroot
drwxrwx--- 3 opc groupa 4096 May 17 18:25 enroot_cache
drwxrwxrwx 2 opc groupa    6 May 17 18:23 enroot_runtime
drwxrwx--- 2 opc groupa    6 May 17 18:27 enroot_tmp
drwxrwxr-x 7 opc groupa  101 May 17 18:27 nfs
[opc@compute-permanent-node-137 ~]$

As opc user, who ran the "enroot create", the below works, but not when another user (pinkesh4) of the group "groupa" tries to do "enroot start xxxx".

[opc@compute-permanent-node-137 scratch]$ enroot start ubuntu_opc
opc@compute-permanent-node-137:/$

As pinkesh4 user, it fails

[opc@compute-permanent-node-137 scratch]$ su pinkesh4
Password: xxxxxx

[pinkesh4@compute-permanent-node-137 scratch]$ enroot start ubuntu_opc
touch: cannot touch ‘/mnt/localdisk/enroot/ubuntu_opc/etc/passwd’: Permission denied
touch: cannot touch ‘/mnt/localdisk/enroot/ubuntu_opc/etc/group’: Permission denied
[ERROR] /etc/enroot/hooks.d/10-shadow.sh exited with return code 1

The above files belong to the container are owned by opc, who created it and hence other users like pinkesh4 of the same group: groupa, don't have permissions and I don't think its best practice to just change permissions on the container files like below to give read and/or write access to the group.

[pinkesh4@compute-permanent-node-137 scratch]$ ls -l  /mnt/localdisk/enroot/ubuntu_opc/etc/  | egrep "passwd|group"
-rw------- 1 opc groupa   458 May 25 01:38 group
-rw------- 1 opc groupa   919 May 25 01:38 passwd
[pinkesh4@compute-permanent-node-137 scratch]$

[3XX0]

You're going to have to adjust the permissions of your container if you want multiple users to use it, there is no way around that.
Depending on the container and file in question you could do one of the following:

• Change your umask on enroot create
• Change the group permissions on files missing it
• Set ACL as required if your filesystem supports it

[pvaldria]

I tried "change the group permissions on files missing it", but enroot start overrides it again once it runs.

chmod 660 /mnt/localdisk/enroot/ubuntu_opc/etc/passwd
chmod 660 /mnt/localdisk/enroot/ubuntu_opc/etc/group

[opc@compute-permanent-node-137 ~]$ ls -l  /mnt/localdisk/enroot/ubuntu_opc/etc/  | egrep "passwd|group"
-rw-rw---- 1 opc groupa   458 Jun  7 17:16 group
-rw-rw---- 1 opc groupa  919 Jun  7 17:16 passwd

Run enroot start

[opc@compute-permanent-node-137 ~]$ enroot start ubuntu_opc
opc@compute-permanent-node-137:/$ exit 
exit
[opc@compute-permanent-node-137 ~]$ ls -l  /mnt/localdisk/enroot/ubuntu_opc/etc/  | egrep "passwd|group"
-rw------- 1 opc opc     458 Jun  7 17:20 group
-rw------- 1 opc opc     919 Jun  7 17:20 passwd

[pvaldria]

I tried change umask on enroot create , and still it doesn't work .

[opc@compute-permanent-node-137 scratch]$ umask 002
[opc@compute-permanent-node-137 scratch]$ enroot create /nfs/scratch/ubuntu_opc.sqsh

Even though I am setting the umask to rwx for group, it is getting ignored or overwritten by enroot.

[opc@compute-permanent-node-137 scratch]$ ls -l  /mnt/localdisk/enroot/ubuntu_opc/etc/  | egrep "passwd|group"
-rw-r--r-- 1 opc opc   446 Apr 28 12:01 group
-rw-r--r-- 1 opc opc   922 Apr 28 12:01 passwd

Let me know if my test steps are correct.

So seems like this is uncharted territory and it's not validated if containers can be shared by multiple users. Is the recommended best practice to share image sqsh file, but not the container among users ?

[3XX0]

This would be the responsibility of the shadow hook.
You can modify it to chmod these files, eventually we could make it the default to preserve the original permissions.