OSC has been using https://github.com/hpc2n/spank-private-tmp/tree/20.02.2 with SLURM 20.11 to do a private /dev/shm and it has worked nicely. That SPANK plugin supports multiple private mounts and we are evaluating also doing a private /tmp. If we moved to job_container/tmpfs it looks like we'd be limited to /dev/shm only and not both /dev/shm and /tmp. Is it possible with job_container/tmpfs to setup multiple private locations like /dev/shm and /tmp ? My read of the config docs and my initial read of code is that only one BasePath per either config or node group is allowed. Maybe this needs to be an enhancement if that's not possible.
I'm going to put this as a SEV5 (RFE) while I look into it. This plugin is new, so the limitations have not all been tested.
The paths (/dev/shm and /tmp) are hardcoded in the plugin currently. I will start the normal process for RFE review.
As far as I can tell /dev/shm is already handled by tmpfs plugin and is not configurable (i.e., plugin always does it):
job_container_tmpfs.c:static int _mount_private_shm(void)
job_container_tmpfs.c: rc = umount("/dev/shm");
job_container_tmpfs.c: error("%s: umount /dev/shm failed: %s\n",
job_container_tmpfs.c: rc = mount("tmpfs", "/dev/shm", "tmpfs", 0, NULL);
job_container_tmpfs.c: error("%s: mounting private /dev/shm failed: %s\n",
job_container_tmpfs.c: rc = _mount_private_shm();
job_container_tmpfs.c: error("%s: could not mount private shm",
Our problem is that we want to have additional "private" temp directories. We currently do this with /tmp, /var/tmp and /var/lock. Maybe an option Dirs that can default to /tmp but allows more than that.
We actually had an earlier version of NERSC plugin that we modified to do that but the version that got merged has diverged too much for this to be directly useful.
For OSC, this RFE maybe not useful since this was initially opened not fully understanding how the job_container/tmpfs plugin worked. I opened RFE #11135 that would actually allow OSC to use this plugin in our current environment.
I going to mark this as a duplicate of bug#11135 then.
*** This bug has been marked as a duplicate of bug 11135 ***