I work on an HPC and often I have to share files with other users. The most approachable solution is to have an external cloud storage and recline back and forth. However there’s some projects that are quite heavy (several TB) and that is unfeasible. We do not have a shared group. The following is the only solution I found which is not to just set al permissions to 777, and I still don’t like it.
Create a directory and set ACL to give access to the selected users. This works fine if the users create new files in there, but it does not work if they copy from somewhere else as default umask is 022. Thus the only appropriate solution is to change default umask to 002, which however affects file creation system wide. The alternative is to change permissions every time you copy something, but you all know very well that is not going to happen.
Does it really have to be such a pain in the ass?
It’s just default Linux permissions. People who get access to the HPC through the same institution are placed in the same group. As you may know scientific collaboration is quite important. Indeed when collaborating we sign all the necessary paperwork, but that does not translate to the HPC administrators who are part of a separate institution. To request a separate group you have to contact the HPC institution, they will have to contact the institutions involved. Those institutions will have to check that NDAs are already in place. If the NDAs are in place they will have to check that the data to be shared is actually covered by the project. I will have to fill a bunch of paperwork. This will be sent to an external auditor to check that everything is correct, and then everything goes back in that chain.
I already waste way too much of my time on paperwork. Worst case scenario is a collaborator leaks some data which will be published publicly in a few months anyway. And those collaborators will have access to such data anyway, just through other less comfortable means.
The fact that you’re sharing this internal policy stuff so openly is definitely a red flag.
I don’t know what your background is, this is mostly hindrances when doing research. Administration has taken over and demands deciding how research should be conducted without having any idea on how it should be conducted.
You may see it as a red flag, myself I have this very clear that I do not want to follow their bullshit. If I’m losing my job so be It, However you may be misunderstanding: I am not going to lose my job over this, nobody is getting hurt, and I am sharing a bureaucratic process that is fairly common over here in public institutions. This is not some large corporation that has to keep secret the time schedule of it’s workers: if you wish to come over I have full liberty of deciding to show you anything I have on my computer and most projects I am working on. Yes, there’s a couple things I can not show you, but everything else is my own job and up to me to show to whoever I wish. Institutions may retain part of the IP if we decide to commercialise stuff, but I am the author and I am free to share anything I don’t have an NDA on.
I am afraid you come from a very different background and you are misunderstanding my situation.
Your job as sysadmin is to adhere to your organization’s policy, no matter how stupid and hindering that policy might seem to you.
You’re knowingly giving your users a workaround to their NDA, which puts all of your jobs and your data confidentiality at risk.
You’ve got no business with root privileges.
I have no root privileges, I’m providing no NDA workaround.