Hi!
Managed to get rancher up and running mostly fine, rancher seems like the best solution in terms simplicity.
One of the problems we are having is persistant storage, as like most tech companies we have an NFS server, I was hoping to use this for storing and sharing gitlab, postgres, redis data across containers.
We tried two things:
- Set up Convoy to mount the NFS volume
- Mount the volumes on the host and mount this to the container
We ran into a few issues with both of these:
- Needing to add the ‘insecure’ flag to the NFS server mount - as mounting a NFS server in docker seemed to map to a port over 1024
- A lot of containers (postgres) seem to chmod/chown directories on start up - they would then fail to start due to permission denied error. To get around this we needed to add no_root_squash to the nfs server mount config.
- All containers that use the NFS mount need to run in privileged mode or they will not run due to the above issue
Even now I am still getting issues with this set up especially with the gitlab container, and due to the two flags I wouldn’t feel comfortable putting this setup in a production environment.
I guess my question is what is the preferred way of dealing with persistent storage in a multi-host environment?
Would it to use data volume container/sidekick for postgres/gitlab data while keeping NFS for just placing user data (documents, backups etc)? Will this allow for scaling?
Or is there some other preferred way of dealing with this in docker? although NFS would be preferred we are open to other better ways.
Thanks for any ideas 
@allistera can you give some clarifications:
-
When you say you tried two things, do you mean you tried those as two separate approaches?
1a. If yes to 1, can you describe the second approach a little more.
1b. If yes to 1, did you see all those problems with both approaches or were some specific to the approcah
-
Did you set this up using compose? If so, would you be able to share some or all of your compose files? This seems like a great use case for us to dig into.
Hi!
Yes we tried those two solutions separately but encountered much of the same issues, except Convoy required the insecure flag added to the NFS server, this might be because it used NFS3 though…
I never used compose, although I did use the default Convoy NFS catalog item with no optional mounting params.
Let me talk though my current architecture
We are running a vSphere cluster with 2 CentOS 7 virtual machines. The latest version of docker is installed on both of these.
The rancher server container is running on the first server, as well as the NFS server. The only host added in rancher is our second CentOS 7 box.
After spinning up Convoy without the ‘insecure’ flag Convoy was unable to mount the volume, looking at the NFS server logs it seems that it was using an ‘unprivileged’ port, after adding this Convoy was able to mount the volume just fine.
My next step involved using the postgres image, mounting it’s /var/lib/postgresql directory to a new convoy location. The container wouldn’t start, complaining about permission denied while attempting to do a chmod. To get around this issue I was required to add no_root_squash to the NFS server mount options, as well as running the postgres container in privileged mode.
All of the issues seem to be the way NFS handles permissions, I am guessing block storage solutions might be a better way of sharing persistent application data.
Currently we have just set up docker data containers and they seem to have gone well so far, although we have a future user story around backing up those.
The “insecure” flag is poorly named to sound scary if enabled and largely misunderstood in 2016. It is useful for a very specific situation where you are allowing access individually to machines which have trusted root accounts, untrusted user accounts, and physical barriers preventing me from just disconnecting the ethernet cable and connecting it to my own client. Once you bring containers into the picture, very little of that is true.