Rancher Storage Driver vs Docker Volume Plugin

Rancher 1.x
1

I was trying to figure out how to solve: java.IllegalStateException: More than one volume exists for volume URI … and started reading the rancher docs on storage in more depth. I noticed that the driver concept in Rancher seems unrelated to Docker Volume Plugins.

Are Docker volume plugins supported for use in Rancher any longer?

2

may be I want to ask the same question… can we use a volume created through “docker volume” inside docker-compose.yml

3

@untoldone We are revamping our volumes documentation. Care to read the PR to see if it is any clearer?
https://github.com/rancher/rancher.github.io/pull/800/files

And here’s the rendered version, fwiw: https://github.com/LLParse/rancher.github.io/blob/402616a927129f3f32fd19c4dc856e1c2f9d8939/rancher/v1.6/en/cattle/volumes/index.md

1 Like
4

Just skimmed it, but I didn’t see anything on how Rancher storage management relates to a proper Docker Volume Plugins (the capitalized concept in Docker itself / Docker volume plugins | Docker Docs).

The terminology section states:

A volume plugin and volume driver are used interchangeably in Docker and Rancher documentation.

But doesn’t clarify if this is in reference to a Docker vs Rancher Volume Plugin. In my opinion, this clarification is important given the different behavior in Rancher vs Docker in compose files.

It would also be very helpful if you included an example v2 compose file that would match the behavior of what the v1 compose file does (e.g. is it local or external). I may be biased here, but the behavior of v1 compose had a breaking changed in some previous version of Rancher (I moved from 1.14 to 1.6 and hit this – in 1.14 it behaved like external in the old storage setup now its local?).

5

I’ve been wondering about this too. Glad to hear the documentation is getting updated. Is there any documentation on the per_container setting? I’ve seen that in a rancher catalog entry but cannot find any documentation on it.

I’ve been using Rancher since the beginning but still have no clue where to start when it comes to volume management. What with gluster, convoy, nfs, longhorn… What would be really helpful is some advice on how to select a suitable persistent storage setup as well as better communications on things that are no longer core approaches, such as, presumably, gluster.

6

Good questions @djskinner and @untoldone.

I see your point about the confusing terminology. Will try to work better definitions into the docs.

In brief, a Rancher Storage Driver provides docker volumes in Rancher. It is extremely confusing in the updated documentation to only refer to storage drivers in the docker storage context (ie devicemapper vs aufs). We’ll have to change that.

But moving on from that, a Rancher Storage Driver is a stack you install that allows you to create and use Rancher volumes with your containers. At the host level, all of our Rancher Storage Drivers (Rancher EFS, NFS, EBS) are implemented as docker volume plugins. But we see that as an implementation detail. If you have a Rancher Storae Driver installed, you can use them as volume drivers in your compose files as outlined in the above article and expect that the volume will be mounted into your container at the expected path.

In Rancher compose, you can also set a volume’s driver to a docker volume plugin that you’ve installed on your hosts out-of-band from Rancher, but Rancher won’t do any intelligent scheduling of that volumes or containers using it because it doesn’t know how to do so.

Hope that clarifies a bit.

7

I’ve referenced this discussion in the PR so that it can be updated based on some of this feedback.

8

@djskinner some guidance on current best practices is a great idea. An entire blog article or video could be devoted to that, but I’ll try to give you the summary briefly:

  • Our two fully officially supported drivers (meaning if someone pays for support, these are the two that are fully covered) are Rancher NFS and Rancher EBS. They are the most stable.
  • Rancher EFS is available, but currently “experimental,” meaning that there are likely some undiscovered or unfixed bugs.
  • Convoy still exists as a project because some use it, but it is not our current focus and we don’t recommend it if you are looking for a storage solution.
  • Gluster was a backed for convoy and we stopped pushing it because what we found is that manage a gluster cluster takes some amount of expertise on the users/admin’s part beyond what we could provide by simply standing up the stack.
9

Thanks for the reply. In response to:

As of last I tested this (I think in 1.5.x) pure docker volume plugins being used as documented in docker-compose v1 documentation didn’t work (as volume assumed local volume + in rancher, but volume didn’t exist in rancher) and didn’t work in v2 without some pretty significant manipulation of things (creating fake service that created a custom rancher storage driver, then shifting all volumes to be external volumes + creating volume template via API was only approach I found to work). I’m not sure if this is expected behavior, if yes, I think its still pretty unclear how its supposed to work in the docs, if this is unexpected, I don’t think I’ve seen bugs for this. Also possible this has been fixed since 1.5.x.