I know… a Docker in Docker question. They’re a pain.
I am running Jenkins in Rancher. I have Jenkins agents/slaves setup in Rancher as well that are, obviously, containers in themselves. These agents need to run builds. The builds themselves are done in Docker containers.
There are a number of approaches to running Docker while in a container, but the current wisdom seems to suggest mounting the /var/run/docker.sock as the safest approach, so that’s what we’re trying to do. This mostly works, but as described in several places, a big issue to overcome is data volumes.
In the Jenkins agent container, I check out the code, then launch the “build” container and mount the checked out code into the build container so that it can be processed. Well, in the bind-mount approach to Docker in Docker, the docker you’re actually running is the Docker from the Host machine so when you pass a volume to be mounted, it tries to do that FROM the host machine. This fails, of course, because the code files are actually checked out in the agent container, not the host.
The articles linked above suggest data volumes to solve this, but I can’t figure out how to get that working in Rancher. Creating the data volume is no problem, and making it available to the agent container is trivial (using a Sidekick at the moment, but I could use a storage pool as well). The problem is making that same data volume accessible to the build container that I’m launching from within the agent container. Technically, I DO have access to the same data volume, as the Docker I’m using is the one from the host, however the problem is identifying the data volume so that I can link it with “–volumes-from” in the build container. I cannot figure out a consistent way to identify the correct volume. Sometimes it is something like r-jenkins-agent-workspace-volumes-1 but other times it doesn’t have a readable name but a UUID instead (after an upgrade, i think).
Any ideas here? Maybe I’m going at this in the completely wrong manner and making things way more complex than they need to be?