Upgrade container without loosing data

Rancher 1.x
1

I’ve got this docker-compose.yml from the community catalog

Upon using docker-compose up --upgrade both containers gitlab-server and gitlab-data are upgraded and therefore all data is lost in gitlab-data.

How can I prevent that? I’m wondering why gitlab-data is using the image gitlab/gitlab-ce:latest anyways. Maybe someone can enlighten me.

gitlab-server:
  hostname: host.name.com
  ports:
  - 8101:22/tcp
  - 8100:8100/tcp
  labels:
    io.rancher.sidekicks: gitlab-data
  environment:
    GITLAB_OMNIBUS_CONFIG: |
      external_url 'http://host.name.com:8100'
      gitlab_rails['gitlab_shell_ssh_port'] = 8101
  image: gitlab/gitlab-ce:latest
  volumes_from:
  - gitlab-data

gitlab-data:
  labels:
    io.rancher.container.start_once: 'true'
  entrypoint:
  - /bin/true
  hostname: gitdata
  image: gitlab/gitlab-ce:latest
  volumes:
  - /etc/gitlab
  - /var/log/gitlab
  - /var/opt/gitlab
2

HI,
gitlab-data is a data container (https://docs.docker.com/engine/userguide/containers/dockervolumes/).
This kind of container is present to avoid this kind of issue (data loss).
Note that 3 volumes are present on this container.
But these volumes are not mapped on the host directory.
So, when you update your container, maybe you’ve got dangling volumes (refer to the docker volumes doc to find them).
You have to map these volume to some host directories, and ideally point to a volume from a storage pool to have some replication (i.e for example a glusterFS volume).
Refer to the rancher documentation for storage pool.
hope it helps,
Charles.

1 Like
3

Thanks a lot, Charles. So mapping data volumes to the host is the recommended way to persist data as of now? I thought you could have data Containers which you replicate instead?

Do you have a clue why git-data uses the same Base Image as the server Container?

Does the new volume API introduced in Docker 1.9 change anything of that procedure?

Sorry for all the questions but it’s difficult to get started in container-land :slight_smile:

4

Hi,my response was not very clear:
to save your data, a data container without any mapping on the host can be the solution (but you should always have a container referring to this data container to not have a dangling volume, and your host must not crash).
But to be sure this container will not be lost, you have multiple options :

  • install this container on multiple hosts but data will not be consistent/synchronized (it’s ok for read-only data)
  • have a container which has got a replication logic (mysql/galera cluster replication system for a database example)
  • backup your data frequently (but you will loose data between backups)
  • map your volume to the host with a network mapping : it can be a NFS system, a gluster FS or another one supported by the host (or another container).Rancher permits to install this kind of network mounting/filesystem replication with dedicated rancher containers (NFS or GlusterFS are supported), with data pool.
    So, to use this kind of solution, you have to install a data pool via rancher, and point your volume from your data container to a path handled by the data pool managed by rancher.

best regards,

Charles.