Using volumes_from from a global service

Is it possible to use volumes_from a global service in rancher?

The UI definitely allows me to do it, but I can’t seem to do it from rancher-compose.

This is the sample docker-compose.yml I am passing to the rancher-compose.

kana-license-web-1:
  environment:
    env: INT_S
    instanceid: '6'
    TZ: '"Australia/Melbourne"'
  tty: true
  image: kana-web-license-fullwss:latest
  stdin_open: true
  volumes_from:
  - kana-web-data 
  - kana-introscope
  labels:
    io.rancher.scheduler.affinity:host_label: hostname=umr71hm7

kana-introscope:
  environment:
    TZ: Australia/Melbourne
  labels:
    io.rancher.scheduler.global: 'true'
  tty: true
  image: kana-introscope
  volumes:
  - /opt/middleware/introscope/current
  stdin_open: true
kana-web-data:
  environment:
    TZ: Australia/Melbourne
  labels:
    io.rancher.scheduler.global: 'true'
  tty: true
  image: kana-web-data-fullwss:54.dec15.19
  volumes:
  - /data/web/release/current
  stdin_open: true

1 Like

Just out of curiosity, what if you do it in the UI, then generate the rancher-compose.yml and see how it does it? Does that then subsequently not work?

When I add the service using the ui with volumes from global service then I can see the global service instance available on that node.

The naming appears as - STACK-CONTAINER-1_1. I have looked at the docker-compose.yml and it seems to have the alpha numeric containerid for the container instance running on that node.

I have honestly not tried using rancher-compose.yml but can give that a try too.

Seems like rancher-compose.yml also calls associated docker-compose.yml and it fails with the same error:

INFO[0009] Creating stack licenseweb
INFO[0009] Creating service kana-introscope
INFO[0009] Creating service kana-license-web-1
INFO[0009] Creating service kana-web-data
ERRO[0009] Failed Creating kana-license-web-1 : Bad response statusCode [422]. Status [422 status code 422]. Body: [code=InvalidReference, fieldName=LaunchConfigName] from [http://localhost:8080/v1/services]
ERRO[0010] Failed to start: kana-license-web-1 : Bad response statusCode [422]. Status [422 status code 422]. Body: [code=InvalidReference, fieldName=LaunchConfigName] from [http://localhost:8080/v1/services]
FATA[0010] Bad response statusCode [422]. Status [422 status code 422]. Body: [code=InvalidReference, fieldName=LaunchConfigName] from [http://localhost:8080/v1/services]

@Gaurav_Mehta, the rancher-compose command uses both a docker-compose.yml and rancher-compose.yml.

I’m pretty sure the volumes_from a global service won’t work in Rancher right now, but if you are running Docker 1.9.1 (this works in 1.9.0 but we strongly encourage 1.9.1) you could use a named volume. That creates a volume on the host (in /var/lib/docker/volumes) and will live in Rancher until the last container referencing it is removed, then the volume is also removed.

This is a really complicated example, but it makes use of named volumes: https://github.com/rancher/compose-templates/tree/master/hadoop/0.3.0

I use a variable, ${stack} and substitute it in with the env file. Something similar should address the issue you are trying to resolve.

1 Like

Thanks Bill. I will give it a try today.

Hi Bill,

I tried the following: I have two hosts: A and B in the rancher environment. I have added the two named volumes to both these hosts:

docker create -v /opt/middleware/introscope/current --name kana-introscope registry/kana-introscope:latest /bin/true
docker create -v /data/web/release/current --name kana-web-data registry/kana-web-data-fullwss:54.dec15.19 /bin/true

If I now use these volumes using docker run or docker-compose they work fine.

docker run --volumes-from kana-web-data --volumes-from kana-introscope --name kana-license-web-1 -e env=INT_S -e instanceid=6 -e TZ="Australia/Melbourne" registry/kana-web-license-fullwss:latest bash

However if I try and use the same using with rancher-compose I get the following error:

[root@umr71hm7 rancher-compose-v0.5.3]# ./rancher-compose --url http://localhost:8080/v1/projects/1a7 --access-key ABC --secret-key DEF -p licenseweb -f license-server.yml up
INFO[0000] Creating stack licenseweb
INFO[0000] Creating service kana-license-web-1
ERRO[0000] Failed Creating kana-license-web-1 : Bad response statusCode [422]. Status [422 status code 422]. Body: [fieldName=LaunchConfigName, code=InvalidReference] from [http://spr71dev10:8080/v1/services]
ERRO[0000] Failed to start: kana-license-web-1 : Bad response statusCode [422]. Status [422 status code 422]. Body: [fieldName=LaunchConfigName, code=InvalidReference] from [http://spr71dev10:8080/v1/services]
FATA[0000] Bad response statusCode [422]. Status [422 status code 422]. Body: [fieldName=LaunchConfigName, code=InvalidReference] from [http://spr71dev10:8080/v1/services]

This is the docker compose i have tried to use.

 kana-license-web-1:
  tty: true
  image: registry/kana-web-license-fullwss:latest
  volumes_from:
  - kana-web-data
  - kana-introscope
  stdin_open: true
  labels:
    io.rancher.container.network: true
  restart: always
  environment:
    env: INT_S
    instanceid: 6
    TZ: "Australia/Melbourne"

I can add the same container using the rancher UI, in which case the container name in volumes_from is replaced with the alphanumeric container id.

I looked at your docker-compose in the hadoop project but could not make out where you had defined the named volumes.

Cheers,
Gaurav

In docker 1.9.x you can do docker volume create --name=myvolume

then when you start a container its docker run -v myvolume:/place/inside/the/container.

In the compose files (based on original compose file) do something like:

kana-license-web-1:
 ...
 volumes:
    - kana-introscope:/opt/middleware/introscope/current
    - kana-web-data:/data/web/release/current
kana-introscope:
 ... 
 volumes:
    - kana-introscope:/opt/middleware/introscope/current
 ...
kana-web-data:
 ...
 volumes:
   - kana-web-data:/data/web/release/current

You can name them whatever you want, but thats how you do it. This example is simple, you would want a unique name for the volume if you were going ot deploy multiple instances in an environment. I would also recommend adding an affinity rule to the licensing host that requires ${stack_name}/kana-introscope and ${stack_name}/kana-web-data that way the volumes will be scheduled and placed before the license.

Hope that helps.

Hi Bill,

I can create the containers and set the scheduling and that bit works. However the volumes that I mount are empty:

Here is a sample where I create a data container and added a file to it.

However when I mount the volume it is empty, and I am not sure what I may be doing wrong here.

## My sample docker file ##
[gmehta@spr71dev10 sample]$ more Dockerfile
FROM ubuntu
RUN mkdir /data
RUN echo "Hello World" > /data/1.log
VOLUME /data


## Docker runs ##
## Expose /data as a named volue ##
[gmehta@spr71dev10 sample]$ docker run -v datavol:/data ubuntu:data
[gmehta@spr71dev10 sample]$ docker volume ls
DRIVER              VOLUME NAME
local               4946d82674fb4e64a1bbe2133351380ffa76d438bfeb7a588c9bb24dd8cc215d
local               989ea043191d015a5bfd2e73266b5328c8d50f0b60529c1c4a56010d22870d22
local               9947f5b1b3ea4026401e32326fa26c1aa486d4463d04b6416c6953b2bbc4ae15
local               e704874c7c50e2059b3c28db6f265b1008f4a31f5f46c2576d669f8a2073a83e
local               bae01de9e6d51d572f9a669a6117060dac40b85a950ff868f720f9981df55f3a
local               9d95e0509aecf075f19420ff87d63e502714a08a3886c25fba10cbb6c1ea2132
local               98640c8dabe4674710fbf02b8b505e13f6a0dbdb136740494d0975c6d58f611c
local               1e6373ec2f7dde443277f6f238d20e091b319dda284c580d4ff3315fbff27c37
local               3c9eafc77c0de25243f31fde8a065d4bd7c43fe37d4087304381f86f5de4b35e
local               78d8278415d71fba358c63c3c91c41318f66da45dfb47fdc3e3c5a8e97052244
local               datavol
local               b25ebbde09bc651009e1a75ba9eb537177f4b6286d16763e3416f97bb5a90e32
local               fac9a53695a5986eca8e9164b8a7456302b8c72759f2987f734ff9700103612b
local               57fed5e13aedc655ebb6e34c37b7c74efb8f4de76a495b6c8553efa9631c7f65

## Mount this volume in another container now ##
[gmehta@spr71dev10 sample]$ docker run -it --rm -v datavol:/mnt ubuntu /bin/sh
# cd /mnt
# ls
# ls -l
total 0
#

Cheers,
Gaurav

We have eventually worked around the problem, but creating named volumes in the docker-compose.yml, and then using the Dockerfile CMD to initialize the new volume by copying the binaires to this mount.

We then just mount this named volume across all the dependent volumes.

Since we use global scheduling for this service, it means that the named volume is available across all the hosts in the environment and available for use by other containers.

Cheers,
Gaurav