Options for dynamic created persistant rancher-ebs volumes when scaling

Rancher 1.x
1

We are trying to use rancher-ebs driver with persistent ebs volumes and as long as I run at scale:1 all is good. We want to run at scale: 3. Also, if I eliminate the persistent ebs storage then all is good. What we are looking for is some way of creating storage volumes with dynamic names based on the scale id:
volumes:
mongo-prod:
…
Which could create with a 1 for scale 1 and 2 for scale 2.

“docker-compose.yml”
version: ‘2’
volumes:
mongo-prod:
driver: rancher-ebs
driver_opts:
name: mongo-prod
size: 1024
volumeType: gp2

services:
mongo-datavolume:
image: busybox
entrypoint:
- /bin/true
network_mode: none
volumes:
- mongo-prod:/data/db
labels:
io.rancher.scheduler.affinity:host_label: ‘’
io.rancher.container.start_once: ‘true’
io.rancher.container.hostname_override: container_name
mongo-cluster:
image: mongo:3.2
environment:
CATTLE_SCRIPT_DEBUG: ‘’
MONGO_SERVICE_NAME: mongo-cluster
entrypoint:
- /opt/rancher/bin/entrypoint.sh
volumes_from:
- mongo-datavolume
- mongo-base
command:
- --replSet
- or1-prod
labels:
io.rancher.scheduler.affinity:host_label: ‘’
io.rancher.sidekicks: mongo-base,mongo-datavolume
io.rancher.container.hostname_override: container_name
mongo-base:
image: rancher/mongodb-conf:v0.1.1
stdin_open: true
entrypoint:
- /bin/true
network_mode: none
labels:
io.rancher.scheduler.affinity:host_label: ‘’
io.rancher.container.start_once: ‘true’
io.rancher.container.hostname_override: container_name
or1-prod-mongo:
image: rancher/lb-service-haproxy:v0.6.4
ports:
- 27017:27017/tcp
labels:
io.rancher.container.agent.role: environmentAdmin
io.rancher.container.create_agent: ‘true’

“rancher-compose.yml”
version: ‘2’
services:
mongo-datavolume:
metadata:
mongo: &id001
yml:
replset.name: or1-prod
retain_ip: true
scale: 3
start_on_create: true
mongo-cluster:
metadata:
mongo: *id001
retain_ip: true
scale: 3
start_on_create: true
mongo-base:
metadata:
mongo: *id001
retain_ip: true
scale: 3
start_on_create: true
or1-prod-mongo:
scale: 2
start_on_create: true
lb_config:
certs:
port_rules:
- priority: 1
protocol: tcp
service: mongo-cluster
source_port: 27017
target_port: 27017
health_check:
response_timeout: 2000
healthy_threshold: 2
port: 42
unhealthy_threshold: 3
initializing_timeout: 60000
interval: 2000
reinitializing_timeout: 60000

Which dynamically creates at scale 3:

ebs_volume_problem2.png2704×404 59.6 KB

2

I don’t think we will implement this directly using the scale number, but we are working on a feature that should accomplish what you need. Is your use case simply that you want to have a volume per instance of the service?

3

That is correct. We really don’t care how it is accomplished, but just that we can have persistent storage using block share like ebs that allow us to use the scale feature still. We have several situations like a mongo database cluster that require unique persistent storage that can be uniquely consumed by individual containers and created if needed. NFS doesn’t really satisfy what we are trying to do and having to pre-mount and attach storage to all the host machines means paying for volumes we may not use. If you are working on something … which is exciting how can we find out details or possible beta options

4

We are also interested in something like this. Ideally we could scale out using the “scale” feature. This brings associated problems though, in order to get EBS working properly we set affinity labels for AZ’s. In the current form this means our service would be entirely within one AZ and would scale within that AZ.

In that case we’d probably still end up with three service definitions each restricted to an AZ and with an initial scale of 1. But with the option to scale each of them and have a second EBS volume provisioned