Building a catalog item (that requires configuration files)

Rancher 1.x
1

As a personal exercise I am trying to build an extended catalog item for Rancher.

I’d like to build a catalog item for Harbor (VMware Enterprise registry). More here: https://github.com/vmware/harbor.

I have looked into some other existing Rancher catalog items and they all seem to assume that you can just pass environment variables to docker-compose / rancher-compose (that Rancher presumably gather during the configuration phase of the item).

Harbor is a little bit more complex. At the high level the setup works like this:

  • You grab a release (which is essentially the docker-compose file + a few other things).
  • you configure the (user-friendly) file harbor.cfg
  • you run the prepare script (which essentially disseminate all the info in harbor.cfg in different configuration files (in the ./common directory)
  • you run docker-compose up which includes proper mapping of various configuration files in the ./common directory.

For reference, this is the docker-compose file:

version: '2'
services:
  log:
    image: vmware/harbor-log:0.5.0-rc2
    container_name: harbor-log 
    restart: always
    volumes:
      - /var/log/harbor/:/var/log/docker/
    ports:
      - 1514:514
  registry:
    image: library/registry:2.5.0
    container_name: registry
    restart: always
    volumes:
      - /data/registry:/storage
      - ./common/config/registry/:/etc/registry/
    environment:
      - GODEBUG=netdns=cgo
    command:
      ["serve", "/etc/registry/config.yml"]
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "registry"
  mysql:
    image: vmware/harbor-db:0.5.0-rc2
    container_name: harbor-db
    restart: always
    volumes:
      - /data/database:/var/lib/mysql
    env_file:
      - ./common/config/db/env
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "mysql"
  ui:
    image: vmware/harbor-ui:0.5.0-rc2
    container_name: harbor-ui
    env_file:
      - ./common/config/ui/env
    restart: always
    volumes:
      - ./common/config/ui/app.conf:/etc/ui/app.conf
      - ./common/config/ui/private_key.pem:/etc/ui/private_key.pem
      - /data:/harbor_storage
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "ui"
  jobservice:
    image: vmware/harbor-jobservice:0.5.0-rc2
    container_name: harbor-jobservice
    env_file:
      - ./common/config/jobservice/env
    restart: always
    volumes:
      - /data/job_logs:/var/log/jobs
      - ./common/config/jobservice/app.conf:/etc/jobservice/app.conf
    depends_on:
      - ui
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "jobservice"
  proxy:
    image: nginx:1.11.5
    container_name: nginx
    restart: always
    volumes:
      - ./common/config/nginx:/etc/nginx
    ports:
      - 80:80
      - 443:443
    depends_on:
      - mysql
      - registry
      - ui
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "proxy"

Is it even possible to create a rancher catalog item for a service like this? If so, how would one go about it? Does it involve creating Rancher volumes (pre-populated with such information) to satisfy the mappings in the compose project?
How would one populate such files at catalog item configuration time?

Thanks.

2

Perhaps you could use something like confd for this?

I noticed that some of the catalog entries (e.g. Elastic Search) use custom confd images to create configuration files, rancher/elasticsearch-conf:v0.4.0 for example. See here: https://github.com/rancher/catalog-dockerfiles/tree/master/elasticsearch/containers/0.4.0/elasticsearch-conf

This may also be a useful read: https://theagileadmin.com/2015/11/12/templating-config-files-in-docker-containers/

This is not something I have much experience with yet, however.

3

Mh… thanks @Daniel_Skinner you may be onto something here.

I am still reading it but I am wondering if I am having my thoughts right here:

  • I should embed all the configuration files inside the container image itself (no need to map external volumes etc).
  • these configuration files should have meaningful default values for the most part.
  • I should use confd to make these files templates of sort that accept inputs from outside (e.g. environment variables)
  • I should tweak the docker-compose file to include variables to pass onto the containers when I up it
  • I can grab these variables via the Rancher catalog framework facility (i.e. when you configure the catalog entry).

Would this be the idea (at the high level)?

4

Yea that sounds like the approach I’d want to take. As soon as you need to put some files in a volume and mount it to get your stacks working then everything becomes messy. Avoid it as long as you possibly can.

How did you get on with it?

5

Hi @Daniel_Skinner. I haven’t updated the thread because I was waiting to get something “done” (which I am in the process of having). After talking to Rancher (the great @rawmind!) we have concluded that the faster approach was to basically dockerize the install procedure.

So instead of re-working the entire setup process (which I may agree would make more sense and it would be more polished) we decided to create a sort of “support” container that takes as an input ALL the variables I need and that THEN run a script that customizes the harbor.cfg file accordingly and THEN prepares all the configuration files, environment files etc that exports as VOLUMES to all the other containers.

I am basically preserving the as-is installer … but building a docker wrapper around it to which I can pass variables (which is what I needed).

So the end result is a compose-file based on the original one that has some tweaks in it + the additional support container.

I have been working on it on and off but I am close to have something working (only thanks to Raul) :slight_smile:

As soon as I am done with it I want to do a blog post and link it here!

Thanks!

6

Ok… not a small thing as I anticipated but I have it done now.

If you are interested: http://www.it20.info/2017/01/vmware-harbor-as-a-rancher-catalog-entry/

/cc @Daniel_Skinner

7

Great. I’ll definitely add that to my reading list.