Why not support a docker-compose file with a configured mysql image to run rancher server?

Hi,
there are two options to run rancher server actually:
either connect it to an external database, or use the one shipped with the server image.
I understand the facility to use the rancher server with a pre-configured database.
But, in my opinion, it has 2 drawbacks :

  • it acts like a black box (some mysql parameters are tuned by rancher scripts, but it is not public nor easily tunable)
  • it is against one popular docker usage of using one process per container

So, i have seen a docker-compose test file from rancher github repository which fulfills this need.
why not supporting officially a docker-compose file to run the rancher server?

best regards,

Charles.

maybe because Rancher is a docker-compose competitor :smile:?

Hi,
i don’t see docker-compose as a competitor.
rancher stands on docker-compose, and extends it with rancher-compose to provide the multi-hosts support, storage management, and metadata service and so on, and a good ui.
One docker product competitor would be Docker Universal Control Plane.
best regards,

Charles.

I don’t think Rancher runs docker-compose direclty … as far as I know they interpret the same syntax, but are not running on top of docker-compose, they have implemented their own “docker-compose” so called “rancher-compose” which understand docker-compose syntax … but not all of it … and yes Docker Universal Control Plane is yet another competitor to Rancher… however I like Rancher because it is open source.

regards

  • We package as one container to make it as easy as possible to try. Installing and using docker-compose is 200% more steps to explain than docker run rancher/server.

  • The built-in database is meant for just that. If you’re at the point where you’re customizing mysql config, yes you should probably be running a separate container. But us providing a template that does nothing but deploy mysql:latest is of minimal value.

  • Most serious installations end up with a multi-master database on multiple hosts, or 3rd-party hosted database (e.g. Amazon RDS). docker-compose helps with neither.

I don’t think Rancher runs docker-compose direclty

docker-compose (originally Fig) is Python, rancher-compose uses libcompose in Go, which we contributed to Docker.

1 Like

@vincent thanks for the clarifications.

What was the problem with docker-compose that made people create libcompose? why do we have to re-implement same functionality? Is it because it was python and not go?
now that we have the version 2 syntax of compose file, libcompose has to be re-implement it

I wonder why doesn’t Rancher just use directly docker-compose? Or are there plans to do that in the future? we would have all those new features quicker in Rancher instead of waiting for libcompose. Sorry for the many questions, it helps me understand where Rancher is heading and how much it will differentiate itself.

1 Like

Most of what both Docker and we (outside of Cattle) do are Go, and we didn’t want to maintain rancher-compose in Python.

libcompose is a library which exposes functionality that can be imported and used in other code (e.g. rancher-compose). docker-compose is an application. Extending it would have ended up being a fork that is worse to maintain. So we (@ibuildthecloud) just wrote libcompose instead.

There was also interest in replacing docker-compose with a Go version that uses libcompose, which is why we contributed it to Docker. That hasn’t happened yet, but I assume they use it for something since vdemeester continues to maintain it.

Hi @vincent,
i’m agree with you about the minimal value.
But provide a mysql container with the mysql parameters rancher use(max_connections, key_buffer_size,sync_bin_log, …), with a working master-master replication would have a high value in my opinion.
At least, publishing the mysql parameters tweaks or recommendations would be great.
I’ve discovered them from the mysql script rancher use.
I’ve seen an experimental galera master-master replication in the catalog, but i doubt it can be used for the mysql server database (chicken and eggs problem?).

best regards,

Charles.

There is nothing in that script that is relevant to running your own server. All of this is modifying things Ubuntu overrides in their config by default, which you wouldn’t have starting from the mysql image, or making it easier to migrate to a standalone database.

max_connections has a suggested minimum value in the documentation which matches the current MySQL default (but Ubuntu changes it).

The value of key_buffer_size isn’t changed at all, just the variable is renamed from key_buffer because Ubuntu’s default my.cnf uses the deprecated name. Similarly they uncomment the bind-address line which makes it listen only on localhost… unhelpful for replicating.

expire_log_days and the whole if block below it are for setting it up as a master so that you can connect a slave and replicate out to move to an external database setup. This is not relevant if you’re running your own container, or you’ll have your own replication setup.

And innodb_file_per_table makes it easier to delete a table and reclaim disk space, which is just convenient for us.

thanks @vincent for this explanation on mysql parameters.

Do you confirm this is odd to try to use the galera stack from rancher catalog, to replicate the mysql database used by the server?

best regards,

Charles.

Yes I would not suggest having Rancher contain its own universe, though it would probably work until something goes wrong and you could probably fix it with manual intervention when it does…

For example, service containers created by Rancher do not actually have the --restart=always flag passed to Docker because Rancher manages restarts. So if the container(s) died and it can no longer make queries it wouldn’t be able to restart the containers.

If you really wanted to do that, a better way would probably be to have a “parent” rancher that is not HA itself but runs the containers for Galera and a separate “child” Rancher installation. This would require dedicating a couple host/VMs to it though.

Thanks @vincent for the explanation and the pics!
:slightly_smiling:
i’m not this kind of frankenstein to test rancher in rancher (although Docker In Docker DIND exists :alien:).
Charles.

Yes we use DinD heavily for testing, and Rancher-in-Rancher occasionally too…

1 Like