Backup Strategy Rancher 2017

1

hi rancher community,

im working on a backup strategy for my rancher server and want to ask whats the current state and best practices today.
i would love to hear how you guys handle the backup and if there are already solutions out there to do that.
my setup is a rancheros with rancher on the top (currently for testing everything is on one server for testing).

my target is to be able to backup my server with all settings and data, reinstall and get quick up and running with the latest running state. i try to figure out whats worth to backup and what not, which looks currently like this:

rancheros:

  • /var/lib/rancher/conf/cloud-config.yml with my rancheros settings

rancher:

  • /var/lib/mysql
  • /home/rancher + /root home folders with settings and scripts …
  • /etc/apt/apt.conf.d/proxy where i setup the proxy settings to get apt working

something else missing here to consider?

rancher stacks:

  • rancher export backup all my stack settings

how to backup the data of the volumes / data containers?
i could loop through the stacks and export the stuff but is there maybe already a solution available or how do you guys backup your data?

rancher-catalog:

  • lives in a git repo so nothing to do here and data gets handled via stack backup strategy

main question is how to deal with the persistent volumes and settings for the services running.
is there already a solution to backup the stacks and volumes to get quick up and running again?

to get stuff out of rancher i read about convoy with nfs which looks nice.
is this stuff ready to use so far? are there some examples for a simple setup?

anyway it would be interesting how you guys handle the backup and if you can recommend some ways to think about, useful scripts or tools for backup and your experience with it …

thanks a lot

cheers maik

2 Likes
2

Hi Maik,

I’m testing Rancher and now I’m working on a backup system. It’s a proof of concept :slight_smile:.

I think that mount a nfs volumen or similar for backup has some problems. I begin to use a object storage for backup. Amazon S3 is a object storage. I use -https://minio.io-, it’s Amazon S3 compatilbe, there are Docker images, you can build a Minion cluster…

I created a golang script for send backups to Minio. I executed it daily from cron systems or similar:

Example:
backup_program bucket_name /path/to/file/or/directory

I can compile the golang script for different architectures (Linux, windows…), I can backup files from systems outside of Rancher. Simply I add a cron entry or periodic task (on windows) with my backup script.

Minio only store the last backup. So, we have a Rsnapshot -http://rsnapshot.org/- server outside Rancher. Rsnapshot take the changes of Minio storage, and save a copies of files during days, weeks, …

3

We run rancher server backed by an external MySQL DB (AWS RDS), and provision the rancher server with terraform.

If we even get the sense that the rancher manager isn’t acting right (which has never happened, just done fire drills), just “terraform taint” the server - which destroys the host, spins up a new host and runs a fresh rancher-server container in a matter of minutes with no lost data or lost “orchestration state”.

I’d definitely recommend keeping the data off of the server so the host running the rancher server can be disposable.

1 Like