Can custom cluster etcd snapshots be backed up to S3?

1

Greetings,
I have a Rancher custom cluster and I’d like to set the ‘etcd Snapshot Backup Target’ to backup to S3. I believe I’ve setup proper permissions in IAM and configured Rancher accordingly but I’m not seeing any backups showing up in the bucket. I’ve enabled logging for the bucket and I don’t see Rancher even attempting to put anything there, so I’m assuming I’m hitting a problem in Rancher. Per the documentation here:

https://rancher.com/docs/rancher/v2.x/en/cluster-admin/backing-up-etcd/

When referencing requirements for S3, it sounds like the assumption is that the cluster is running on EC2 instances. Is it possible to backup etcd snapshots from a custom on-prem cluster to S3?

Are there any logs Rancher side I can look at to gain some insight on why this isn’t working?

2

Can you share the sanitized config for s3 backups? The error is currently not surfaced in the UI (which is fixed in https://github.com/rancher/rancher/issues/20685), but there should be an error in the Rancher log regarding EtcdBackupController.

3

Thanks @superseb. Once I saw this in the logs:

level=fatal msg="failed to set s3 server: bucket ******-etcd-backups is not found"

It was quickly apparent I’d made a typo mistake: backup != backups.

Also good to know there’s a fix in master for the error to get up to the UI.