Was curious if anyone here was running or building a large-scale minIO cluster on top of a rancher deployed Kube cluster.
I’m trying to build a large cluster using nodes with 40 ssd’s. Hitting issues with disks not getting mounted in the containers. Can’t find any errors in any logs. The pods start, the minIO cluster starts. I can see it’s directory structure under the mount point specified, it’s just not my disk but the root filesystem.
Thx,
BH
Please share the resource files (or Helm chart) you are using to deploy it, and the steps you ran on the hosts itself to prepare them, so we can see what you are trying to do and help you solve it.
My organization has built an on-site data-analytics platform using Rancher to manage the worker instances. We store Parquet files used for analysis and archive using Minio.
We started off running Minio inside Kubernetes, but we ran into the same kind of challenges you describe in terms of maintaining the veracity of persistent volumes—though we’ve gotten much better at wrangling those in our VSphere cloud provider since we began—when we were developing the platform it gave me pause about what I would have to do to ensure all my long-term storage was accessible in disaster or time-sensitive scenarios.
I decided ultimately NOT to use Kubernetes for this, but to run Minio as a VM in the same network segment as Rancher/k8s, but not managed by it. This has worked out very well. I would encourage you to consider that while k8s is an awesome, super powerful tool, it is not the right tool for every job. Sometimes simplicity in one area gives you the ability to manage complexity in another one.