Issue adding an Amazon EBS Volume


Hi. I have been experimenting with 2.0 beta 2 and I am having issues configuring an Amazon EBS Volume in my Amazon EC2 cluster.

When I try to add a Persistent Volume based on Amazon EBS Disk and get the following error:

Persistentvolumes “pv1” is forbidden: error querying AWS EBS volume vol-03a566dd4648dd3eb: error finding instance i-05043dc178782dbc3: “error listing AWS instances: “NoCredentialProviders: no valid providers in chain. Deprecated. \n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors””

Any idea what I am doing wrong? Is this type of volume supported in Beta 2?


Using EBS volumes and ELB balancers requires that the cloud provider be configured as AWS for the cluster, and that the nodes of the cluster have IAM instance profiles that allow access to the APIs to manipulate them. The UI does not configure the cloud provider until next release, and you likely don’t have instance profiles setup.


Any chance that I can configure this via API or similar?


It’s under Cluster Options in beta3 (today); IAM instance profile is in the node template (which you’ll have to recreate because youc an’t currently edit).


I created a brand new Rancher 2.0 Beta 3 cluster using Amazon EC2. The resulting cluster did not contain any Storage Volume Class definition. I think the cluster config script is not setting things up properly. Using the GKE or Azure Kubernetes Service work just fine. Storage class are created and Persistant Volumes are properly created.

Hope this help pinpoint the issue with the Amazon EC2 cluster build.


GKE and AKS are hosted kubernetes clusters. Once created they show whatever storage classes Google/Azure creates for you by default. EC2 is bare machines, there is nothing to decide what storage classes to create.

For them to work if you create your own, you need to create the cluster with the Amazon cloud provider chosen. Then the nodes need to have an IAM instance profile set that allows the related operations (creating EBS volumes, mounting them to instances, etc).


Is there a howto on doing this? I have tried but can’t get it to work. Are there public AMIs already configured for this?


any update on this. looking to add ebs volumes and running into same problems.


I am having the exact same problem. Any idea on how to fix this?


I’m having a similar issue. I have the appropriate IAM policies set for my Rancher Nodes, basically anything ec2:…Volume but persistent volumes are getting created.

Are they volumes that are created first in AWS and then connected to? Seems like Rancher is supposed to be provisioning them.

I have AWS set as the cloud provider and the creation of the clusters works just fine.

Is there a tutorial on this out there somewhere?


A persistent volume (PV) refers to a piece of storage that already exists. So creating one on its own does not make anything in AWS, the volume id has to refer to an existing volume in EBS.

To dynamically provision storage, you need a cluster configured with a cloud-provider that supports it (the AWS one does). Then you define one or more Storage Classes (some providers preconfigure one, but AWS doesn’t), and create a persistent volume claim (PVC) that requests X GB using Y storage class. The provisioner then creates the underlying storage resource and makes a PV pointing to it and binds the PVC to that PV.


Ah I think the volume id referring to the volume id in AWS might be what I was missing. Will test that out and let you know. Thanks!


Thanks @vincent that did the trick!