Getting service.upgrade.wait.exception on webhook-triggered upgrade

Rancher 1.x
1

(Rancher 1.6.0 on Digital Ocean)

I can upgrade manually, but when an upgrade is triggered by my dockerhub webhook, the new instance fails with this message:

Failed to allocate instance [container:1i301]: Bad instance [container:1i301] in state [error]: Allocation failed: needs host with label hosttype: web

I have verified that I have the correct label in the scheduler and on the host.

What am I missing?

2

@jmbldwn Are you using start before stopping? If your service has ports exposed and if you want to upgrade them on the same host, you will have the ports conflicts, although we don’t show the correct scheduling error.(this is an issue and will be fixed in 1.6.1 https://github.com/rancher/rancher/issues/8636)

3

Indeed that’s the issue.

I suppose in production I’ll just upgrade one container at a time and won’t need start-before-stop, but it does occur to me that there must be a way to fire up the new instance and swap port configurations once it’s running.

Thanks for clearing this up.