Broken/unstable host/stack config

Rancher 1.x
1

I’ve made a mistake by bypassing Rancher and removed a container manually and now I always see this error :frowning:

Failed to allocate instance [container:1i4730]: Bad instance [container:1i4730] in state [error]:
Allocation failed: volume [6369] must have exactly these pool(s): [25], valid host(s) [28]

Here are the steps (using Rancher 1.6.12):

  1. I had a stack deployed on a host with volume mapping local dir to container dir:
services:
  slave:
    volumes:
      - /var/jenkins:/home/jenkins
  1. I created a new version of this stack replacing this local volume by a Docker volume
services:
  slave:
    volumes:
      - jenkins-slave:/home/jenkins
volumes:
  jenkins-slave:
    driver: local
  1. As I had an error on stack upgrade, I logged in on the host and manually removed (!!!) stack container :frowning:
  2. Since then, I can’t use this host within my environment… Host is well added but my stack can’t be deployed to it…
    I’ve tried to reinstall a fresh OS (but kept IP) but with same results…
Failed to allocate instance [container:1i4730]: Bad instance [container:1i4730] in state [error]:
Allocation failed: volume [6369] must have exactly these pool(s): [25], valid host(s) [28]

Is there a way to fix/reset Rancher database for a host?

UPDATE 1:
It’s even worse than I thought: adding new fresh host gives me the same error!
It seems that it’s the stack/container/volume config that is broken :scream:
Any idea of what should I do to fix my mistakes?

UPDATE 2:
Same error with a fresh new environment & clean hosts :frowning:

  • I’ve evacuated & removed my hosts from broken environment
  • I’ve deleted this environment
  • I’ve removed all containers & volumes from my hosts (just to be sure there is no trace of previous stuff)
  • I’ve created a fresh new environment
  • I’ve added these clean hosts to my new env
    –> Jenkins master & one slave is OK, but Rancher is not able to deploy slave to other host :frowning:

I really don’t understand as I started from a clean env with clean hosts…

Failed to allocate instance [container:1i4992]: Bad instance [container:1i4992] in state [error]: 
Allocation failed: valid host(s) [48], volume [7098] must have exactly these pool(s): [45]