[Rancher HA] - Cleaning hosts servers for fresh reinstall

slinty · July 19, 2016, 8:13pm

Dear all,

I’ve failed in installing Rancher HA (3 nodes, external DB and external Nginx loadbalancer) I’ve done it again, and it seems that something (and I cannot precisely point what) is preventing my reinstallation to work properly.
I would like to reinstallon clean hosts, without having to reinstall servers OS.
Is there’s a way of being sure to remove previous Rancher HA installation and start over on clean servers ?

Hosts : centos 7
Rancher/server v1.1.1 (previous install was done with 1.1.0-dev5)

Thanks for your help.
Slinty

johnrengelman · July 21, 2016, 1:50am

Did you run this command (from: http://docs.rancher.com/rancher/latest/en/upgrading/multi-nodes/)
sudo docker rm -f $(sudo docker ps -a | grep rancher | awk {'print $1'})
to remove the existing containers?

slinty · July 21, 2016, 7:21am

Hi,
First of all, thanks for your reply.
Yes, this command removes rancher containers and it works more or less. I was thinking to something more radical, such as empty /var/lib/rancher/ directory and/or wiping databases. In order to be 105% sure to do a fresh start.

slinty · July 29, 2016, 2:24pm

Hi all !

Just to give more detail about my problem.

In the cattle logs we have this error on all ranche nodes

2016-07-29 14:15:31,184 INFO    [main] [ConsoleStatus] [6/32] [7008ms] [2788ms] Loading system-services 
2016-07-29 14:15:31,314 INFO    [main] [ConsoleStatus] [7/32] [7138ms] [130ms] Loading agent-server 
2016-07-29 14:15:31,442 INFO    [main] [ConsoleStatus] [8/32] [7266ms] [128ms] Loading allocator-server 
2016-07-29 14:15:32,193 INFO    [main] [ConsoleStatus] [9/32] [8017ms] [751ms] Loading api-server 
2016-07-29 14:15:33,440 INFO    [main] [ConsoleStatus] [10/32] [9264ms] [1247ms] Loading iaas-api 
2016-07-29 14:15:33,477 INFO    [main] [ConsoleStatus] [11/32] [9301ms] [37ms] Loading archaius 
2016-07-29 14:15:33,597 INFO    [main] [ConsoleStatus] [12/32] [9421ms] [120ms] Loading core-model 
2016-07-29 14:15:33,622 INFO    [main] [ConsoleStatus] [13/32] [9446ms] [25ms] Loading core-object-defaults 
2016-07-29 14:15:33,647 INFO    [main] [ConsoleStatus] [14/32] [9471ms] [24ms] Loading encryption 
2016-07-29 14:15:35,206 INFO    [main] [ConsoleStatus] [15/32] [11030ms] [1559ms] Loading process 
2016-07-29 14:15:35,242 INFO    [main] [ConsoleStatus] [16/32] [11066ms] [36ms] Loading redis 
2016-07-29 14:15:35,287 INFO    [main] [ConsoleStatus] [17/32] [11111ms] [45ms] Starting bootstrap 
2016-07-29 14:15:35,287 INFO    [main] [ConsoleStatus] [18/32] [11111ms] [0ms] Starting config-defaults 
2016-07-29 14:15:35,287 INFO    [main] [ConsoleStatus] [19/32] [11111ms] [0ms] Starting system 
2016-07-29 14:15:35,287 INFO    [main] [ConsoleStatus] [20/32] [11111ms] [0ms] Starting defaults 
2016-07-29 14:15:35,287 INFO    [main] [ConsoleStatus] [21/32] [11111ms] [0ms] Starting types 
14:15:51.817 [main] WARN  o.eclipse.jetty.webapp.WebAppContext - Failed startup of context o.e.j.w.WebAppContext@2e5fc5fc{/,file:/usr/share/cattle/8cd6a40e20d8c693d8b4ac1f1d27d4b2/,STARTING}{file:/usr/share/cattle/8cd6a40e20d8c693d8b4ac1f1d27d4b2/}
io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:92) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_95]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_95]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_95]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_95]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
14:15:51.818 [main] WARN  o.e.j.u.component.AbstractLifeCycle - FAILED o.e.j.w.WebAppContext@2e5fc5fc{/,file:/usr/share/cattle/8cd6a40e20d8c693d8b4ac1f1d27d4b2/,UNAVAILABLE}{file:/usr/share/cattle/8cd6a40e20d8c693d8b4ac1f1d27d4b2/}: io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:92) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_95]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_95]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_95]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_95]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
14:15:51.827 [main] WARN  o.e.j.u.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@191ac0df: io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:92) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_95]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_95]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_95]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_95]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:92)
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97)
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114)
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135)
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117)
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114)
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40)
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33)
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13)
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37)
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114)
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15)
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100)
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56)
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11)
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15)
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:387)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:354)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at io.cattle.platform.launcher.Main.run(Main.java:188)
	at io.cattle.platform.launcher.Main.main(Main.java:251)
14:15:51.832 [main] ERROR ConsoleStatus - Startup Failed [30346ms]
io.cattle.platform.async.utils.TimeoutException: Object [account:20] failed to satisfy predicate [15000] millis
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:92) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
STARTUP FAILED [30348] ms
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) ~[8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_95]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_95]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_95]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_95]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [8cd6a40e20d8c693d8b4ac1f1d27d4b2/:na]

aemneina · August 25, 2016, 9:58pm

To clean up and try redeploying it doesnt hurt to drop and recreate the database. /var/lib/rancher/state on each host should be removed as well.

What does your nginx config file look like?

Topic		Replies	Views
Rancher HA Stack "Degraded" Rancher 1.x	27	5843	May 2, 2016
New rancher-server for old nodes Rancher 1.x	4	1331	July 27, 2017
Thoughts / Questions after bringing Rancher to Production on AWS Rancher 1.x	2	1170	September 27, 2016
Migrated rancher 1.6 disconnted from digitalocean host Rancher 1.x	2	1012	January 11, 2019
Register a currently running host to a new Rancher Server Rancher 1.x	0	683	September 12, 2018

[Rancher HA] - Cleaning hosts servers for fresh reinstall

Related Topics