Rancher with external DB(mariaDB) not starting up

gowrirajeev · September 16, 2016, 7:12pm

Hello,

I was trying to set up rancher with external DB but I am getting the following error and the rancher container is not starting up:

   19:09:50.327 [main] WARN  o.e.j.u.component.AbstractLifeCycle - FAILED o.e.j.w.WebAppContext@38033a21{/,file:/usr/share/cattle/2b0025124d75a78e32e6a4abac36d2cd/,UNAVAILABLE}{file:/usr/share/cattle/2b0025124d75a78e32e6a4abac36d2cd/}: org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database.
org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database.
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:453) ~[jooq-3.3.0.jar:na]
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:430) ~[jooq-3.3.0.jar:na]
	at io.cattle.platform.object.impl.JooqObjectManager.reload(JooqObjectManager.java:90) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:67) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [2b0025124d75a78e32e6a4abac36d2cd/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_101]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_101]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_101]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_101]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [2b0025124d75a78e32e6a4abac36d2cd/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [2b0025124d75a78e32e6a4abac36d2cd/:na]
org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database.
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:453)
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:430)
	at io.cattle.platform.object.impl.JooqObjectManager.reload(JooqObjectManager.java:90)
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:67)
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97)
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114)
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135)
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117)
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114)
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40)
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33)
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13)
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37)
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114)
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15)
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100)
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56)
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11)
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15)
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:387)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:354)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at io.cattle.platform.launcher.Main.run(Main.java:188)
	at io.cattle.platform.launcher.Main.main(Main.java:251)
19:09:50.352 [main] WARN  o.e.j.u.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@11e96e0: org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database.
org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database.
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:453) ~[jooq-3.3.0.jar:na]
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:430) ~[jooq-3.3.0.jar:na]
	at io.cattle.platform.object.impl.JooqObjectManager.reload(JooqObjectManager.java:90) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:67) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [2b0025124d75a78e32e6a4abac36d2cd/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_101]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_101]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_101]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_101]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [2b0025124d75a78e32e6a4abac36d2cd/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [2b0025124d75a78e32e6a4abac36d2cd/:na]
19:09:50.359 [main] ERROR ConsoleStatus - Startup Failed [86458ms]
org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database.
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:453) ~[jooq-3.3.0.jar:na]
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:430) ~[jooq-3.3.0.jar:na]
	at io.cattle.platform.object.impl.JooqObjectManager.reload(JooqObjectManager.java:90) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:67) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.runHookInContexts(DefaultModuleDefinitionSet.java:131) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:123) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:100) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.start(Server.java:387) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.server.Server.doStart(Server.java:354) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
	at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) ~[2b0025124d75a78e32e6a4abac36d2cd/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_101]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_101]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_101]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_101]
	at io.cattle.platform.launcher.Main.run(Main.java:188) [2b0025124d75a78e32e6a4abac36d2cd/:na]
	at io.cattle.platform.launcher.Main.main(Main.java:251) [2b0025124d75a78e32e6a4abac36d2cd/:na]
STARTUP FAILED [86462] ms

DB used: Server version: 10.1.12-MariaDB MariaDB Server
Rancher version: latest
I also tried to export the DB of my old rancher instance and dumped it into this DB, in this scenario, it starts up but still throws many exceptions and some features dont work.

CONTAINER STARTUP COMMAND:

sudo docker run -d -p 8080:8080 -e CATTLE_DB_CATTLE_MYSQL_HOST=<host-ip> -e CATTLE_DB_CATTLE_MYSQL_PORT=3306 -e CATTLE_DB_CATTLE_MYSQL_NAME=rancher -e CATTLE_DB_CATTLE_USERNAME=<user> -e CATTLE_DB_CATTLE_PASSWORD=<password> -v /var/run/docker.sock:/var/run/docker.sock rancher/server

Database creation:
`CREATE DATABASE IF NOT EXISTS rancher;

I created a database called rancher before running the container and I see that it connects to the DB and also populates it with 83 rows.

Could someone please help me with this?
Thanks!!

aemneina · September 19, 2016, 9:48pm

hey @gowrirajeev are you able to log into the db with the credentials youre providing to rancher? Can that user create tables? That might be a good first step to troubleshooting this.

gowrirajeev · September 19, 2016, 9:57pm

Hey @aemneina,

I am able to login with the provided credentials.
So when I spin up rancher server container, I see that it connects to the db and also populates it with 83 rows.

When I tailed the container logs during start up, I found the following warning:

2016-09-19 21:55:26,110 WARN [main] [liquibase] modifyDataType will lose primary key/autoincrement/not null settings for mysql. Use <sql> and re-specify all configuration if this is the case

After which I see this:

2016-09-19 21:55:26,110 WARN [main] [liquibase] modifyDataType will lose primary key/autoincrement/not null settings for mysql. Use <sql> and re-specify all configuration if this is the case 2016-09-19 21:56:29,451 INFO [main] [ConsoleStatus] [3/32] [67886ms] [67741ms] Loading system 2016-09-19 21:56:29,494 INFO [main] [ConsoleStatus] [4/32] [67929ms] [43ms] Loading defaults 2016-09-19 21:56:29,656 INFO [main] [ConsoleStatus] [5/32] [68091ms] [162ms] Loading types 2016-09-19 21:56:33,914 INFO [main] [ConsoleStatus] [6/32] [72349ms] [4258ms] Loading system-services 2016-09-19 21:56:34,118 INFO [main] [ConsoleStatus] [7/32] [72553ms] [204ms] Loading agent-server 2016-09-19 21:56:34,304 INFO [main] [ConsoleStatus] [8/32] [72739ms] [186ms] Loading allocator-server 2016-09-19 21:56:35,547 INFO [main] [ConsoleStatus] [9/32] [73982ms] [1243ms] Loading api-server 2016-09-19 21:56:37,529 INFO [main] [ConsoleStatus] [10/32] [75964ms] [1981ms] Loading iaas-api 2016-09-19 21:56:37,581 INFO [main] [ConsoleStatus] [11/32] [76016ms] [52ms] Loading archaius 2016-09-19 21:56:37,825 INFO [main] [ConsoleStatus] [12/32] [76260ms] [244ms] Loading core-model 2016-09-19 21:56:37,870 INFO [main] [ConsoleStatus] [13/32] [76305ms] [45ms] Loading core-object-defaults 2016-09-19 21:56:37,910 INFO [main] [ConsoleStatus] [14/32] [76345ms] [40ms] Loading encryption 2016-09-19 21:56:40,762 INFO [main] [ConsoleStatus] [15/32] [79197ms] [2852ms] Loading process 2016-09-19 21:56:40,809 INFO [main] [ConsoleStatus] [16/32] [79244ms] [47ms] Loading redis 2016-09-19 21:56:40,944 INFO [main] [ConsoleStatus] [17/32] [79379ms] [135ms] Starting bootstrap 2016-09-19 21:56:40,944 INFO [main] [ConsoleStatus] [18/32] [79379ms] [0ms] Starting config-defaults 2016-09-19 21:56:40,944 INFO [main] [ConsoleStatus] [19/32] [79379ms] [0ms] Starting system 2016-09-19 21:56:40,944 INFO [main] [ConsoleStatus] [20/32] [79379ms] [0ms] Starting defaults 2016-09-19 21:56:40,944 INFO [main] [ConsoleStatus] [21/32] [79379ms] [0ms] Starting types 21:56:46.445 [main] WARN o.eclipse.jetty.webapp.WebAppContext - Failed startup of context o.e.j.w.WebAppContext@25295985{/,file:/usr/share/cattle/2b0025124d75a78e32e6a4abac36d2cd/,STARTING}{file:/usr/share/cattle/2b0025124d75a78e32e6a4abac36d2cd/} org.jooq.exception.InvalidResultException: Exactly one row expected for refresh. Record does not exist in database. at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:453) ~[jooq-3.3.0.jar:na] at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:430) ~[jooq-3.3.0.jar:na] at io.cattle.platform.object.impl.JooqObjectManager.reload(JooqObjectManager.java:90) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:67) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitFor(ResourceMonitorImpl.java:97) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.object.monitor.impl.ResourceMonitorImpl.waitForState(ResourceMonitorImpl.java:114) ~[cattle-framework-resource-monitor-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredentialLock(GenericServiceLauncher.java:135) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:117) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.service.launcher.GenericServiceLauncher$2.doWithLock(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.service.launcher.GenericServiceLauncher.getCredential(GenericServiceLauncher.java:114) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.service.launcher.ServiceAccountCreateStartup.start(ServiceAccountCreateStartup.java:15) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.extension.spring.BackgroundStart.run(BackgroundStart.java:63) ~[cattle-framework-extension-spring-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:139) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na].... ..... ...

aemneina · September 19, 2016, 10:16pm

what version docker are you using? how about the host OS and version there?

gowrirajeev · September 19, 2016, 10:18pm

Docker version: Docker version 1.12.1, build 23cf638
OS: CentOS Linux release 7.2.1511 (Core)
I am using the following DB: 10.1.12-MariaDB MariaDB Server (A redundant database cluster of 3 nodes)

Topic		Replies	Views
Rancher database exception after restart Rancher 1.x	0	848	January 3, 2017
Rancher Server Failing to Start RancherOS	2	2616	December 18, 2015
Restart Loop - Table cattle.instance doesn't exist Rancher 1.x	3	1234	February 23, 2016
Running Rancher with external MySQL Database	4	5808	June 13, 2016
Can't get Rancher back up Rancher 1.x	3	2023	September 14, 2017

Rancher with external DB(mariaDB) not starting up

CONTAINER STARTUP COMMAND:

Related topics