Hi, I’m very new to Rancher and have been going through a lot of the help guides and stuff. Specifically with GlusterFS and Convoy, I have that up and working correctly. I have a 3 node setup with Rancher server on a different VM. Two nodes are bare metal RHEL7.1 running Docker 1.10.3, while the third node is a VM running Ubuntu 14.04.2 LTS and Docker 1.9.1. My Rancher server is a VM running Ubuntu 14.04.3 LTS and Docker 1.10.3.
Trying to deploy the Jenkins stack that’s modified to use my Convoy GlusterFS setup for persistent storage works if it lands on my Ubuntu node, but both RHEL7.1 nodes are failing.
Here’s my docker-compose.yml
jenkins-primary: image: "jenkins:1.642.3" ports: - "8080:8080" labels: io.rancher.sidekicks: jenkins-plugins,jenkins-datavolume io.rancher.container.hostname_override: container_name volumes_from: - jenkins-plugins - jenkins-datavolume entrypoint: /usr/share/jenkins/rancher/jenkins.sh jenkins-plugins: image: rancher/jenkins-plugins:v0.1.1 jenkins-datavolume: image: "busybox" volume_driver: convoy-gluster volumes: - jh2:/var/jenkins_home labels: io.rancher.container.start_once: true entrypoint: ["chown", "-R", "1000:1000", "/var/jenkins_home"]
Here’s the Rancher Server stacktrace in the logs:
2016-04-13 13:26:41,275 ERROR [5ff343e9-8ae9-46bf-8c19-c120bcc9f3f7:2485] [instance:78->instanceHostMap:78] [instance.start->(InstanceStart)->instancehostmap.activate] [] [erviceReplay-16] [c.p.e.p.i.DefaultProcessInstanceImpl] Unknown exception io.cattle.platform.eventing.exception.EventExecutionException: 500 Server Error: Internal Server Error ("get jh2: invalid character 'H' looking for beginning of value") at io.cattle.platform.eventing.exception.EventExecutionException.fromEvent(EventExecutionException.java:53) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.agent.impl.RemoteAgentImpl.callSync(RemoteAgentImpl.java:87) ~[cattle-iaas-agent-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.agent.impl.RemoteAgentImpl.callSync(RemoteAgentImpl.java:135) ~[cattle-iaas-agent-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.process.common.handler.AgentBasedProcessHandler.callSync(AgentBasedProcessHandler.java:180) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.process.common.handler.AgentBasedProcessHandler.handleEvent(AgentBasedProcessHandler.java:166) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.process.common.handler.AgentBasedProcessHandler.handle(AgentBasedProcessHandler.java:104) ~[cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runHandler(DefaultProcessInstanceImpl.java:446) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$4.execute(DefaultProcessInstanceImpl.java:393) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$4.execute(DefaultProcessInstanceImpl.java:387) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.idempotent.Idempotent.execute(Idempotent.java:42) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runHandlers(DefaultProcessInstanceImpl.java:387) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runLogic(DefaultProcessInstanceImpl.java:493) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runWithProcessLock(DefaultProcessInstanceImpl.java:320) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$2.doWithLockNoResult(DefaultProcessInstanceImpl.java:260) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.acquireLockAndRun(DefaultProcessInstanceImpl.java:257) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runDelegateLoop(DefaultProcessInstanceImpl.java:185) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.executeWithProcessInstanceLock(DefaultProcessInstanceImpl.java:158) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$1.doWithLock(DefaultProcessInstanceImpl.java:108) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$1.doWithLock(DefaultProcessInstanceImpl.java:105) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.execute(DefaultProcessInstanceImpl.java:105) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.object.process.impl.DefaultObjectProcessManager.executeStandardProcess(DefaultObjectProcessManager.java:29) [cattle-framework-object-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.process.common.handler.AbstractObjectProcessLogic.activate(AbstractObjectProcessLogic.java:31) [cattle-iaas-logic-common-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.process.instance.InstanceStart.compute(InstanceStart.java:229) [cattle-iaas-logic-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.process.instance.InstanceStart.handle(InstanceStart.java:92) [cattle-iaas-logic-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runHandler(DefaultProcessInstanceImpl.java:446) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$4.execute(DefaultProcessInstanceImpl.java:393) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$4.execute(DefaultProcessInstanceImpl.java:387) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.idempotent.Idempotent.execute(Idempotent.java:42) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runHandlers(DefaultProcessInstanceImpl.java:387) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runLogic(DefaultProcessInstanceImpl.java:493) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runWithProcessLock(DefaultProcessInstanceImpl.java:320) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$2.doWithLockNoResult(DefaultProcessInstanceImpl.java:260) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.acquireLockAndRun(DefaultProcessInstanceImpl.java:257) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runDelegateLoop(DefaultProcessInstanceImpl.java:185) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.executeWithProcessInstanceLock(DefaultProcessInstanceImpl.java:158) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$1.doWithLock(DefaultProcessInstanceImpl.java:108) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$1.doWithLock(DefaultProcessInstanceImpl.java:105) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.execute(DefaultProcessInstanceImpl.java:105) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.eventing.impl.ProcessEventListenerImpl.processExecute(ProcessEventListenerImpl.java:74) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at io.cattle.platform.engine.server.impl.ProcessInstanceParallelDispatcher$1.runInContext(ProcessInstanceParallelDispatcher.java:27) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:108) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na] at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_95] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_95] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_95] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_95] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_95]