Why do I get Failed to get ping from agent error?

Duddu_Praveen · July 10, 2015, 11:03am

I am trying to start RancherServer and RancherAgents on CoreOS machines. Unfortunately I get the following errors in my Rancher Server logs

2015-07-10 10:59:39,763 ERROR [:] [] [] [] [cutorService-44] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [143]
2015-07-10 10:59:39,764 ERROR [:] [] [] [] [cutorService-44] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 10:59:44,759 ERROR [:] [] [] [] [cutorService-54] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [144]
2015-07-10 10:59:44,760 ERROR [:] [] [] [] [cutorService-54] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 10:59:44,975 ERROR [:] [] [] [] [cutorService-57] [o.a.c.m.context.NoExceptionRunnable ] Uncaught exception org.jooq.exception.DataChangedException: Database record has been changed
        at org.jooq.impl.UpdatableRecordImpl.checkIfChanged(UpdatableRecordImpl.java:550) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.storeUpdate0(UpdatableRecordImpl.java:291) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.access$200(UpdatableRecordImpl.java:90) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl$3.operate(UpdatableRecordImpl.java:260) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:123) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.storeUpdate(UpdatableRecordImpl.java:255) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.update(UpdatableRecordImpl.java:149) ~[jooq-3.3.0.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager.persistRecord(JooqObjectManager.java:216) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager.setFieldsInternal(JooqObjectManager.java:125) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager$3.execute(JooqObjectManager.java:113) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.engine.idempotent.Idempotent.change(Idempotent.java:88) ~[cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager.setFields(JooqObjectManager.java:110) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.agent.server.resource.impl.AgentResourcesMonitorImpl.setHosts(AgentResourcesMonitorImpl.java:206) ~[cattle-iaas-agent-server-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.agent.server.resource.impl.AgentResourcesMonitorImpl$2.doWithLockNoResult(AgentResourcesMonitorImpl.java:125) ~[cattle-iaas-agent-server-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.agent.server.resource.impl.AgentResourcesMonitorImpl.pingReply(AgentResourcesMonitorImpl.java:116) ~[cattle-iaas-agent-server-0.5.0-SNAPSHOT.jar:na]
        at sun.reflect.GeneratedMethodAccessor726.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
        at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
        at io.cattle.platform.eventing.annotation.MethodInvokingListener$1.doWithLockNoResult(MethodInvokingListener.java:69) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.eventing.annotation.MethodInvokingListener.onEvent(MethodInvokingListener.java:65) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.eventing.impl.AbstractThreadPoolingEventService$2.doRun(AbstractThreadPoolingEventService.java:135) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.NoExceptionRunnable.runInContext(NoExceptionRunnable.java:13) ~[cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:108) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

2015-07-10 10:59:49,767 ERROR [:] [] [] [] [cutorService-44] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [145]
2015-07-10 10:59:49,768 ERROR [:] [] [] [] [cutorService-44] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 10:59:54,761 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [146]
2015-07-10 10:59:54,763 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 10:59:59,773 ERROR [:] [] [] [] [cutorService-55] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [147]
2015-07-10 10:59:59,775 ERROR [:] [] [] [] [cutorService-55] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 11:00:04,764 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [148]
2015-07-10 11:00:04,765 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 11:00:09,764 ERROR [:] [] [] [] [cutorService-54] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [149]
2015-07-10 11:00:09,764 ERROR [:] [] [] [] [cutorService-54] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 11:00:14,761 ERROR [:] [] [] [] [cutorService-47] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [150]
2015-07-10 11:00:14,763 ERROR [:] [] [] [] [cutorService-47] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 11:00:14,882 ERROR [:] [] [] [] [cutorService-57] [o.a.c.m.context.NoExceptionRunnable ] Uncaught exception org.jooq.exception.DataChangedException: Database record has been changed
        at org.jooq.impl.UpdatableRecordImpl.checkIfChanged(UpdatableRecordImpl.java:550) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.storeUpdate0(UpdatableRecordImpl.java:291) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.access$200(UpdatableRecordImpl.java:90) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl$3.operate(UpdatableRecordImpl.java:260) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:123) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.storeUpdate(UpdatableRecordImpl.java:255) ~[jooq-3.3.0.jar:na]
        at org.jooq.impl.UpdatableRecordImpl.update(UpdatableRecordImpl.java:149) ~[jooq-3.3.0.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager.persistRecord(JooqObjectManager.java:216) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager.setFieldsInternal(JooqObjectManager.java:125) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager$3.execute(JooqObjectManager.java:113) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.engine.idempotent.Idempotent.change(Idempotent.java:88) ~[cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.object.impl.JooqObjectManager.setFields(JooqObjectManager.java:110) ~[cattle-framework-object-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.agent.server.resource.impl.AgentResourcesMonitorImpl.setHosts(AgentResourcesMonitorImpl.java:206) ~[cattle-iaas-agent-server-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.agent.server.resource.impl.AgentResourcesMonitorImpl$2.doWithLockNoResult(AgentResourcesMonitorImpl.java:125) ~[cattle-iaas-agent-server-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.agent.server.resource.impl.AgentResourcesMonitorImpl.pingReply(AgentResourcesMonitorImpl.java:116) ~[cattle-iaas-agent-server-0.5.0-SNAPSHOT.jar:na]
        at sun.reflect.GeneratedMethodAccessor726.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
        at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
        at io.cattle.platform.eventing.annotation.MethodInvokingListener$1.doWithLockNoResult(MethodInvokingListener.java:69) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.eventing.annotation.MethodInvokingListener.onEvent(MethodInvokingListener.java:65) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]
        at io.cattle.platform.eventing.impl.AbstractThreadPoolingEventService$2.doRun(AbstractThreadPoolingEventService.java:135) ~[cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.NoExceptionRunnable.runInContext(NoExceptionRunnable.java:13) ~[cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:108) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

2015-07-10 11:00:19,760 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [151]
2015-07-10 11:00:19,761 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]
2015-07-10 11:00:24,761 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Failed to get ping from agent [1] count [152]
2015-07-10 11:00:24,761 ERROR [:] [] [] [] [cutorService-57] [i.c.p.a.s.ping.impl.PingMonitorImpl ] Scheduling reconnect for [1]

denise · July 13, 2015, 4:26pm

Did the agent ever show up on Rancher server when you were setting it up? Or has it always had the ping agent error?

Can you provide the logs of the Rancher agent container?

It looks as if there is a network connectivity issue between your server and agent. Can you also do a wget on the Rancher server URL from the agent to confirm network connectivity?

Topic		Replies	Views
Failed to get ping from agent Rancher 1.x	0	1251	January 26, 2017
Host registration Error in Default enviroment Rancher 1.x	2	1082	May 16, 2017
Rancher v1.0.0 cross host networking fails Rancher 1.x	17	3949	April 21, 2016
ERROR: https://RANCHER_IP:8443/ping is not accessible (The requested URL returned error: 404) on cattle-node-agent nodes	1	2941	March 31, 2020
Rancher/agent error sg="Error processing event" err=Timeout getting IP address with ustom Hosts Rancher 1.x	1	1392	March 13, 2018

Why do I get Failed to get ping from agent error?

Related Topics