Rancher Server (1.5.1) and agent same Host

pwFoo · March 13, 2017, 4:18pm

Hi,

I try to add the local Rancher Server as a host without success.

network setup

WAN interface: public internet connection for containers
MANAGEMENT interface: internal management interface between hosts (rancher server, rancher hosts, storage backends)

So I started Rancher Server with -p MANAGE_IP:8080:8080 and try to deploy the host with additional CATTLE_AGENT_IP="MANAGE_IP". Without deeper debugging it looks like a ipsec port conflict?

I could bind the agent to a second MANAGE_IP2, but don’t know how to do it.

Any idea? Anyone also use a management subnet / interface like described above?

pwFoo · March 14, 2017, 10:02am

@denise
I try to bind a 2nd manage ip to the cattle agent by CATTLE_AGENT_IP to avoid the ip / port conflict it looks like (host to rancher server ipsec, both use the 1st manage ip), but it doesn’t work.

How to deploy local host as rancher host? How to change the agent ip?

Rancher Server and agents are in the same management network:

By default, the IP of a VM with a private IP and public IP will be set to match the IP specified in the registration URL. For example, if a private IP is used in the registration URL, then the host’s private IP will be used. If you wanted to change the host’s IP address, you’ll need to edit the command provided from the UI. In order for the Rancher agent container to be launched correctly, set the CATTLE_AGENT_IP environment variable to the desired IP address. All the hosts within Rancher will need to be on the same network as Rancher server.

vincent · March 14, 2017, 12:04pm

The host connects to the server using whatever in the registration URL setting, and registers itself as whatever CATTLE_AGENT_IP is (or whether IP the request to the host comes in from of not specified, which is why you need to tell us the ip of its on the same host).

Registering does not involve IPSec. Once the host is showing up the IPSec service will be scheduled onto it. i don’t think it actually binds 500 & 4500 to only the agent IP, so if you are using them elsewhere that is going to be a problem.

pwFoo · March 14, 2017, 1:05pm

Hi @vincent,

two servers without any running services. Just

Rancher Server + Agent / Host
Rancher Agent / Host

So no ports should be blocked by running services.

Two servers with one public and one management (internal) interface. Both are rancher hosts, one is also rancher server. So I don’t know why it doesn’t work. I’ll check setup und firewall again.

pwFoo · March 14, 2017, 5:00pm

@vincent

Debugging info and fix: host firewall problem.

ipsec container log

+ trap 'exit 1' SIGTERM SIGINT
+ curl http://localhost:8111
+ export CHARON_PID_FILE=/var/run/charon.pid
+ CHARON_PID_FILE=/var/run/charon.pid
+ rm -f /var/run/charon.pid
+ export PIDFILE=/var/run/rancher-net.pid
+ PIDFILE=/var/run/rancher-net.pid
+ GCM=false
+ (( i=0 ))
+ (( i<6 ))
+ ip xfrm state add src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp mode tunnel aead 'rfc4106(gcm(aes))' 0x0000000000000000000000000000000000000001 128 sel src 1.1.1.1 dst 1.1.1.1
+ GCM=true
+ ip xfrm state del src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp
+ break
+ '[' false == true ']'
+ DEBUG=
+ mkdir -p /etc/ipsec
+ curl -f -u 22A5C3D91313052CEE3C:Wjw3xNAbKopSRiUpR45TEcPjKbS5tKLPkZbnioHW http://<MANAGEMENT_IP>:8080/v1/configcontent/psk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to <MANAGEMENT_IP> port 8080: No route to host
+ trap 'exit 1' SIGTERM SIGINT
+ curl http://localhost:8111
+ export CHARON_PID_FILE=/var/run/charon.pid
+ CHARON_PID_FILE=/var/run/charon.pid
+ rm -f /var/run/charon.pid
+ export PIDFILE=/var/run/rancher-net.pid
+ PIDFILE=/var/run/rancher-net.pid
+ GCM=false
+ (( i=0 ))
+ (( i<6 ))
+ ip xfrm state add src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp mode tunnel aead 'rfc4106(gcm(aes))' 0x0000000000000000000000000000000000000001 128 sel src 1.1.1.1 dst 1.1.1.1
+ GCM=true
+ ip xfrm state del src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp
+ break
+ '[' false == true ']'
+ DEBUG=
+ mkdir -p /etc/ipsec
+ curl -f -u 22A5C3D91313052CEE3C:Wjw3xNAbKopSRiUpR45TEcPjKbS5tKLPkZbnioHW http://<MANAGEMENT_IP>:8080/v1/configcontent/psk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to <MANAGEMENT_IP> port 8080: No route to host
+ trap 'exit 1' SIGTERM SIGINT
+ curl http://localhost:8111
+ export CHARON_PID_FILE=/var/run/charon.pid
+ CHARON_PID_FILE=/var/run/charon.pid
+ rm -f /var/run/charon.pid
+ export PIDFILE=/var/run/rancher-net.pid
+ PIDFILE=/var/run/rancher-net.pid
+ GCM=false
+ (( i=0 ))
+ (( i<6 ))
+ ip xfrm state add src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp mode tunnel aead 'rfc4106(gcm(aes))' 0x0000000000000000000000000000000000000001 128 sel src 1.1.1.1 dst 1.1.1.1
+ GCM=true
+ ip xfrm state del src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp
+ break
+ '[' false == true ']'
+ DEBUG=
+ mkdir -p /etc/ipsec
+ curl -f -u 22A5C3D91313052CEE3C:Wjw3xNAbKopSRiUpR45TEcPjKbS5tKLPkZbnioHW http://<MANAGEMENT_IP>:8080/v1/configcontent/psk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to <MANAGEMENT_IP> port 8080: No route to host
+ trap 'exit 1' SIGTERM SIGINT
+ curl http://localhost:8111
+ export CHARON_PID_FILE=/var/run/charon.pid
+ CHARON_PID_FILE=/var/run/charon.pid
+ rm -f /var/run/charon.pid
+ export PIDFILE=/var/run/rancher-net.pid
+ PIDFILE=/var/run/rancher-net.pid
+ GCM=false
+ (( i=0 ))
+ (( i<6 ))
+ ip xfrm state add src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp mode tunnel aead 'rfc4106(gcm(aes))' 0x0000000000000000000000000000000000000001 128 sel src 1.1.1.1 dst 1.1.1.1
+ GCM=true
+ ip xfrm state del src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp
+ break
+ '[' false == true ']'
+ DEBUG=
+ mkdir -p /etc/ipsec
+ curl -f -u 22A5C3D91313052CEE3C:Wjw3xNAbKopSRiUpR45TEcPjKbS5tKLPkZbnioHW http://<MANAGEMENT_IP>:8080/v1/configcontent/psk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to <MANAGEMENT_IP> port 8080: No route to host
+ trap 'exit 1' SIGTERM SIGINT
+ curl http://localhost:8111
+ export CHARON_PID_FILE=/var/run/charon.pid
+ CHARON_PID_FILE=/var/run/charon.pid
+ rm -f /var/run/charon.pid
+ export PIDFILE=/var/run/rancher-net.pid
+ PIDFILE=/var/run/rancher-net.pid
+ GCM=false
+ (( i=0 ))
+ (( i<6 ))
+ ip xfrm state add src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp mode tunnel aead 'rfc4106(gcm(aes))' 0x0000000000000000000000000000000000000001 128 sel src 1.1.1.1 dst 1.1.1.1
+ GCM=true
+ ip xfrm state del src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp
+ break
+ '[' false == true ']'
+ DEBUG=
+ mkdir -p /etc/ipsec
+ curl -f -u 22A5C3D91313052CEE3C:Wjw3xNAbKopSRiUpR45TEcPjKbS5tKLPkZbnioHW http://<MANAGEMENT_IP>:8080/v1/configcontent/psk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to <MANAGEMENT_IP> port 8080: No route to host
+ trap 'exit 1' SIGTERM SIGINT
+ curl http://localhost:8111
+ export CHARON_PID_FILE=/var/run/charon.pid
+ CHARON_PID_FILE=/var/run/charon.pid
+ rm -f /var/run/charon.pid
+ export PIDFILE=/var/run/rancher-net.pid
+ PIDFILE=/var/run/rancher-net.pid
+ GCM=false
+ (( i=0 ))
+ (( i<6 ))
+ ip xfrm state add src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp mode tunnel aead 'rfc4106(gcm(aes))' 0x0000000000000000000000000000000000000001 128 sel src 1.1.1.1 dst 1.1.1.1
+ GCM=true
+ ip xfrm state del src 1.1.1.1 dst 1.1.1.1 spi 42 proto esp
+ break
+ '[' false == true ']'
+ DEBUG=
+ mkdir -p /etc/ipsec
+ curl -f -u 22A5C3D91313052CEE3C:Wjw3xNAbKopSRiUpR45TEcPjKbS5tKLPkZbnioHW http://<MANAGEMENT_IP>:8080/v1/configcontent/psk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to <MANAGEMENT_IP> port 8080: No route to host

ip address / route before ipsec container restarts

docker exec -ti 0f63d0cbbe5c ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: eth0@if131: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:59:ac:77:c0:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.42.98.195/16 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::504f:97ff:fe2c:ef8d/64 scope link tentative dadfailed 
       valid_lft forever preferred_lft forever

docker exec -ti 0f63d0cbbe5c ip route
default via 10.42.0.1 dev eth0 
10.42.0.0/16 dev eth0  proto kernel  scope link  src 10.42.98.195 
169.254.169.250 via 10.42.0.1 dev eth0

docker host

ip a | grep 10.42.0.1
    inet 10.42.0.1/16 scope global docker0

pwFoo · March 14, 2017, 5:02pm

@vincent

is the container IP fixed or changes every update?
Would be nice to separate management connections (rancher server, host, agent, …) from production containers.

vincent · March 15, 2017, 5:46am

The IPs of a service’s containers generally change every time they are upgraded.

Topic		Replies	Views
Cross host container linking doesn't work (AWS) Rancher 1.x	3	1259	January 11, 2016
Rancher agent on same host as server RancherOS	3	2287	February 29, 2016
How to set up rancher with public and private hosts? Rancher 1.x	0	1409	September 25, 2017
Rancher server and agent on same vps Rancher	5	3033	June 18, 2018
Unable to add second host again Rancher 1.x	2	1332	March 21, 2016

Rancher Server (1.5.1) and agent same Host

Related topics