I have a Rancher Server running on Rancher OS on an AWS t2.micro instance with external MySQL db.
I seemed to be having some trouble getting my hosts to talk to each other when trying to use and AWS ELB. I also decided that I’d like to not have to rely on AWS for my setup so much. Even though I’m still using an EC2 instance, the config below should be able to run from anywhere (i.e. it does not rely on a specific AWS component like ELB and so is more in the spirit of Rancher and easily portable elsewhere).
So I’m configuring Rancher OS to stand up a single node Rancher Server with a cloud-config that I paste into the EC2 instance via the plain text user-data field that you can modify when starting up an EC2 intance. This same cloud-config could be used by directly creating it and saving it to ‘/var/lib/rancher/conf/cloud-config.yml’ on your Rancher OS server if you are not running on EC2. At some point I’m hoping to modify this cloud-config and move to a 3 node HA configuration, but one-step at a time and so this is only single node for the moment.
So my cloud-config adds my ssh key and then sets up all the stuff required for nginx-proxy, automated Let’s Encrypt certificate registration and renewal for my chosen rancher server domain and then setup of Rancher Server itself. My cloud-config looks like this:
#cloud-config
ssh_authorized_keys:
- ssh-rsa <my-public-key>
rancher:
services:
nginx-proxy:
image: jwilder/nginx-proxy:0.4.0
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- '/home/docker/nginx-proxy/ssl:/etc/nginx/certs:ro'
- '/etc/nginx/vhost.d'
- '/usr/share/nginx/html'
- '/var/run/docker.sock:/tmp/docker.sock:ro'
letsencrypt-nginx-proxy-companion:
image: jrcs/letsencrypt-nginx-proxy-companion:v1.3
restart: unless-stopped
volumes_from:
- nginx-proxy
volumes:
- '/home/docker/nginx-proxy/ssl:/etc/nginx/certs:rw'
- '/var/run/docker.sock:/var/run/docker.sock:ro'
rancher-server:
image: rancher/server:v1.2.0-pre3
restart: unless-stopped
environment:
- VIRTUAL_PORT=8080
- VIRTUAL_HOST=<my.domain.com>
- LETSENCRYPT_TEST=false
- LETSENCRYPT_HOST=<my.domain.com>
- LETSENCRYPT_EMAIL=<myemail@com>
- CATTLE_DB_CATTLE_MYSQL_HOST=<mysql-ip-or-hostname>
- CATTLE_DB_CATTLE_MYSQL_PORT=<mysql-port>
- CATTLE_DB_CATTLE_MYSQL_NAME=<mysql-schema-name>
- CATTLE_DB_CATTLE_USERNAME=<mysql-user>
- CATTLE_DB_CATTLE_PASSWORD=<my-sql-pass>
I allow traffic from anywhere through to my EC2 instance on port 80 and port 443 and then create an A record at my Registrar to point <my.domain.com> to the Elastic IP I am using for the EC2 instance. I also open up port 22 for my IP so I can login to the EC2 instance and check on what the three docker containers are doing. I also keep an eye on activity on the MySQL server to make sure the Rancher tables are being created. After about 10 mins everything seems to have worked and I’m able to navigate to my Let’s Encrypt secured https://my.domain.com. My Rancher Server is up and running and I can secure it with one of the Rancher Access Control methods.
Now the thing I want to check, before I go further and add hosts, is whether I need to modify the nginx configuration that the cloud-config above gave me. I’m not that familiar with nginx but it looks like what I have as a result of the above (see below) does pretty much everything that the Rancher Docs for nginx proxying describes here. It’s just some of it is not in ‘/ location’ part of the ssl server and I’m not using --link in my nginx startup. I don’t think I need --link because the AWS internal VPC ip is being used instead. I realise I’m missing the proxy_read_timeout bit to set from 1min to 15mins, but I can modify my Dockerfile to sed this in I think.
Can someone more familiar with nginx proxying to rancher server tell me if I’m missing anything crucial in the nginx below? If it’s all good then hopefully this post will be useful to someone trying to quickly setup a secure single node Rancher Server. It took me a couple of days to figure out and so I hope it saves someone some time and frustration.
# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
default $http_x_forwarded_proto;
'' $scheme;
}
# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
# Connection header that may have been passed to this server
map $http_upgrade $proxy_connection {
default upgrade;
'' close;
}
gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
log_format vhost '$host $remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
access_log off;
# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
server {
server_name _; # This is just an invalid value which will never trigger on a real hostname.
listen 80;
access_log /var/log/nginx/access.log vhost;
return 503;
}
upstream <my.domain.com> {
## Can be connect with "bridge" network
# rancher-server
server <internal-aws-vpc-ip>:8080;
}
server {
server_name <my.domain.com>;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
return 301 https://$host$request_uri;
}
server {
server_name <my.domain.com>;
listen 443 ssl http2 ;
access_log /var/log/nginx/access.log vhost;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA;
ssl_prefer_server_ciphers on;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:50m;
ssl_certificate /etc/nginx/certs/<my.domain.com>.crt;
ssl_certificate_key /etc/nginx/certs/<my.domain.com>.key;
ssl_dhparam /etc/nginx/certs/<my.domain.com>.dhparam.pem;
add_header Strict-Transport-Security "max-age=31536000";
include /etc/nginx/vhost.d/default;
location / {
proxy_pass http://<my.domain.com>;
}
}