Hello,
Last time i’ve check, Traefik wasn’t working out of the box on the Rancher Products.
This is following my another here about RancherOS/Rancher.
- Using Helm package didn’t get our Ingress directives as ready on the UI
- Using the guide on the traefik wasn’t working as it is not up to date of the 2.0 Rancher release.
- We are going to use the lastest release of Traefik 2.0
So what can we do ? Try, try and try until we figure it out
But first a bet of reminder about what is Traefik:
Traefik is a modern HTTP reverse proxy and load balancer that makes deploying microservices easy. Traefik integrates with your existing infrastructure components (Docker, Swarm mode, Kubernetes, Marathon, Consul, Etcd, Rancher, Amazon ECS, …) and configures itself automatically and dynamically. Pointing Traefik at your orchestrator should be the only configuration step you need
Disclaimer
We well not talk about how to set or use Rancher products and associate tools. You need to be aware of how to use Rancher Products and kubectl tool.
This topic could be upgraded at any times regarding feedback so don’t hesitate to comment.
Objectives
Running Rancher on top of a single RancherOS server i needed to have a simple solution in order to expose my services to the WWW. Traefik have a single feature that i didn’t found on any other (yet) Ingress controller (Ambassador, Istio, Haproxy, External-DNS etc…) which is to be able to use Letsencrypt DNS Challenge with a support of OVH as a provider.
Let’s do it 
:
Beware that i’ve been stuck to make network communication worked between projects as i’m using Canal as my network provider with “Project Network Isolation” set to Enabled at first. In order to manager/organize your application between several projects like backend/frontend/backoffice etc you will need to set this settings to false and manage Network Policy to restrict your network flows.
- Create a dedicated ns ( or not it’s up to you but i recommend it ) without a restricting policy in place as we are going to bind to restricted port ( 80/443/22 )
kubectl create ns traefik
- Create the Traefik Custom Ressources Definitions
This is a new configuration required regarding Traefik 1.7.
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutes.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRoute
plural: ingressroutes
singular: ingressroute
scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutetcps.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRouteTCP
plural: ingressroutetcps
singular: ingressroutetcp
scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: middlewares.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: Middleware
plural: middlewares
singular: middleware
scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tlsoptions.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: TLSOption
plural: tlsoptions
singular: tlsoption
scope: Namespaced
- RBAC Settings
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- traefik.containo.us
resources:
- middlewares
verbs:
- get
- list
- watch
- apiGroups:
- traefik.containo.us
resources:
- ingressroutes
verbs:
- get
- list
- watch
- apiGroups:
- traefik.containo.us
resources:
- ingressroutetcps
verbs:
- get
- list
- watch
- apiGroups:
- traefik.containo.us
resources:
- tlsoptions
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: traefik
- Traefik Configuration file
Traefik 2.0 can use static file & dynamic configuration through labels and annotations. In our case we are going an traefik.yml file.
## Static configuration
entryPoints:
web:
address: ":8000"
web-secure:
address: ":4443"
ssh:
address: ":2222"
certificatesResolvers:
default:
acme:
email: YOUREMAIL@acme.com
storage: /data/acme.json
dnsChallenge:
provider: ovh
resolvers:
- "1.1.1.1:53"
- "8.8.8.8:53"
providers:
kubernetesCRD:
namespaces:
- "default"
- "production"
- "traefik"
- "x"
- "y"
file:
directory: /local/traefik-file-provider
api:
dashboard: true
accessLog: {}
traefik-file-provider.yml
http:
routers:
api-dashboard:
entryPoints:
- "web-secure"
rule : Host(`traefik.example.acme`) && PathPrefix(`/api`) || Host(`traefik.example.acme`) && PathPrefix(`/dashboard`)
tls:
certResolver: default
service: api@internal
middlewares:
- auth-dashboard
middlewares:
auth-dashboard:
basicAuth:
users:
- "test:$apr1$Jse1O.XT$zJu7lNeaM1zJT8PuZMrvW1"
- "test2:$apr1$d9hr9HBB$4HxwgUir3HP4EsggP/QNo0"
kubectl create configmap traefik-config-yml --from-file=./config-maps/traefik -n traefik
- Environment variables
To be able to reach OVH Api you need to have some api crendential in order to the Trafik Controller to reach and interact with your OVH Account.
Have a look on thoses links:
Let’s create the secret file which will have my api keys.
apiVersion: v1
kind: Secret
metadata:
name: ovh-credentials
type: Opaque
data:
ovh_endpoint: BASE64
ovh_application_key: BASE64
ovh_application_secret: BASE64
ovh_consumer_key: BASE64
Now that we have our file let’s import it on the Rancher
kubectl create -n traefik -f traefik-ovh-secret.yml
- Deployment of Traefik
Check using kubectl get pods -n traefik to see how your’re pods are going on and to check the logs if any goes wrong.
I’m using the same logic as the official Traefik documentation.
apiVersion: v1
kind: Service
metadata:
name: traefik
spec:
ports:
- protocol: TCP
name: web
port: 8000
- protocol: TCP
name: admin
port: 8080
- protocol: TCP
name: websecure
port: 4443
- protocol: TCP
name: ssh
port: 2222
selector:
app: traefik
---
apiVersion: v1
kind: Service
metadata:
name: whoami
spec:
ports:
- protocol: TCP
name: web
port: 80
selector:
app: whoami
---
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: traefik
name: traefik-ingress-controller
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik
labels:
app: traefik
spec:
replicas: 1
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
env:
- name: TZ
value: 'Europe/Brussels'
- name: OVH_ENDPOINT
valueFrom:
secretKeyRef:
name: ovh-credentials
key: ovh_endpoint
- name: OVH_APPLICATION_KEY
valueFrom:
secretKeyRef:
name: ovh-credentials
key: ovh_application_key
- name: OVH_APPLICATION_SECRET
valueFrom:
secretKeyRef:
name: ovh-credentials
key: ovh_application_secret
- name: OVH_CONSUMER_KEY
valueFrom:
secretKeyRef:
name: ovh-credentials
key: ovh_consumer_key
image: traefik:v2.0
args:
- --configFile=/local/traefik/traefik.yml
ports:
- name: web
containerPort: 8000
hostPort: 80
- name: websecure
containerPort: 4443
hostPort: 443
protocol: TCP
- name: ssh
containerPort: 2222
hostPort: 22
protocol: TCP
- name: admin
containerPort: 8080
volumeMounts:
- mountPath: /local/traefik
name: traefik-config
- mountPath: /local/traefik-file-provider
name: traefik-file-provider
- mountPath: /data
name: traefik-ssl-storage
volumes:
- configMap:
defaultMode: 256
name: traefik-config-yml
optional: false
name: traefik-config
- configMap:
defaultMode: 256
name: traefik-file-provider
optional: false
name: traefik-file-provider
- hostPath:
path: /mnt/system/traefik-data
type: DirectoryOrCreate
name: traefik-ssl-storage
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: whoami
labels:
app: whoami
spec:
replicas: 2
selector:
matchLabels:
app: whoami
template:
metadata:
labels:
app: whoami
spec:
containers:
- name: whoami
image: containous/whoami
ports:
- name: web
containerPort: 80
- IngressRoute
Since the 2.0 i didn’t find a way throught the documentation of the use of acme certificate generation using classic ingress definition. I’ve been force to move my ingress to an IngressRoute which is not available through the UI of Rancher.
As the documentation say, let’s deploy an IngressRoute for the whoami container.
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: simpleingressroute
spec:
entryPoints:
- web
routes:
- match: Host(`whoami.YOURDOMAIN.com`) && PathPrefix(`/notls`)
kind: Rule
services:
- name: whoami
port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutetls
spec:
entryPoints:
- web-secure
routes:
- match: Host(`whoami.YOURDOMAIN.com`) && PathPrefix(`/tls`)
kind: Rule
services:
- name: whoami
port: 80
tls:
certResolver: default
- Way of doing storage related to security
As you saw i’ve been only using “hostPath” instruction for most of my examples.
This could be changing on VolumesClaims to classic volumes which could also use “hostPath”.
Using VolumeClaims will let you use restricted policy inside of your Rancher and let you feel more safe about yours pods
RancherOS SSH Port Settings
Open a console into your rancheros server then use a file like :
rancher:
ssh:
port: 2022
Then use ros config merge -i cloud-ssh.yml in order to change the ssh port of your server.
Gitlab Repository
I’ve created an Gitlab repository in order to host my apps yaml files to provide live example.
Bibliography
Traefik CRD TLS Documentation - Traefik https://github.com/helm/charts/tree/master/stable/traefik
Configuring Traefik for the dns-01 challenge with OVH as DNS provider | by Antoine Hamon | Nephely | Medium
.