Deploying Scalable Traefik Ingress with NFS Persistence Behind External HAProxy Load Balancers

Introduction

In this guide, we will explore how to deploy a scalable Traefik ingress controller with NFS persistence behind external HAProxy load balancers. This setup is ideal for managing incoming traffic to your Docker Swarm or Kubernetes clusters while ensuring high availability and persistence for your applications.

Prerequisites

Before we begin, ensure you have the following prerequisites in place:

A server or virtual machine with Docker and Docker Swarm installed.
HAProxy installed and configured as an external load balancer.
An NFS server set up for persistent storage.
Basic knowledge of Docker, Docker Swarm, and Traefik.

Overview of the Deployment

The deployment consists of the following components:

HAProxy Load Balancers: These will handle incoming traffic and distribute it to the Traefik ingress controllers.
Traefik Ingress Controllers: These will manage routing of requests to the Docker Swarm or Kubernetes services.
NFS Persistence: This will provide persistent storage for the Traefik configurations and certificates.

Diagram

+-------------------+          +------------------+          +------------------+
|                   |          |                  |          |                  |       
|   Client Browser  +--------->|   HAProxy        +--------->|   Traefik        |
|                   |          |   Load Balancer  |          |   Ingress        |
+-------------------+          +--------+---------+          +--------+---------+

                                        |                           |
                                        |                           |
                                +--------v---------+         +-------v--------+
                                |                  |         |                |
                                |   NFS Server     |         |   Backend Apps |
                                |   (Persistence)  |         |                |
                                +------------------+         +----------------+

Directory Structure

/mnt/nfs-share/traefik/
├── acme.json          # Traefik ACME (Let's Encrypt) storage/ if we use self-signed certs
├── certs/            # Directory for SSL certificates 
├── traefik.yml       # Traefik configuration file  
└── logs/             # Directory for Traefik logs

Benifites of traefik ingress with nfs persistence behind haproxy

Scalability: Easily scale Traefik instances based on traffic demands.
High Availability: HAProxy ensures that traffic is evenly distributed and provides failover capabilities.
Persistence: NFS provides a reliable way to store Traefik configurations and SSL certificates.
Centralized Management: Traefik’s dynamic configuration capabilities allow for easy management of routing rules and services.

Step by step Deployment

Step 1: Configure HAProxy Load Balancer

Refer to the Implementation of HAProxy for SSL/TLS Termination and Traffic Distribution guide for detailed instructions on setting up HAProxy as a load balancer.

Step 2: Set Up NFS Server

Refer to the Setting Up an NFS Server for Persistent Storage guide for instructions on configuring an NFS server.

Step 3: Deploy Traefik Ingress Controller

a. Create a traefik.yml configuration file for Traefik:

api:
  dashboard: true ## Enable the Traefik dashboard
  insecure: false ## Disable insecure access to the dashboard

entryPoints:
  web:
    address: ":80"
    # 1. Accept the binary Proxy Protocol from HAProxy
    proxyProtocol:
      trustedIPs:
        - "127.0.0.1/32"  # Localhost
        - "192.168.xx.xx/32"  # Your standalone HAProxy Node IP
    # 2. Trust the HTTP Headers (X-Forwarded-For)
    forwardedHeaders: ## Trust the X-Forwarded headers from HAProxy
      trustedIPs:
        - "127.0.0.1/32"
        - "192.168.xx.xx/32"

  websecure: ## Secure entry point for HTTPS
    address: ":443"
    proxyProtocol:
      trustedIPs:
        - "127.0.0.1/32"
        - "192.168.xx.xx/32"
    forwardedHeaders:
      trustedIPs:
        - "127.0.0.1/32"
        - "192.168.xx.xx/32"
  metrics: ## Metrics endpoint
    address: ":8082"

metrics:
  prometheus:
    entryPoint: metrics
    addEntryPointsLabels: true ## Add labels for entry points
    addRoutersLabels: true ## Add labels for routers
    addServicesLabels: true ## Add labels for services

ping:
  entryPoint: web ## Ping endpoint for health checks

providers:
  # This is the dedicated Swarm provider for Traefik v3
  swarm:
    endpoint: "unix:///var/run/docker.sock" ## Docker socket for Swarm
    exposedByDefault: false
    network: traefik

log:
  level: INFO

b. Create the necessary directories for Traefik on the NFS share:

sudo mkdir -p /mnt/nfs-share/traefik/certs
sudo mkdir -p /mnt/nfs-share/traefik/logs
sudo touch /mnt/nfs-share/traefik/acme.json
sudo chmod 600 /mnt/nfs-share/traefik/acme.json

c. For self signed certificates, place your cert.pem and key.pem files in the certs directory:

sudo mv cert.pem /mnt/nfs-share/traefik/certs/cert.pem
sudo mv key.pem /mnt/nfs-share/traefik/certs/key.pem

d. Deploy Traefik using Docker Swarm with the following docker-compose.yml file:

services:
  traefik:
    image: traefik:v3.6.7
    hostname: "{{.Node.Hostname}}-traefik" # Set hostname for each Traefik instance
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /mnt/nfs-share/traefik/traefik.yml:/etc/traefik/traefik.yml:ro # Traefik config
    ports:
      - target: 80 # Public HTTP Port
        published: 80
        protocol: tcp # TCP protocol
        mode: host
      - target: 443       
        published: 443      
        protocol: tcp       
        mode: host          
      - target: 8080
        published: 8080
        protocol: tcp
        mode: host
    networks:
      - traefik
      - monitoring
    environment:
      - TZ=Asia/Dhaka
    deploy:
      mode: global # Deploy one instance per node
      placement:
        constraints: [node.role == manager] # Only on manager nodes
      labels:
        - "traefik.enable=true"
        - "traefik.swarm.network=traefik" # Specify the Traefik network
        # Dashboard Router Configuration
        - "traefik.http.routers.traefik-dashboard.rule=Host(`traefik.riad.com.bd`)"
        - "traefik.http.routers.traefik-dashboard.entrypoints=web"
        - "traefik.http.routers.traefik-dashboard.service=api@internal"
        # THE FIX: Link the router to the auth middleware
        - "traefik.http.routers.traefik-dashboard.middlewares=dashboard-auth"
        # Define the Authentication Middleware
        - "traefik.http.middlewares.dashboard-auth.basicauth.users=admin:$$2y$$05$$4fBcZsW4Oa7KhuxDpJJ9vu/D0g2.T9JrOFuFfLauoIKWJEuFr1ouu"
        # THE FIX: Explicitly define the port for the traefik-traefik service
        - "traefik.http.services.traefik-traefik.loadbalancer.server.port=8080"

networks:
  traefik:
    external: true
  monitoring:
    external: true

e. Deploy the stack to Docker Swarm:

docker stack deploy -c docker-compose.yml traefik

Conclusion

You have successfully deployed a scalable Traefik ingress controller with NFS persistence behind external HAProxy load balancers in a Docker Swarm
environment. This setup ensures high availability, scalability, and persistence for your applications. You can now manage your services and routing rules through the Traefik dashboard and take advantage of the robust features provided by Traefik and HAProxy. For further customization and advanced configurations, refer to the Traefik documentation and HAProxy documentation.

Next Steps

Explore advanced Traefik features such as middleware, rate limiting, and circuit breakers.
Monitor Traefik and HAProxy performance using Prometheus and Grafana.
Implement SSL/TLS certificates using Let’s Encrypt with Traefik.
Scale your Docker Swarm services and observe how Traefik manages the traffic routing.

Additional Resources

FAQ

Q1: Can I use this setup with Kubernetes instead of Docker Swarm?

Yes, you can adapt this setup for Kubernetes by deploying Traefik as an ingress controller in your Kubernetes cluster and configuring HAProxy as an external load balancer. Ensure that you adjust the configurations accordingly to fit the Kubernetes environment.

Q2: How do I secure the Traefik dashboard?

The Traefik dashboard can be secured using basic authentication middleware, as demonstrated in the docker-compose.yml file. You can customize the username and password hash to enhance security. Additionally, consider restricting access to the dashboard to specific IP addresses or networks.

Q3: What are the benefits of using NFS for persistence?

NFS provides a centralized and reliable storage solution that allows multiple Traefik instances to share the
same configuration and SSL certificates. This ensures consistency across instances and simplifies management, especially in a scalable environment.

Q4: How can I monitor the performance of Traefik and HAProxy?

You can monitor the performance of Traefik and HAProxy using Prometheus and Grafana. Both Traefik and HAProxy expose metrics that can be scraped by Prometheus. You can then visualize these metrics in Grafana dashboards to gain insights into traffic patterns, response times, and overall performance.

Q5: Can I use Let’s Encrypt with this setup for SSL certificates?

Yes, Traefik has built-in support for Let’s Encrypt, allowing you to automatically obtain and renew SSL certificates. You can configure Traefik to use Let’s Encrypt by adding the appropriate settings in the traefik.yml configuration file. Make sure to set up the necessary DNS records and ensure that your HAProxy load balancer forwards traffic correctly to Traefik for certificate validation.

Operating Systems

Containerization

Automation Tools

Deploying Scalable Traefik Ingress with NFS Persistence Behind External HAProxy Load Balancers

Introduction

Prerequisites

Overview of the Deployment

Diagram

Directory Structure

Benifites of traefik ingress with nfs persistence behind haproxy

Step by step Deployment

Step 1: Configure HAProxy Load Balancer

Step 2: Set Up NFS Server

Step 3: Deploy Traefik Ingress Controller

Conclusion

Next Steps

Additional Resources

FAQ

Q1: Can I use this setup with Kubernetes instead of Docker Swarm?

Q2: How do I secure the Traefik dashboard?

Q3: What are the benefits of using NFS for persistence?

Q4: How can I monitor the performance of Traefik and HAProxy?

Q5: Can I use Let’s Encrypt with this setup for SSL certificates?

​Introduction

​Prerequisites

​Overview of the Deployment

​Diagram

​Directory Structure

​Benifites of traefik ingress with nfs persistence behind haproxy

​Step by step Deployment

​Step 1: Configure HAProxy Load Balancer

​Step 2: Set Up NFS Server

​Step 3: Deploy Traefik Ingress Controller

​Conclusion

​Next Steps

​Additional Resources

​FAQ

​Q1: Can I use this setup with Kubernetes instead of Docker Swarm?

​Q2: How do I secure the Traefik dashboard?

​Q3: What are the benefits of using NFS for persistence?

​Q4: How can I monitor the performance of Traefik and HAProxy?

​Q5: Can I use Let’s Encrypt with this setup for SSL certificates?

Introduction

Prerequisites

Overview of the Deployment

Diagram

Directory Structure

Benifites of traefik ingress with nfs persistence behind haproxy

Step by step Deployment

Step 1: Configure HAProxy Load Balancer

Step 2: Set Up NFS Server

Step 3: Deploy Traefik Ingress Controller

Conclusion

Next Steps

Additional Resources

FAQ

Q1: Can I use this setup with Kubernetes instead of Docker Swarm?

Q2: How do I secure the Traefik dashboard?

Q3: What are the benefits of using NFS for persistence?

Q4: How can I monitor the performance of Traefik and HAProxy?

Q5: Can I use Let’s Encrypt with this setup for SSL certificates?