Implementing Docker Swarm for Application High Availability

In this project, I address the challenges faced by many large global organizations with containerized applications, including frequent workload failures, potential data loss, and lack of robust orchestration. The objective was to implement Docker Swarm for high availability, enable automatic rescheduling of failed containers, and set up a scalable infrastructure — and do all of this quickly so data can be restored for the containers that are no longer in a healthy state.

Problem:
Companies often face significant challenges with their containerized applications. The issues included frequent failures of containerized workloads, potential data loss due to the inability to reschedule failed containers, lack of a robust container orchestration system, and the need for a scalable and fault-tolerant infrastructure. Often times, teams lack the knowledge about Docker Swarm and require a comprehensive guide for setup and management.

Solution:
To address these challenges, I implemented a solution using Docker Swarm on AWS EC2 instances. The solution involved the following steps:

1. Provisioned AWS EC2 Instances
   - Launched three AWS EC2 Ubuntu instances to serve as networked hosts for Swarm. I configured the instances to be in the same VPC and subnet.
   - Configured the instances in the same security group and opened ports 22 (SSH), 2377 (Swarm), 4789 (overlay network), and 7946 (communication).
2. Installed Docker:
   - Connected to each EC2 instance via SSH using remote terminal in Visual Studio Code.
   - Installed Docker on all three hosts and verified it was in an active state on each host.

# Set up Docker's apt repository

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install the latest version of Docker
 sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Start and enable Docker
sudo systemctl start docker
sudo systemctl enable docker

# Verify Docker installation:
sudo docker --version
sudo systemctl status docker

3. Configured Hostnames:
- Changed the hostname on the master node and two worker nodes to clearly identify them.

sudo hostnamectl set-hostname master-node

sudo hostnamectl set-hostname worker-node-1

sudo hostnamectl set-hostname worker-node-2

4. Validated Security Group and Added SSH Key:
- Confirmed that the two worker nodes shared the same security group as the master node so they could properly communicate.
- Added the SSH key to each node for secure access.

5. Created the Docker Swarm:
- Initialized Swarm on the master node.
- Joined the two worker nodes to the Swarm.

sudo docker swarm join-token worker

sudo docker swarm join --token <token> <master-node-private-ip>:2377

6. Deployed the Docker Global Service:
- Using the Docker CLI, created and ran a global service for the application.

sudo docker service create --name global-service --mode global --publish published=80,target=80 nginx

7. Verified Swarm Status:
- Checked the status of the Docker nodes and verified the service was running on all nodes.

sudo docker node ls

sudo docker service ls
sudo docker service ps global-service

These documented steps serve as a guide for installing Docker. By following these steps, anyone unfamiliar with Docker Swarm can successfully set up a Docker Swarm cluster using AWS EC2 instances to ensure applications can handle container orchestration and failover, which mitigates the risk of data loss and service disruptions.

In summary, by implementing this Docker Swarm solution companies can achieve several benefits:

1. High Availability: The Swarm ensures that containers are automatically rescheduled if they fail, minimizing downtime.
2. Scalability: The global service allows easy scaling of the application across all nodes in the Swarm.
3. Fault Tolerance: By distributing containers across multiple nodes, the system becomes more resilient to individual node failures.
4. Efficient Resource Utilization: Docker Swarm’s orchestration capabilities optimize the use of available resources across the cluster.
5. Simplified Management: The Swarm provides a unified view of the entire cluster, making it easier to manage and monitor the application.

This project allowed me to gain experience setting up and configuring Docker Swarm. I especially appreciated seeing the power of AMS EC2 combined with Docker Swarm firsthand.