Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pa/kodekloud-assignment
kodekloud docker swarm cluster assignment
https://github.com/pa/kodekloud-assignment
apparmor-profile dind docker docker-compose docker-swarm-cluster priviliged-conatiner
Last synced: about 2 months ago
JSON representation
kodekloud docker swarm cluster assignment
- Host: GitHub
- URL: https://github.com/pa/kodekloud-assignment
- Owner: pa
- Created: 2022-06-03T11:08:55.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-06-08T18:36:16.000Z (over 2 years ago)
- Last Synced: 2023-10-20T09:02:05.708Z (about 1 year ago)
- Topics: apparmor-profile, dind, docker, docker-compose, docker-swarm-cluster, priviliged-conatiner
- Language: Shell
- Homepage: https://www.loom.com/share/00a853072d9445279cb9c85213e48913
- Size: 1.71 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# kodekloud-assignment
This repo contains solutions for two tasks,
1. Deploy a docker swarm stack that would run a container with `privileged` mode. For more details check [here](task-one/README.md).
2. Run a ubuntu based docker container which can inturn run docker inside `(docker in docker mode)`. For more details check [here](task-two/README.md).
## Directory Structure
```bash
.
├── README.md
├── docker-compose.yml
├── task-one
│ ├── Dockerfile
│ ├── README.md
│ ├── arch_diagram.png
│ ├── container-handler.sh
│ └── docker-compose.yml
└── task-two
├── Dockerfile
├── README.md
├── apparmor-profiles
│ ├── audit-all-writes
│ └── deny-all-writes
├── arch_diagram.png
├── container-handler.sh
└── docker-compose.yml
```
- [docker-compose.yml](docker-compose.yml) - Used to deploy docker swarm stack with two services ([task-one](task-one) and [task-two](task-two)) in a docker swarm cluster.
- [task-one](task-one)
- [Dockerfile](task-one/Dockerfile) - Copies [container-handler](task-one/container-handler.sh) script into the filesystem of the container and execute it within the container
- [README.md](task-one/README.md) - Contains instructions to build image and deploy docker swarm stack in swarm cluster
- [arch_diagram.png](task-one/arch_diagram.png) - Architecture diagram for task one docker swarm stack
- [container-handler.sh](task-one/container-handler.sh) - Creates two sibiling containers, one with `priviliged` mode enabled. Also waits for `docker stack rm ` signal and kills the sibiling when `SIGTERM` is received
- [docker-compose.yml](task-one/docker-compose.yml) - It has one service and uses an existing network to deploy `container-handler` container to bring up sibiling containers, one with `priviliged` mode enabled
- [task-two](task-two)
- [Dockerfile](task-two/Dockerfile) - Copies [apparmor profiles](task-two/apparmor-profiles/) and [container-handler](task-two/container-handler.sh) script into the filesystem of the container and execute the script within container
- [README.md](task-two/README.md) - Contains instructions to build image and deploy docker swarm stack in swarm cluster
- [apparmor-profiles](task-two/apparmor-profiles/)
- [audit-all-writes](task-two/apparmor-profiles/audit-all-writes) - This profile will audit all the writes `(i.e creating dirs/files)` happening within the container and logs it to the kernel log
- [deny-all-writes](task-two/apparmor-profiles/deny-all-writes) - This profile will deny all the writes `(i.e creating dirs/files)` happening within the container and logs it to the kernel log
- [arch_diagram.png](task-two/arch_diagram.png) - Architecture diagram for task two docker swarm stack
- [container-handler.sh](task-two/container-handler.sh) - Creates one sibiling upper containers with `priviliged` mode, updates/installs apparmor packages, copies apparmor profiles to `/etc/apparmo.d/` in the upper container, creates two child inner containers with apparmor profiles applied. Also waits for `docker stack rm ` signal and kills the sibiling upper container `(including child inner containers)` when `SIGTERM` is received
- [docker-compose.yml](task-two/docker-compose.yml) - It has one service and uses an existing network to deploy `container-handler` container to bring up sibiling upper container with `priviliged` mode enabled and two child inner containers
## Deploy [task-one](task-one) and [task-two](task-two) to Swarm Cluster
In this section, we will be deploying both the tasks [task-one](task-one) and [task-two](task-two) in swarm cluster using a single [docker-compose](docker-compose.yml) file. We can deploy this stack to any cloud platform or even a local machine (with docker installed). For this deployment I'm going to use AWS Cloud platform.
### Pre-requisites
#### Docker Images
- Task one service image - You can either build image using [Dockerfile](task-one/Dockerfile) (_if you also want to use custom image name and tag, make sure to change the same in [docker-compose.yml](docker-compose.yml#L12)_) or you can use my docker image in [pramodhayyappan/kk-task-one-container-handler](https://hub.docker.com/repository/docker/pramodhayyappan/kk-task-one-container-handler) in dockerhub
```bash
# To build image with custom name and tag. By default, it uses the latest tag
docker build -f task-one/Dockerfile task-one -t pramodhayyappan/kk-task-one-container-handler:
```
- Task two service image - You can either build image using [Dockerfile](task-one/Dockerfile) (_if you also want to use custom image name, make sure to change the same image name in [docker-compose.yml](docker-compose.yml#L23)_) or you can use my docker image in [pramodhayyappan/kk-task-two-container-handler](https://hub.docker.com/repository/docker/pramodhayyappan/kk-task-two-container-handler) in dockerhub
```bash
# To build image with custom name and tag. By default, it uses the latest tag
docker build -f task-two/Dockerfile task-two -t pramodhayyappan/kk-task-two-container-handler:
```
#### Cloud Infra deployment
- Assuming that you have already set up your AWS Credentials in your local machine, create a Security Group(SG) to allow communication between nodes and make note of the SG name
```bash
aws ec2 create-security-group --group-name swarm-cluster-sg --description "swarm cluster security group" --vpc-id
```
- Create ingress rules with the protocols and ports mentioned in the [offical doc](https://docs.docker.com/engine/swarm/swarm-tutorial/#open-protocols-and-ports-between-the-hosts)
```bash
aws ec2 authorize-security-group-ingress --group-id --protocol tcp --port 22 --cidr
aws ec2 authorize-security-group-ingress --group-id --protocol tcp --port 2377 --cidr
aws ec2 authorize-security-group-ingress --group-id --protocol tcp --port 7946 --cidr
aws ec2 authorize-security-group-ingress --group-id --protocol udp --port 7946 --cidr
aws ec2 authorize-security-group-ingress --group-id --protocol udp --port 4789 --cidr
```
- Create `user-data.sh` script file with below content
```bash
#!/bin/bash
sudo apt-get update
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release -y
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin git -y
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
```
- Create two EC2 instances for Docker Swarm cluster
```bash
aws ec2 run-instances --image-id --count 1 --instance-type t2.xlarge --key-name --security-group-ids --subnet-id --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=ManagerNode}]' --user-data file://user-data.sh
aws ec2 run-instances --image-id --count 1 --instance-type t2.xlarge --key-name --security-group-ids --subnet-id --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=WorkerNode1}]' --user-data file://user-data.sh
```
- The infrastructure for docker swarm cluster with docker installed is ready
### Deploy docker stack
#### Stack deployment
- Create a swarm cluster by executing the below command in manager node
```bash
docker swarm init --advertise-addr
```
- Join worker node to the cluster by using the output of the init command or use `join-token` to generate the join command. The command will look like
```bash
docker swarm join-token worker
docker swarm join --token :
```
- Clone this repo to the manager node
```bash
git clone https://github.com/pa/kodekloud-assignment.git
cd kodekloud-assignment
```
- Deploy docker stack
```bash
docker stack deploy --compose-file docker-compose.yml
```
- Some useful commands to list stack, services, container and inspect container
```bash
# To list docker stacks
docker stack ls
# To list services deployed by stack
docker stack services
# To list docker containers
docker ps
# to inspect docker container
docker inspect
# runs a new command on a running container
docker exec -it
# to remove a docker stack
docker stack rm
```
#### Demo
Demonstration and Testing of docker stack deployment
[](https://asciinema.org/a/500320)