How to Deploy CI/CD pipeline on AWS ECS using GitLab runner

In this blog I have explained how to Develop a simple CI/CD pipeline which builds a docker container and runs it on AWS ECS using GitLab runner.

Gitlab repository- gitlab.com/akabhishek4444/flaskappdeploy

In this project I have completed the following stages:

Stage 1: Dockerization

Dockerize a simple Hello World Flask Application which responds with a message that is set up as an environment variable and deploy the docker image to AWS ECR.

Stage 2: CI/CD Pipeline

Develop a CI/CD pipeline(Github Actions/Jenkins or your own choice) which would lint the python code and push the updated image to ECR.

Stage 3: Deployment to ECS

Setup a workflow which would deploy the given ECR image to ECS on EC2 instance or Fargate.

and also BONUS:

ECS could be configured for autoscaling and/or load-balancing.

CI/CD WORKFLOW

workflow.png

Step1.

Setup virtual python environment

Step2.

Create and Run virtual Flask Application

Step3.

Building the Dockerfile and running the container

Step4.

Pushing the Docker image to AWS ECR (elastic container registry)

Step5.

Create a New repository on GitLab and push the project code into that

Step6.

Setup local Gitlab runner

Step7.

Create a new AWS cluster and link the image to Dockerhub

Applications and Services used in the project

Docker

Git

Docker is a programme that uses containers to make it easier to construct, deploy, and operate applications. Containers enable a developer to bundle a programme with all of its components, including libraries and other dependencies, and deliver it as a single package. The developer may feel comfortable that the application will work on any other Linux machine, independent of any specific settings the machine may have that differ from the machine used for authoring and testing the code, owing to the container.

Flask

Git

Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. It has no database abstraction layer, form validation, or any other components where pre-existing third-party libraries provide common functions.

GitLab

download.png

GitLab is a web-based Git repository that provides free open and private repositories, issue-following capabilities, and wikis. It is a complete DevOps platform that enables professionals to perform all the tasks in a project—from project planning and source code management to monitoring and security.

Gitlab-runner

Git

GitLab runner is a build instance which is used to run the jobs over multiple machines and send the results to GitLab and which can be placed on separate users, servers, and local machines. You can register the runner as shared or specific after installing it.

AWS ECR(Elastic Container Registry)

Git

Amazon ECR is a fully managed container registry offering high-performance hosting, so you can reliably deploy application images and artefacts anywhere.

AWS ECS Fargate (Elastic Container Services)

Git

Amazon ECS is a fully managed container orchestration service that makes it easy for you to deploy, manage, and scale containerized applications.

AWS Fargate is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. AWS Fargate is compatible with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS).

Stage 1: Dockerization

dockerfile.png

step1.1.png I have to build a docker image of the flask application ready to deploy complete with all libraries and dependencies using Dockerfile.

step1.4.png

I have provided an environment variable in our image which can take a message and display it on our website. Environment variable makes it easy to change our message with each container without changing the base image every time.

Capture.png

Then we uploaded our image to our registry in AWS ECR to make a backup of it on the cloud. AWS ECR provides us with a URI which can be used by anyone around the world to access our image easily.

Our website with default Value of environment variable:

step1.3.png

Our Website with environment variable value changed with docker run command

step1.5.png

Stage 2: CI/CD Pipeline

step2.png

For our CI/CD pipeline, we have used Gitlab to automatically build a new image and push the same in AWS ECR automatically.

step2.png We have authenticated with our registry and then uploaded the image on AWS ECR

step1.9.png

We Have renamed our image and pushed it to AWS ECR.

1.png

step1.10.png

Gitlab runner has automatically up-updated the docker image on AWS ECR with the latest image.

Stage 3: Deployment to ECS

image.png

image.png

image.png

We have used GitLab-runner to set up a workflow which would deploy the given ECR image to ECS on Fargate.

image.png

image.png

Our ECS cluster running on AWS ECS

Final Website on Cloud

image.png

Original Image

image.png

BONUS

image.png

image.png

I have set up a Load balancer and autoscaling on our website to ensure zero downtime on our website due to increased traffic.

Script

image.png

This file has updated the value of the environment variable and setup the flask in our container

what I learned from this project

I learned about various new applications and services and how to use them such as AWS ECR, AWS ECS and Flask. I learned how to deploy CI/CD workflow on AWS using GitLab-runner. I also learned how to update the image on ECR and deploy the same on ECS Fargate. I also learned the benefit of auto-scaling and load balancer to manage the traffic on the website. It was a good experience and I learned a lot from facing many difficulties and problems while making this project.

Repository on Gitlab

link =gitlab.com/akabhishek4444/flaskappdeploy