Photo by Glenn Carstens-Peters on Unsplash
How to Deploy CI/CD pipeline on AWS ECS using GitLab runner
In this blog I have explained how to Develop a simple CI/CD pipeline which builds a docker container and runs it on AWS ECS using GitLab runner.
Gitlab repository- gitlab.com/akabhishek4444/flaskappdeploy
In this project I have completed the following stages:
Stage 1: Dockerization
Dockerize a simple Hello World Flask Application which responds with a message that is set up as an environment variable and deploy the docker image to AWS ECR.
Stage 2: CI/CD Pipeline
Develop a CI/CD pipeline(Github Actions/Jenkins or your own choice) which would lint the python code and push the updated image to ECR.
Stage 3: Deployment to ECS
Setup a workflow which would deploy the given ECR image to ECS on EC2 instance or Fargate.
and also BONUS:
ECS could be configured for autoscaling and/or load-balancing.
CI/CD WORKFLOW
Step1.
Setup virtual python environment
Step2.
Create and Run virtual Flask Application
Step3.
Building the Dockerfile and running the container
Step4.
Pushing the Docker image to AWS ECR (elastic container registry)
Step5.
Create a New repository on GitLab and push the project code into that
Step6.
Setup local Gitlab runner
Step7.
Create a new AWS cluster and link the image to Dockerhub
Applications and Services used in the project
Docker
Docker is a programme that uses containers to make it easier to construct, deploy, and operate applications. Containers enable a developer to bundle a programme with all of its components, including libraries and other dependencies, and deliver it as a single package. The developer may feel comfortable that the application will work on any other Linux machine, independent of any specific settings the machine may have that differ from the machine used for authoring and testing the code, owing to the container.
Flask
Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. It has no database abstraction layer, form validation, or any other components where pre-existing third-party libraries provide common functions.
GitLab
GitLab is a web-based Git repository that provides free open and private repositories, issue-following capabilities, and wikis. It is a complete DevOps platform that enables professionals to perform all the tasks in a project—from project planning and source code management to monitoring and security.
Gitlab-runner
GitLab runner is a build instance which is used to run the jobs over multiple machines and send the results to GitLab and which can be placed on separate users, servers, and local machines. You can register the runner as shared or specific after installing it.
AWS ECR(Elastic Container Registry)
Amazon ECR is a fully managed container registry offering high-performance hosting, so you can reliably deploy application images and artefacts anywhere.
AWS ECS Fargate (Elastic Container Services)
Amazon ECS is a fully managed container orchestration service that makes it easy for you to deploy, manage, and scale containerized applications.
AWS Fargate is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. AWS Fargate is compatible with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS).
Stage 1: Dockerization
I have to build a docker image of the flask application ready to deploy complete with all libraries and dependencies using Dockerfile.
I have provided an environment variable in our image which can take a message and display it on our website. Environment variable makes it easy to change our message with each container without changing the base image every time.
Then we uploaded our image to our registry in AWS ECR to make a backup of it on the cloud. AWS ECR provides us with a URI which can be used by anyone around the world to access our image easily.
Our website with default Value of environment variable:
Our Website with environment variable value changed with docker run command
Stage 2: CI/CD Pipeline
For our CI/CD pipeline, we have used Gitlab to automatically build a new image and push the same in AWS ECR automatically.
We have authenticated with our registry and then uploaded the image on AWS ECR
We Have renamed our image and pushed it to AWS ECR.
Gitlab runner has automatically up-updated the docker image on AWS ECR with the latest image.
Stage 3: Deployment to ECS
We have used GitLab-runner to set up a workflow which would deploy the given ECR image to ECS on Fargate.
Our ECS cluster running on AWS ECS
Final Website on Cloud
Original Image
BONUS
I have set up a Load balancer and autoscaling on our website to ensure zero downtime on our website due to increased traffic.
Script
This file has updated the value of the environment variable and setup the flask in our container
what I learned from this project
I learned about various new applications and services and how to use them such as AWS ECR, AWS ECS and Flask. I learned how to deploy CI/CD workflow on AWS using GitLab-runner. I also learned how to update the image on ECR and deploy the same on ECS Fargate. I also learned the benefit of auto-scaling and load balancer to manage the traffic on the website. It was a good experience and I learned a lot from facing many difficulties and problems while making this project.