As software development becomes more complex and fast-paced, it is crucial to have an efficient and automated process for building, testing, and deploying code changes to production environments. Continuous Integration and Continuous Deployment (CI/CD) pipelines are essential tools for achieving this goal. In this article, we will walk through the process of setting up a CI/CD pipeline for a Django application using Jenkins and Digital Ocean, with Slack integration for notifications related to deployments.
Youtube Video link: https://youtu.be/pXNB8uAGlx8
Github repo: https://github.com/rashiddaha/blogprojectdrf
You can Watch Video as well.
Why is CI/CD Important?
CI/CD is important because it helps teams to develop and deliver software changes more quickly, reliably, and with fewer errors.
Speed and Efficiency: By automating the build, test, and deployment process, teams can reduce the time and effort required to release new features and bug fixes.
Reliability: By automating the testing process, teams can catch and fix errors early, before they become bigger problems. This helps to ensure that the software is of high quality and works as expected.
Collaboration: By integrating code changes into a shared repository, teams can work together more easily and efficiently. This can help to reduce the risk of conflicts and improve the overall quality of the code.
Continuous Feedback: By continuously testing and deploying software changes, teams can receive feedback more quickly. This allows them to make adjustments and improve the software more rapidly.
Better Security : By continuously testing and deploying software changes, teams can catch security vulnerabilities early and fix them quickly, which can help to protect the software and the users from potential threats.
Overall, CI/CD helps to streamline the software development process, improve the quality of the software, and reduce the risk of errors, which ultimately helps organizations to be more competitive in the market.
Building a CI/CD django pipeline in AWS
In this section, we are going over how to set up a CI/CD pipeline in AWS for a django project.
To follow up with this tutorial, you will need to meet up with the prerequisites:
An AWS account
I have a Githubrepository given at the start of article you can use as a guide for this tutorial. Without wasting more of your time let’s get started.
Create a New IAM Role
Log in to your AWS account, the first thing we have to create is an IAM role, this role will help assign some permissions to some services such as CodeDeploy and S3 access for our EC2 which we will need in the future.
In the search bar, type in IAM, and click on the first option.
On the IAM home page, look at the left pane, click on Roles
On the Roles home page, click on Create Role.
Now, we should be on the Create Role page, on the Trusted entity type, select AWS service.
For the use case option, select EC2.
Click on the Next button to go to the next page.
Here we need to add the permissions for our Role, search for codedeploy, and select the AmazonEC2RoleForAWSCodeDeployLimited permission.
Click on Next to move to the final page, here we will name our new IAM role, and leave every other option as default.
Create an EC2 instance
The next thing we need to do is create our ec2 instance.
It is important to use the Ubuntu 20.04 or Ubuntu 20.10 image when spinning up our instance.
This is because they are part of the image types that currently support the codedeploy-agent package
The codedeploy-agent will help automate the deployment configurations on our server.
In the search bar, search for ec2, click on the first option.
On the ec2 home page, on the left pane, click on EC2 Dashboard, while in the dashboard, click on the Launch Instance button.
Now in the launch instance configurations page, give your instance a name. In our case, we are naming it django_server.
In the Application and Os images, make sure to select Ubuntu server 20.04 Free tier option
Leave the default configurations on the Instance Type option. On the Key pair (login) option, make sure to create a new .pem key. We will use this key to log into our ubuntu later on.
In the Network settings option check the Allow HTTP traffic from the internet option
Next, click on the Launch instance button. Our instance will be provisioned immediately.
Now we will need to attach our newly created IAM role to our instance. On the instance home page, we will select our instance and click on the Actions button in the top right corner.
Select the Security option, and click on Modify IAM Role. Then, select the ec2-codedeploy-s3 role from the dropdown.
Finally, click on the Update IAM role button.
Next, we need to reboot our instance for the changes to take effect.
Installing Codedeploy-Agent on Our Server
With our instance provisioned, we will now install the codedeploy-agent on our server.
First, we need to log into our server. There are guides on how to do this on the Connect page, click on the Connect button in the top right corner of the instance page.
When you have logged into your server, run this command to update the repositories.
sudo apt update
Next, we need to install the ruby-full package:
sudo apt install ruby-full
‍```
Finally, we need to install the wget package:
```ts
sudo apt install wget‍
After installing the necessary packages, we then need to change our directory to /home/ubuntu like so:
cd /home/ubuntu
Now in the ubuntu directory, we need to enter this command:
wget https://bucket-name.s3.region-identifier.amazonaws.com/latest/install
In this command, the bucket-name is the s3 bucket that contains the CodeDeploy installation files for your region. The region-identifier is simply your region.
For example, for the US-East-1 region (N.Virginia), replace bucket-name with aws-codedeploy-us-east-1, and region-identifier with us-east-1.
You can get a list of CodeDeploy bucket-names and region-identifiers here.
Next up, we need to change the permission on the install file we will get after running the command above.
chmod +x ./install
Finally, to install the codedeploy-agent, run this command:
sudo ./install auto > /tmp/logfile
Here we are logging the output of the installation to the /tmp/logfile file. To check if the codedeploy-agent is running, enter this command:
sudo service codedeploy-agent status
If it is not running, enter this command to start the codedeploy-agent service:
sudo service codedeploy-agent status
Setting up Configuration Files
Now that we have configured our server we will need to start working on our configuration files.
There are two important configuration files we need which are:
buildspec.yml
appspec.yml
The buildspec.yml file holds the configurations for how our code is going to be built in AWS CodeBuild, while the appspec.yml file holds the configurations for how our code is going to be deployed in AWS CodeDeploy.
I will not be going into detail on how each configuration step works, but we need to make sure that these files are present in our GitHub repository, in our working root directory. Let’s create our buildspec.yml file.
In our root working directory, create a file called buildspec.yml and add this block of code to it.
version: 0.1
environment_variables:
plaintext:
DJANGO_SETTINGS_MODULE: config.settings.test
SECRET_KEY: nosecret
DATABASE_DEFAULT_URL: sqlite:///db1.sqlite3
DATABASE_STREAMDATA_URL: sqlite:///db2.sqlite3
OPBEAT_ENABLED: False
phases:
pre_build:
commands:
- echo Prebuild ops
- pip3 install -r requirements.txt
build:
commands:
- echo Building the application
- python manage.py runserver
post_build:
commands:
- echo Build completed on `date`
In our buildspec.yml file, we can see that our build consists of 3 phases, the pre_build, build, and post_builds. We define the processes we want to run within these three phases.
Next, we need to create our appspec.yml file.
After creating our appspec.yml file, we need to add this block of code to it.
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/blogprojectdrf
permissions:
- object: /home/ubuntu/blogprojectdrf
owner: ubuntu
group: ubuntu
hooks:
BeforeInstall:
- location: scripts/clean_instance.sh
timeout: 300
runas: ubuntu
AfterInstall:
- location: scripts/instance_os_dependencies.sh
timeout: 300
runas: ubuntu
- location: scripts/python_dependencies.sh
timeout: 300
runas: ubuntu
- location: scripts/gunicorn.sh
timeout: 300
runas: ubuntu
- location: scripts/nginx.sh
timeout: 300
runas: ubuntu
ApplicationStop:
- location: scripts/stop_app.sh
timeout: 300
runas: ubuntu
ApplicationStart:
- location: scripts/start_app.sh
timeout: 300
runas: ubuntu
The appspec.yml file executes the different deployment stages within its hooks, these hooks are essential because they contain paths to configuration files that CodeDeploy uses to automate the deployment of our application.
Creating our CodePipeline
In our AWS management console, in the search bar, search and select for CodePipeline.
You should now be on the CodePipeline home page, look at the left pane, you will see the steps outlined for a ci/cd pipeline.
Now, let’s create our first build. On the left pane of the CodePipeline page, click on build.
We should now be moved to the CodeBuild page where we can create our first build.
Here you can name your build, and give it a description. I named mine django_code_build.
Next, we need to specify the source of our code, i.e. where our code will be pulled from for building. In this section, make sure to choose GitHub.
We will need to connect to our Github account first, after that we can choose which repository we want to pick from with its branch.
Next, is the ENVIRONMENT configuration. We need to describe the environment in which are builds and tests are going to be carried out.
Select the options that are selected in the image below.
Next, for the service role, select the New Service Role option.
For the Buildspec section select the use a buildspec file option. This option makes the build process select the buildspec.yml file in our root working directory
If you have your buildspec.yml file within another directory, specify the path of the buildspec.yml file in the field shown.
Next, select the option for No Artifacts in the artifacts section.
Finally, for the logs, if you want to see and store the logs for your builds, select the option to store those logs in CloudWatch, and give the CloudWatch group and stream names
Now we are done with the configuration of our build in CloudBuild, click on the Create build project button.
When we check our build projects, we should see our newly created build project.
Next, we will need to create a CodeDeploy application. This application will handle the CD aspect of our CI/CD pipeline.
On the left pane, click on Deploy and click on Getting Started. We should see the page for creating our first CodeDeploy application.
On the Create application page add your application name and select the EC2/Premise option for the Compute platform.
After creating our deployment application, we will be navigated to the deployment group page, where we will create a new deployment group for our deployment application.
Finally. let us create our CodePipeline.
On the left pane, click on Pipelines, and click on the Create pipeline button.
On the Choose pipelines settings give your pipeline a name, snd leave everything else as default, and click Next.
Next, we need to add the source of our code, we will be selecting Github (Version 1) in this step.
We will need to connect our Github account with AWS CodePipeline.
After connecting our account, we will need to choose the repository and branch that will be used.
Next, we need to specificy our build, we will be using CodeBuild here, select the build project we created above. Leave all other options as default.
Next, we need to add our deployment stage, we will be using CodeDeploy for our deployment stage, select the application name and deployment group we created above.
Afterwards, click on Next, and then click on Create pipeline.
Creating our pipeline automatically triggers a new release, this will clone our repository, build and run any specified tests, then deploy our application to our server.
We are Done. The project is Live now
You can Folllow me at:
Note: Thank You for reading!
Note: For Quries and for projects work you cna react out to me at codewithmuh@gmail.com