Table of Content
Introduction
Setup simple Nodejs project
Setup AWS ec2 Instance
SetupSonarQube
Setup Ansible server
Setup Jenkins server
6.0) Required Installation
6.1) Setup Webhook
6.2) Setup DockerHub
6.3) Setup Sonarqube
6.4) Setup Ansible
Jenkinsfile
1) Introduction
--> CI/CD (Continuous Integration/Continuous Delivery) pipeline is a methodology used in software development and DevOps practices to automate the process of integrating code changes, testing them, and delivering the changes to production or other environments in a consistent and efficient manner.
--> Git is a distributed version control system, allowing developers to track changes in source code.
--> GitHub, a web-based platform built on Git, offers collaboration tools and repository hosting. It facilitates teamwork, code sharing, and project management, serving as a central hub for software development workflows.
--> A webhook is a mechanism that allows web applications to send real-time notifications or trigger actions in response to events occurring in another system or application.
--> Jenkins is an open-source automation server widely used for continuous integration and continuous delivery (CI/CD) pipelines. It automates various stages of software development, including building, testing, and deploying applications. Jenkins supports a wide range of plugins, enabling integration with various tools and technologies. It facilitates collaboration among development teams by providing a centralized platform for managing and executing automated workflows, improving efficiency and software quality.
--> SonarQube is a static code analysis tool that identifies bugs, vulnerabilities, and code smells in software projects. It offers comprehensive reports and metrics to improve code quality and maintainability. Integrating seamlessly into the development workflow, SonarQube assists in enhancing overall software reliability and facilitating efficient collaboration among development teams.
--> Ansible is an open-source automation tool used for configuration management, application deployment, and orchestration. It employs a simple YAML syntax to define tasks and workflows, enabling efficient management of infrastructure and applications at scale. Ansible's agentless architecture makes it easy to deploy and manage across diverse environments.
--> Docker is a platform that allows developers to package applications and their dependencies into lightweight containers, ensuring consistency across different environments.
--> DockerHub is a cloud-based registry service where users can store and share Docker images. It provides a vast repository of pre-built images for popular software stacks, facilitating rapid application deployment. DockerHub also enables collaboration by allowing developers to publish and distribute their Docker images publicly or privately.
--> Nginx is a high-performance web server and reverse proxy server known for its speed and efficiency. It is commonly used to serve static content, manage SSL/TLS encryption, load balance traffic, and act as a reverse proxy for web applications.
Work Flows
First of all , we will develop the simple Nodejs/Expess Js project with Dockerfile through Coding
Push the code through git to github
Then setup the github webhook for the jenkins server
Then jenkins server build the image from the Dockerfile and push to the DockerHub
Now then, Pass the code to the sonarqube server for proper analysis of the code quality, bug analysis , possible suggestion and so on .
Then it triggers the ansible playbook from the jenkins server and the playbook inside the ansible server run the playbook inside targeted ec2 instance.
The playbook build the image from the DockerHub and build the image and create the container and run the container.
We have to configure the nginx server inside the targeted machine for the app running inside the container port.
2) Setup simple Nodejs project
Github Repo :- https://github.com/SahadevDahit/cicd-jenkins-sonarqube-ansible-dockerHub.git
npm init
npm install express dotenv
index.js
// Import the express module
const express = require('express');
const app = express();
app.get('/', (req, res) => {
res.send('Hello, World! Welcome to the cicd pipeline'); // Send a response to the client
});
app.get('/products', (req, res) => {
// Assuming products are stored in an array called 'products'
const products = [{
id: 1,
name: 'Product 1',
price: 10
},
{
id: 2,
name: 'Product 2',
price: 20
},
{
id: 3,
name: 'Product 3',
price: 30
}
];
// Send the products as a JSON response to the client
res.json(products);
});
const port = 4000;
//this secret_key is testing for sonarqube
const secret_key = 'shgiuginoshviu7868jb';
// Start the server and listen on the specified port
app.listen(port, () => {
console.log(`Server is running on http://localhost:${port}`);
});
Create the Dockerfile
FROM node:20
WORKDIR /
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 4000
CMD ["node", "index.js"]
Push this project to the github
3) Setup AWS ec2 Instance
Here, there are four ec2 instances, for sonarqube, jenkins, anisble-master and ansible-target respectively. And the security group allows the port 8080 for jenkins, port 9000 for the sonarqube, port 80 for the nginx, port 4000 for web app.
Make sure that jenkins server have minimum 2CPU and 2GB RAM
4) Setup SonarQube
wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.9.4.87374.zip
sudo apt install unzip # Install unzip tool if not already installed
unzip sonarqube-9.9.4.87374.zip
create project manually
after sucessful run from the ansible server we get
5) Setup Ansible server
Here, one is ansible master server from where we trigger the targeted machine having nginx
setup for the targeted machine
# setup nodejs in targeted machine
curl -fsSL https://deb.nodesource.com/setup_current.x | sudo -E bash -
sudo apt install -y nodejs
#setup the nginx server in the targeted machine
sudo apt update
sudo apt install nginx
sudo systemctl start nginx
sudo systemctl enable nginx
sudo ufw allow 'Nginx Full'
cd etc/nginx/sites-enabled
sudo rm default
sudo nano myapp
Then paste the following code
server {
listen 80;
server_name example.com; # Change this to your domain name or public IP address
location / {
proxy_pass http://localhost:4000; # Assuming your Node.js app is running on port 3000
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
setup nginx server for the port 4000 as our web app is running in port 4000
# then
sudo service nginx restart
Setup the docker
# setup docker in the targeted machine
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce
sudo usermod -aG docker your_username
sudo visudo -f /etc/sudoers.d/docker
#In the file opened by the command above, add the following line:
%docker ALL=(ALL) NOPASSWD: ALL
setup for the ansible server
sudo apt install ansible
this is the key file to acess the targeted machine from the ansible server
then move into the etc and create the ansible folder and inside ansible create hosts and playbook with necessary permission
sudo nano hosts
GNU nano 6.2 hosts *
[servers]
3.111.40.69 ansible_ssh_private_key_file=/home/ubuntu/keys/ansible-key.pem
# ip of the targeted machine and file path of the pem file
04_node_from_dockerHub this ansible-playbook contains
---
- name: Pull latest Docker image from Docker Hub and run container
hosts: servers
become: yes # Enable sudo privileges
tasks:
- name: Ensure Docker container is stopped and removed if it exists
docker_container:
name: test_node # Specify the name of the container
state: absent # Ensure the container is absent
- name: Ensure Docker image is removed if it exists
docker_image:
name: dahitsahadev/test-node:latest # Specify the name of the image to check
force: yes
state: absent # Ensure the image is absent
- name: Pull latest Docker image from Docker Hub
docker_image:
name: dahitsahadev/test-node:latest # Specify the name of the image to pull from Docker Hub
source: pull # Indicate to pull the image from Docker Hub
- name: Run Docker container from the pulled image
docker_container:
name: test_node # Specify the name of the container to run
image: dahitsahadev/test-node:latest # Specify the name of the image to use for the container
state: started # Ensure that the container is started
ports:
- "4000:4000" # Specify the port mapping
The Ansible playbook pulls the latest Docker image dahitsahadev/test-node:latest from Docker Hub. It ensures that any existing container with the name test_node is stopped and removed. Then, it starts a new Docker container named test_node using the pulled image, mapping port 4000 on the host to port 4000 in the container. The playbook utilizes Ansible's docker_container and docker_image modules for Docker management.
6) Setup Jenkins server
6.0) Required Installation
install node js
curl -fsSL https://deb.nodesource.com/setup_current.x | sudo -E bash -
sudo apt install -y nodejs
install open jdk
sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version
openjdk version "17.0.8" 2023-07-18
OpenJDK Runtime Environment (build 17.0.8+7-Debian-1deb12u1)
OpenJDK 64-Bit Server VM (build 17.0.8+7-Debian-1deb12u1, mixed mode, sharing)
install jenkins
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
install docker with necessary permission
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce
sudo usermod -aG docker your_username
sudo visudo -f /etc/sudoers.d/docker
#In the file opened by the command above, add the following line:
%docker ALL=(ALL) NOPASSWD: ALL
sudo usermod -aG docker jenkins
sudo systemctl restart jenkins
sudo -u jenkins docker ps
install sonar-scanner
wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.6.2.2472-linux.zip
sudo apt install unzip # Install unzip tool if not already installed
unzip sonar-scanner-cli-*.zip
sudo mv sonar-scanner-* /opt/sonar-scanner
export PATH="/path/to/sonar-scanner/bin:$PATH"
source ~/.bashrc
sonar-scanner --version
put the ansible-key.pem file inside this directory in the jenkins server to ssh the ansible-master server
jenkins plugins
6.1) Setup Webhook
6.2) Setup DockerHub
create and sign in into your dockehub account and create one repository
link : docker push dahitsahadev/test-node:latest
6.3) Setup Sonarqube
6.4) Setup Ansible
put the ansible-key.pem inside this directory that will be used for the ssh from the jenkins server to the ansible-master server. You can copy from your local or you can create the file with necessary permission and copy the key contents.
stage('Run Ansible Playbook') {
steps {
script {
sh """
ssh -o StrictHostKeyChecking=no -i /var/lib/jenkins/workspace/test-node/ansible-key.pem ${ANSIBLE_SERVER_IP} 'cd /; cd /etc/ansible/playbook; ansible-playbook 04_node_from_dockerHub'
"""
}
}
}
7) JenkinsFile
pipeline {
agent any
environment {
DOCKER_IMAGE_NAME = "dahitsahadev/test-node"
DOCKER_IMAGE_TAG = "latest"
ANSIBLE_INVENTORY = "/etc/ansible/playbook"
SSH_PRIVATE_KEY = "/var/lib/jenkins/workspace/test-node/ansible-key.pem"
ANSIBLE_SERVER_IP = "ubuntu@ec2-65-0-5-146.ap-south-1.compute.amazonaws.com"
SONAR_HOST_URL = 'http://13.201.47.149:9000'
}
stages {
stage('Checkout') {
steps {
// Checkout the source code from your Git repository
git branch: 'main',
url: 'https://github.com/SahadevDahit/cicd-jenkins.git'
}
}
stage('Remove Existing Docker Image') {
steps {
// Remove the Docker image if it exists
sh "docker rmi ${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG} || true"
}
}
stage('Build Docker Image') {
steps {
script {
sh "docker build --rm -t ${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG} ."
}
}
}
stage('Push Docker Image to Registry') {
steps {
withCredentials([usernamePassword(credentialsId: 'docker-hub-credentials', usernameVariable: 'DOCKERHUB_USERNAME', passwordVariable: 'DOCKERHUB_PASSWORD')]) {
script {
sh "echo ${DOCKERHUB_PASSWORD} | docker login -u ${DOCKERHUB_USERNAME} --password-stdin"
sh "docker push ${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}"
}
}
}
}
stage('SonarQube Analysis') {
steps {
withSonarQubeEnv('sonar_qube') {
sh 'sonar-scanner --version'
sh "sonar-scanner \
-Dsonar.projectKey=test-node \
-Dsonar.sources=. \
-Dsonar.exclusions=node_modules/** \
-Dsonar.host.url=${SONAR_HOST_URL} \
-Dsonar.login=sqp_3f1c98433e3d99368028e1f5816dfc820f9869ef"
}
}
}
stage('Run Ansible Playbook') {
steps {
script {
sh """
ssh -o StrictHostKeyChecking=no -i /var/lib/jenkins/workspace/test-node/ansible-key.pem ${ANSIBLE_SERVER_IP} 'cd /; cd /etc/ansible/playbook; ansible-playbook 04_node_from_dockerHub'
"""
}
}
}
stage('Deploy') {
steps {
echo 'Deploying the server'
}
}
}
post {
success {
echo 'Build successful! Deploy your application.'
// Send notifications or perform additional actions on success
}
failure {
echo 'Build failed! Check the logs for errors.'
// Send notifications or perform additional actions on failure
}
}
}
This Jenkins pipeline automates the CI/CD process for a Node.js application:
Checkout: Clones the source code from the specified Git repository.
Build Docker Image: Builds a Docker image for the Node.js application using the Dockerfile in the repository.
Push Docker Image to Registry: Pushes the built Docker image to a Docker registry using provided credentials.
SonarQube Analysis: Performs a code analysis using SonarQube, specifying project key, sources, exclusions, SonarQube host URL, and login credentials.
Run Ansible Playbook: Executes an Ansible playbook on a remote server. It connects via SSH, disables strict host key checking, and runs the specified playbook.
Deploy: Placeholder stage for deploying the application (not implemented).
Post-Build Actions: Displays success or failure messages and performs additional actions based on the outcome of the pipeline.
This pipeline integrates source code management, Docker image building and pushing, code analysis with SonarQube, and Ansible-based deployment automation, providing a streamlined workflow for continuous integration and deployment of the Node.js application.
Thanks for reading .........................................