"Mastering Jenkins: A Step-by-Step Guide to Creating and Setting Up a Pipeline Project with GitHub Integration and Declarative Pipeline Environment"

1- Setting up Master and Agent (Worker) Server:

Pre-requisite: Do Install - Java, Jenkins, Docker, and Docker-compose in the Master server and Install - Java, Docker, and Docker-compose except for Jenkins in the Agent server because Jenkins will run through the Master server only.

Step-1: Create Two Instances of server - One for the 'Master' And the Second for the 'Agent or Worker' server

Step-2: On the 'Master' server Go to the .ssh location through cd .ssh And run the command ssh-keygen to generate an ssh key, Then copy the public key through cat rsa.pub . Here rsa is a file name.As shown in the Images below.

Step-3: Next go to the 'Agent or Worker' server - Go to the ssh directory cd .ssh and ls for the list, you will see authorized_keys file, Open the file with an editor vim authorized_key And once the editor opens - copy and paste the Master public key to the Agent authorized file. As shown in the images below

Step-4: To verify the connection - Go to the Master server and on the same ssh location run the command ssh username@IP-address here ssh ubuntu@ Ip address of the Agent server and ubuntu is the username. Here we can see we have established a connection that ensures the setup was a success and to exit to logout.

Step-5: Now, Open Jenkins and Start setting up an Agent from the Dashboard select the 'set up an Agent' option and give a name to the Agent and click on 'Create'.

Step-6: An node form will open to set up an agent - Give a Description

Give path location for the Agent server for the build workspace - we have given just the home path.

Now it's important to give labels because labels act as an identification for the agent and it's important to mention labels in the pipeline script. Here our's labels will be - Dev

Select the usage

Select the launch method as - SSH because we made servers connections through ssh.

Now copy the IP address of the Agent server from the AWS Instances dashboard

Paste the IP address in the Launch method in the Jenkins Node form

Now to add credentials - select Add and Jenkins

Step-7: Now, we set up credentials - In Kind select - SSH username with private key

Just give a simple ID and Description as per your choice And give the username of the Agent server which is Ubuntu in our Agent server

Now in a private key section Select - Enter directly and select Add

Go to your Master server and open the private key which is id_rsa with cat command and copy the whole private key

And paste the private key in the key section and select Add.

Step-8: Now, after saving credentials we are directed back to the Node form now in the credentials section select the credentials we just add.

And in the Host key verification strategy Select - Non verifying Verification Strategy

And Save

Step-9: Agent Node is created and go inside the Agent node

As we are inside the Agent node it's showing offline because we didn't install Java in the Agent server

Let's install Java and verify the version - the Java version should be the same as the Master server Java version - And why do we need to install Java because Jenkins works with Java only

Step-10: As we install Java in the Agent server - Now go to the Jenkins Agent node and select Launch Agent. As we see the logs it shows successfully connected

Go to the Nodes it's showing - In sync and free space and other information.

We successfully Create a Master and Agent server and set up the Agent in Jenkins And connected successfully. Now let's build some jobs

2- Create a job with Pipeline:

Step-1: On the Jenkins dashboard, 'Click Create a Job'

Give Job a Name and Choose - 'Pipeline' and select OK.

Configuration Page will open - First, give a description for the job and select GitHub project - As this will be a GitHub-integrated project using Webhook.

Copy the GitHub repository IP Address URL, not the code clone URL, and paste it into the configuration as the above Image.

Further, in the configuration in Build triggers select - Github hook for SCM polling

Now Go to the GitHub repository code and select settings and select webhook and in the payload URL - give the Jenkins IP/github-webhook/ we have to add GitHub-webhooks because that's the syntax for webhook. And at last add a webhook, a green tick will show it as active.

Step-2: Next we write a pipeline script in the configuration for - 1st- git clone, 2nd- docker Image build, 3rd- (Deploy) docker-compose down and up And save.

we can simply use docker-compose for the build, but we add git clone and image build for a purpose.

Step-4: Now if will install docker and docker-compose in the Agent server if we didn't already do so and also give the user docker permission because it will require permission And Reboot the server to make update the changes.

Step-5: Next, click on build now, And the build is a success with all 3 stages, - As we can see I have a few failed builds it was because of the syntax error in the pipeline script - so be attentive while writing the script.

Check the build on the Agent Server - ls for git clone list as - workspace folder and docker ps and docker images, both show the images, and the container is created

Add port 8000 in the Agent server as we build the container exposing port 8000.

Copy the Agent server IP address add port 8000 and open in the New window tab - The application is running successfully.

Step-6: Now we do a little experiment with GitHub Code - As we add GitHub webhook SCM polling, Let's see if it triggers any commit change or update in Code. For example- I just wrote 'Test' in the README file and commit the change.

As we can see any Change in code Auto-trigger the build. This means github webhook works.

3- Setup environment for Adding Credentials for Dockerhub and Make the Whole project fully Automate by Adding 'Jenkinsfile'- (Declarative):

Step-1: Go to Jenkins Dashboard and select Manage Jenkins and select Plugins.

On the Plugins - Search for - environment- and Select 'Environment Injector' and after that select- Download now and install after restart.

Opt for Restart

After installation, Jenkins will restart and login into Jenkins.

Step-2: Next, we will go to the job configuration and Add another stage for - the Dockerhub login and Image push, Now we just rite the echo syntax to test the script.

And build now - as we can see - the build is successfully being created. Our additional stage is good to go. Next, we add syntax for the docker hub login

Step-3: Next, we set up credentials for the docker hub username and password. So that we can add a username and password to the script through the credentials.

Again Go to - Dashboard - Manage Jenkins - Manage credentials

Select 'global' - which is in the Second Row.

Select Add Credentials

Give your - Dockerhub username and password and also, write ID and description as per your choice.

Now in the docker login and docker image push Stage - we add our script for docker hub credentials

And build now - the build is successful with the docker hub login.

Step-4: Next, is to log in to the docker hub account and we can see we have two docker images from before.

With the docker hub credentials add to the script, Now we add syntax for tagging the docker image and push it to our docker hub account in the - 2nd Stage for build by ading our dockerhub username

Next, Click on Build now with the new updated script - build seem's successfully executed with Docker Image push.

As we can see New Image has been pushed to dockerhub

Step-5: For next - we do some changes in the code - in docker-compose file - we change the 'build' syntax with the 'Image' syntax - So that now we can directly build the container through using docker image using docker-compose. That's why we earlier add the build syntax in the script to push Docker image.

Now again we do - build now to test the updated script - which executed successfully.

Step-6: Now the Interesting part - we create and add a 'Jenkinsfile' in the github code where we copy the whole pipeline script and paste it in the Jenkinsfile.

next in the job configuration in the pipeline section we opt for 'pipeline script from SCM' instead of 'pipeline script'.

next is to copy the code clone URL from github

paste the URL in the SCM repository URL

select branch as 'Master' same as github and script path as 'Jenkinsfiles' which come by default. And Save.

Step-7: Finally, We click on build now - and our fully automate declarative pipeline is executed successfully- without any code in the pipeline script. which executed through github integration and Jenkinsfile.

Our application is up and running.

This is the pipeline script for reference:

pipeline {
    agent {label 'Dev'}

    stages {
        stage('Code'){
            steps {
                git url: 'https://github.com/amitkmr076/node-todo-cicd.git', branch: 'master'
            }
        }
        stage('Build and Test'){
            steps {
                sh 'docker build . -t amitkmr076/node-todo-app:latest'
            }
        }
        stage('login and push Image'){
            steps {
                echo 'logging into Dockerhub and Push Image-'
                withCredentials([usernamePassword(credentialsId: 'dockerhub', passwordVariable: 'dockerhubPassword', usernameVariable: 'dockerhubUser')]) {
                    sh "docker login -u ${env.dockerhubUser} -p ${env.dockerhubPassword}"
                    sh "docker push amitkmr076/node-todo-app:latest"
                }
            }
        }
        stage('Deploy'){
            steps {
                sh 'docker-compose down && docker-compose up -d'
            }
        }
    }
}

My experience with the Jenkins declarative pipeline project was both challenging and rewarding. While setting up the pipeline, I encountered several issues and made mistakes along the way. However, these mistakes provided me with valuable learning opportunities and helped me to better understand the project.

Thank you for taking the time to read my blog! I hope you found it informative and engaging. If you have any feedback or suggestions, please feel free to reach out to me.