serverless aws project

Today we will be Building a cutting-edge Serverless Task Master App with AWS and GitHub integration. Harness the power of AWS Lambda to create a robust, scalable, and serverless task management application. Our step-by-step guide will walk you through setting up AWS services and integrating Lambda functions for smooth task handling. Implement CI/CD using GitHub for seamless updates and automated deployments. Unlock the potential of serverless architecture and modern CI/CD practices to build a secure, efficient, and cloud-native app on AWS.

First, Open the Serverless Framework web page and sign up or register yourself and create a Serverless Account

Once you have created an account it will look like this and as you can see our User-name is shown above Apps which is "amitkr" - our "Username" is important, as we have to add the username in our code to build serverless infrastructure

Next, log in to the AWS console and launch the EC2 instance (VM), we only need the Instance to clone the code from GitHub and also run commands for deployment, but as this is a serverless project we won't be needing the EC2 instance later on as we go for CI/CD automation.

Connect to your EC2 instance

Next, before we install a serverless framework, we will have to install node.js on our EC2 server.

When installing Serverless Framework on your server, you need Node.js as a prerequisite because the Serverless Framework itself is built on top of Node.js.

we can check if node.js is successfully installed by running command to check the version

Next, we will install serverless on our server.

we can copy the command from the serverless framework web page - Documentation - Setup

Add sudo before the command and run the command to install

We also need to install AWS-CLI to interact with various AWS services directly from the command line

Create a folder named "Projects" and move into the folder, inside which we will build our project

Now, we will run the serverless command - serverless --org=username .

The username will be your serverless username and This command informs the Serverless Framework that you are working within the context of the "amitkr" organization, and any serverless operations or deployments will be associated with that organization

As soon as you run the command you will get several options to choose from and for now, we will select the Node.js API project option, just select using arrow key

As we select our Node.js API, it will be asked to name the project so in our case we named our project - "First-serverless-project"

As you gave the name and entered, serverless will give you a link to login into the Serverless framework through CLI, just copy the link and open it in a new tab, and you will get logged in automatically.

Next, Serverless will ask for AWS credentials you can choose "local access key" or "skip"

we will opt for the AWS Access key and press enter and again enter yes for confirmation. Next, we need to create an IAM user with the necessary policies attach and get the access key

Go to your AWS Console and go to IAM and select Create new user, after that give it a name and next

For Policies, we will just give it "Administrator Access"

Next, select the newly created User and go to security and select Create an access key

select CLI option

you will get the Access key and secret access key.

Note: Do note these keys are very important make sure not to share them with others, for safe side download the access key file

Now, enter the Access Key and the Secret access key

And as soon we entered the key and also enter "Yes" to confirm to Deploy and now we can see our project is deploying

And Also Notice it shows updating Cloudformation stack which means our project will be built into a cloud formation stack we will check it later on

After the Deployment is successful we can see it gave us the endpoint URL to access the API

On our server in the folder projects that we created earlier, our serverless project is created there, now go inside our first serverless project to see the type of file in there and as we see there are the - readme, index.js, and serverless.yml files

If we go to our AWS console and go to cloud formation we can see our first-serverless-project stack is successfully built

Select the Stack and we can see various information such as resources, here we can see how many resources are used to build our project

To view the design - Select "update stack" - select "edit template in design" and below you will get the option to "view in design" Select it

With the help to design we can easily see the number of resources created

And we can check our S3 bucket here a bucket is created for project, bucket helps in-store the file along with different versions of it and help to manage state file

Next, let us see the Severless.yml file

The serverless.yml file is a configuration file used in the Serverless Framework. It is a crucial component of defining and deploying serverless applications.

And this is the index.js file

let's edit the index.js file and custom our message

And deploy the project again using the command - serverless deploy

Copy the endpoint URL and open it in a new tab

here we can access our API and also can see our message on top

let's edit and remove the input to clearly see our message

As we make changes to our file we again deploy our project using the command - serverless deploy

Now that we removed the input, we are able to see the message clearly

And if we check our S3 bucket and go inside our project folder we will be able to see the different versions that we deployed

Now let's deploy a taskmaster app from GitHub

Code credit to Shubham londhe: https://github.com/LondheShubham153/serverless-taskmaster-aws-app

First, we clone the code in our server using git clone and paste the git repo URL

Go inside the serverless-taskmaster folder and we will see various files including the serverless.yml file

For the project, we will remove the serverless file and create a new serverless file, it is necessary to create a new serverless file of course to learn and also it needs a few changes let's see

org: amitkr
app: serverless-taskmaster-aws-app
service: serverless-taskmaster-app

provider:
  name: aws
  region: us-east-1
  runtime: nodejs14.x
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:*
      Resource:
        - arn:aws:dynamodb:us-east-1:381035501714:table/KaamKaro




resources:
  Resources:
    KaamKaro:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: KaamKaro
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:

As we see in the above serverless.yml file the first syntax is the "org : amitkr" which is the serverless username that we have to mention in the file, make sure you do the same

Now, here we are creating a Dynamodb table so that we can get the arn (Amazon Resource Name)

just create a table with any name

Go to the table and copy the arn

Now, copy the arn in the resources section and table name here, we gave the name "Kaamkaro" and make sure your serverless.yml file table name matches with the sources code table name

Now everything is good to go, we go to our serverless framework page and in apps select "create app"

Here select "existing project"

Copy the command to run on our server

As we run the command, it shows us an Error that we have some indentation error in our serverless.yml file

Once the error is fixed, again run the command and it gave us the option to select = As our previous build of the first serverless project OR "create new app"

we select the Create New App

Now give your project A name and press enter

And select "Yes" to confirm, And our taskmaster app is successfully deployed

We can see on our serverless page our new taskmaster app is created

Now As same as before we can check our AWS cloud formation stack can see our new taskmaster project stack

we can view the design - as we can see not many resources are created for now because we have not yet added our Sources code in serverless.yml file

A new table is created named "KaamKaro"

Now, let's add our source code in the serverless.yml file, And you might ask what is sources code and where are these codes.

It is the set of statements and commands that make up a computer program. Programmers write the source code to create software applications, websites, and other computer programs.

And These codes are located in the folder named "src" in the GitHub taskmaster code that we cloned in our servers, you can check

First, we add the "hello" sources code under "functions" Now this function command is for AWS Lambda Function

org: amitkr
app: serverless-taskmaster-aws-app
service: serverless-taskmaster-app

provider:
  name: aws
  region: us-east-1
  runtime: nodejs14.x
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:*
      Resource:
        - arn:aws:dynamodb:us-east-1:381035501714:table/KaamKaro

functions:
  hello:
    handler: src/hello.handler
    events:
      - httpApi:
          path: /
          method: get


resources:
  Resources:
    KaamKaro:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: KaamKaro

And we save the changes and deploy the project and copy the endpoint URL

Open it in a new tab, here we can see the message from the "hello" source code

Again we check our project design and now we see a bunch of other resources that is created

And a new S3 bucket is created named - serverless-taskmaster

And At last, we add other 3 sources code in our serverless.yml file and save the file

org: amitkr
app: serverless-taskmaster-aws-app
service: serverless-taskmaster-app

provider:
  name: aws
  region: us-east-1
  runtime: nodejs14.x
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:*
      Resource:
        - arn:aws:dynamodb:us-east-1:381035501714:table/KaamKaro

functions:
  hello:
    handler: src/hello.handler
    events:
      - httpApi:
          path: /
          method: get
  kaamBharo:
    handler: src/kaamBharo.handler
    events:
      - httpApi:
          path: /kaam
          method: post
  kaamDikhao:
    handler: src/kaamDikhao.handler
    events:
      - httpApi:
          path: /kaam
          method: get
  kaamKhatamKaro:
    handler: src/kaamKhatamKaro.handler
    events:
      - httpApi:
          path: /kaam/{id}
          method: put


resources:
  Resources:
    KaamKaro:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: KaamKaro
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH

And finally, deploy the serverless.yml file using the command serverless deploy

Here we can see we got 4 endpoint URLs because we added 4 sources code and also 4 lambda function

Now At last, we can see our final build design and here we can see various resources is created

A cloud watch is also created to keep logs

Now, we can check our endpoints in - Postman

Here we check the endpoint for the "Get" kaam endpoint but, it's showing us "Internal server error"

We check the cloud watch logs

An error was shown because we didn't install the package file from the code, As we can see there are package files in the taskmaster project folder. Next, we run the command "npm install" to install the packages

After installing the package we again deploy our taskmaster project

And now if we check the Kaam "Get" URL it shows no errors

here we are using the "Post" function to add messages and we get an id - which we will need to update the change

here we use the "Put" function to update the change we did in the last "Post", here with the URL add the id that was created in "Post" We can see after we added the "completed = true" syntax it's showing the job is being completed - kaam khatam kar diya

Now = {{dev_url}} is the endpoint URL we saved in the Postman environment because the URL is the same for all 4 APIs otherwise you can copy and paste the URL there is no problem

Now,if we again select the Kaam "Get" API we can see the change is updated

ls -a the command will show the hidden folder and we can see the serverless state file

Now, in order to monitor the log with serverless integrated monitoring, we just need to run the command serverless dev OR sls dev

"Serverless" and "sls" both can be used in place of each other and "dev" is the environment in which our project is created, we can check the serverless page there we will dev environment

Copy the URL and open in a new tab just to log in - after accessing url we might be asked for login just type mail and password don't create a stack if asked go back to the server you will this prompt of login and enter yes and it will take 5 to 10 minutes to build

After the build is complete copy the URL and open it in a new tab

You will be able the monitor the logs here

Now we will Integrate Github and AWS for CI/CD and automate the deployment

Next, we go to the serverless page and go to apps and on the right-side panel select settings

Go to CI/CD and the page will open to add GitHub and AWS

Here connect your GitHub and refresh this serverless page and select the code repository and please do note to make sure you push this code to your own GitHub account in order to have full access to the code while committing changes and triggering the build

Scroll down and select the deployment region and here we can see that serverless will build a 2GB server to deploy the project and Select the GitHub branch "main" and environment which is "dev", choose as per your requirements

Next, we connect to AWS = provide the IAM user Access key that we created earlier

AWS provider is successfully created

Here we can see both our GitHub and AWS are connected.

Now we do not need the EC2 server to run the deployment command as the build will trigger with any change committed in the code

We are in Serverless-Taskmaster-app

go to the app - select the project - select deploy, here we can see our last build was 1 hour ago

We make a simple change and commit the change to trigger the build

Go to ci/cd - Here we can see the build is deployed as we commit the change

we can check the deployment in progress in the app - deploy section, here we can see a Server is running for deployment

The deployment failed why?

The Error shows that the "KaamKaro" Dynamaodb table already exists and now to deploy the project successfully we only need to change the table name in all the sources code including serverless.yml and push those changes to GitHub and the deployment is start with a new table

The Good thing here is that our Deployment worked we just need to change the table name, I hope till this stage you got what we need to do.

Any change we commit will trigger the deployment of the new version of our App

Now in order to deploy the app in new or different environments we need to create a new branch, here I created a "production" branch

And run the command sls deploy --stage=production to deploy the project in a new production environment.

Now it is sowing the same error as earlier that table "KaamKaro" already exists, So I guess you know what you have to do, that right we need to change the table name in the sources code for the production stage and deploy our app in the production environment.

So now you know how to build the app in a different environment

Here, we can see we have 2 serverless taskmaster apps the green one is which we manually build and the red one is the GitHub ci/cd build which failed

And at last to delete all the resources we created run the command sls remove , but make sure that you have deleted all the resources by checking in your AWS console.

Congratulation we have successfully created a serverless project, it was fun to learn and also challenging it took me several hours to build the project as I went through many syntax errors and indentation errors. but when you see that it is a success then all the hard work pays off.

Thank you for reading the blog I hope we all learn something new!!!!

"Building a Serverless Task Master App with AWS and GitHub Integration: A Guide to CI/CD Implementation"

Now we will Integrate Github and AWS for CI/CD and automate the deployment