Serverless Web Application Architecture using React with Amplify: Part1

In this series we shall be building Jotter (a note taking app). Jotter is a serverless web application built with React and Amplify using AWS cloud services. The illustration below depicts the serverless architecture we are going to build.

After deployment:

Understanding Serverless:

Just put your code on cloud and run it. Without worrying about infrastructure.

Servers are still there in serverless, but managed by service provider. In our case, the service provider is AWS. Let us learn a little about AWS & its services. Feel free to skip, if you are acquainted with AWS know-how.

Understanding AWS:

Amazon Web Services is a cloud services platform that offers various cloud services. You could build reliable, scalable applications without worrying about managing infrastructure. Amazon Web Services offers a broad set of global cloud-based products including compute, storage, databases, analytics, networking, mobile, developer tools, management tools, IoT, security and enterprise applications. These services help organizations move faster, lower IT costs, and scale.

In a nutshell, AWS is like playing Lego or Minecraft. In lego, you use blocks to create different structures right? We neither create nor maintain those blocks. All we do is join & disjoin blocks logically to construct structures. Similarly, in AWS each service is like a block, we compose these services to develop our serverless application. You could also build a server based web application using AWS EC2 instance configured with AWS Route53, but that’s not the scope of this series.

Advantages:

1. Granular control over each block. Testing becomes easy, as if a block is malfunctioning, we only have to replace/fix that one block.
2. Reusability: Certain individual blocks or groups of blocks can be reused for other structures.
3. We only pay for the usage.
4. Scalable, high availability and fault tolerance.

That’s broadly about AWS as a platform. We shall first understand these services and then build the serverless web application architecture we viewed earlier.

Services used in the architecture:

1. DynamoDB (NoSQL Database)
2. Lambda (Server side computing)
3. API Gateway (HTTP calls interface)
4. Cognito (Authentication)
5. S3 (Cloud storage)
6. Amplify and React (Client side)

Understanding DynamoDB:

DynamoDB is a distributed NoSQL, schemaless, key-value storage system. Extremely scalable as the amount of data stored mainly depends on the physical memory of the system. While in DynamoDB you don’t have any such limits as you can scale the system horizontally. You will pay only for the resources you provision.

Though it is schemaless it is still represented as a table. Each table is a collection of items. Value(Attributes) of each item can be a scalar, JSON, set. Item size should be less than 400KB (binary, UTF-8). Each item in the table is uniquely identified with a Primary key and is mandatory while creating the table. Primary key can be the same as Partition key or a combination of Partition key and Sort key. If it is a combination of both it is also called Composite primary key.

Note: Primary key cannot be modified once created. Partition key and Sort key values are internally used as input for a hash function to determine storage.

More on indexes and DynamoDB, Parallel scan here.

Understanding Lambda:

Lambda is a serverless compute service. It lets you run code without provisioning or managing servers. You pay only for the compute time you consume. It supports NodeJS, Python, Java, GO etc. Lambda can be triggered from a variety of AWS services. Learn more about Lambda here.

To Every Lambda function handler, 3 objects can passed as argument. 

1. The first argument is the event object, which contains information from the invoker. The invoker passes this information as a JSON-formatted string when it invokes Lambda. When an AWS service invokes your function, the event object structure varies by service.
2. The second argument is the context object, which contains information about the invocation, function, and execution environment. In the preceding example, the function gets the name of the log stream from the context object and returns it to the invoker.
3. The third argument, callback, is a function that you can call in non-async functions to send a response to invoker. The callback function takes two arguments: an Error and a response. The response object must be compatible with JSON.stringify. Error should be null for successful response.

In our app, Lambda is used as a mediator for incoming HTTP requests & DynamoDB. Lambda writes, reads and processes data to/from DynamoDB accordingly.

Understanding API Gateway:

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. 

With a few clicks in the AWS Management Console, you can create REST and WebSocket APIs that act as a front door for applications to access data, business logic, or functionality from your backend services, such as workloads running on EC2, code running on Lambda, any web application, or real-time communication applications.

API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management.

In our app, we use API Gateway to invoke different Lambda functions for different API calls.

Understanding Cognito:

Amazon Cognito User Pool makes it easy for developers to add sign-up and sign-in functionality to web and mobile applications. It serves as your own identity provider to maintain a user directory. It supports user registration and sign-in, as well as provisioning identity tokens for signed-in users.

Our Jotter app needs to handle user accounts and authentication in a secure and reliable way. We are going to use Cognito User Pool for it.

Understanding Amplify:

AWS Amplify is a framework provided by AWS to develop applications, with AWS cloud services. Amplify makes the process of stitching cloud services with our application hassle free. Amplify provides different libraries for different apps(iOS, Android, Web, React Native). Amplify javascript library is available as an npm package(aws-amplify). The aws-amplify client library uses a config file to connect AWS services. The services which amplify provides include Database, API, Lambda/serverless, Authentication, Hosting, Storage, Analytics. 

Note: One could also use AWS Amplify CLI to provision AWS services. The aws-amplify client library and Amplify CLI are two different things.

Amplify CLI internally uses cloudformation to provision/create, while aws-amplify client library is used to connect to AWS services. Using Amplify CLI is inconvenient as you are not creating services directly from AWS console but by using a CloudFormation stack internally. If successful, it returns a config file with all the metadata of different services provisioned. Instead a simple approach is to create required services from AWS console and update the config file manually and use it with aws-amplify client library.

In our Jotter app, we will use aws-amplify client javascript library to interact with AWS services.

Building the serverless architecture

Let us first build this setup and then add the remaining.

Working with DynamoDB:

Create table:

Go to AWS console > DynamoDB > Tables, choose Create table. As we learned earlier each table is a collection of items and each item is identified with a primary key.

So, give the table a name, set the schema for primary key. Each jot is uniquely identified with the combination of userid (partition key) and jotid (sort key).

Click on Create.

This will create an empty table with 2 columns namely userid, jotid.

Adding sample data:

Each jot item is a json object with the following structure: 

{  
   "userid":"<32 digit uuid code>",
   "jotid":"<32 digit uuid code>",
   "title":"title content",
   "body":"body content"
}

Initially you will be shown only 2 keys userid, jotid as they are part of the primary key. We shall add the remaining 2 attributes (title, body). I have generated 5 random uuid codes for populating the table. We shall be adding 4 items, all of them belong to the same user which means they will have the same userid but different jotid. Click on Create item.

Similarly, add remaining 3 to the table.

We have successfully created a table and added 4 items to the table, now let us create Lambda functions to process data from DynamoDB.

Working with Lambda:

As we have learned earlier, Lambda lets us run code without provisioning or managing servers. We shall use Lambda for our server side computing or business logic in simple terms.

Creating a Lambda Function:
Let us first understand our server side operations that Jotter app needs.

• Read a list of all items of a user from DynamoDB table.
• Write a new item to DynamoDB table by a user.
• Read the full content of an item of a user.
• Delete an item of a user.

We shall create 4 Lambda functions that will do each of these jobs/operations.

Get all items Lambda:

Every Lambda has to be configured with an IAM role. This IAM role defines access control to other AWS services. We shall create an IAM role for Lambda to access DynamoDB.

Go to IAM > Roles. Choose create role. After role is created, we could attach policies or rules in simple terms. We could either attach default policies provided by AWS or create custom policies tailored to our specific needs.

Custom policy:
Choose service, and then operations. Create policy.

This way you could add custom policies to the role or use default AWS policies. I am giving AmazonDynamoDBFullAccess policy as i will be using this role for all my Lambda functions. A better approach would be creating different roles with custom policies for specific operations. Please do exercise all options. Do not give permissions more than needed.

Once role is created, go back to Lambda console, create the Lambda function with that role.

• Choose Author from scratch
• Give function a name(get-all-jots)
• Choose runtime environment(Nodejs 10x)
• Permissions, choose existing role and add the role created earlier.
• Create function.

After creation, replace index.js code with below:
In the Lambda code, I used the javascript aws-sdk to connect to DynamoDB.

get-all-jots Lambda:

var AWS = require('aws-sdk');
var ddb = new AWS.DynamoDB({region: 'your table region', apiVersion: '2012-08-10'});
exports.handler = (event,context,callback) => {
    console.log(event.userid);
    // TODO implement
   var params = {
  ExpressionAttributeNames: {
   "#T": "title",
   "#JI": "jotid"
  }, 
  ExpressionAttributeValues: {
   ":id": {
     S: ""+event.userid
    }
  }, 
  FilterExpression: "userid = :id", 
  ProjectionExpression: "#JI,#T", 
  TableName: "jotter"
 };
 ddb.scan(params, function(err, data) {
   if (err){
        console.log(err, err.stack); // an error occurred
        callback(err)
   } 
   else{
        console.log(data);
        let jots = [];
        data.Items.map((item) => {
         let jot = {
          title: item.title.S,
          jotid: item.jotid.S
         }
        jots.push(jot)
        })
        console.log("helloworld",jots);
        callback(null,jots);
   }    
 });
};

Testing get-all-jots Lambda function:
Choose to configure test events beside Test button. Here we configure the event object that is passed as parameter to Lambda function.

Save it, and test the Lambda function. You should see DynamoDB data in the response.

If you don’t see the results, enjoy debugging the logs in CloudWatch. You could navigate to logs from monitoring tab in Lambda console.

Similarly create the remaining 3 Lambda functions and test them by configuring event objects.

create-new-jot Lambda:

var AWS = require('aws-sdk');
var ddb = new AWS.DynamoDB({region: 'your table region', apiVersion: '2012-08-10'});

exports.handler = (event,context, callback) => {
    var params = {
      TableName: 'jotter',
      Item: {
        'userid' : {S: ''+event.userid},
        'title' : {S: ''+event.title},
        'body':{S: ''+event.body},
        'jotid':{S: ''+event.jotid}
      }
    };

ddb.putItem(params, function(err, data) {
  if (err) {
    console.log("Error", err);
    callback(err);
  } 
  else {
    console.log("Success", data);
    callback(null, data);
  }
});
};

Event object for test configuration: (create-new-jot)

{  
   "userid":"<32 digit uuid code>",
   "jotid":"<32 digit uuid code>",
   "title":"title content",
   "body":"body content"
}

get-jot Lambda:

var AWS = require('aws-sdk');
var ddb = new AWS.DynamoDB({region: 'your table region', apiVersion: '2012-08-10'});

exports.handler = (event, context,callback) => {
 let jotid = event.jotid;
 let userid = event.userid;
 
 var params = {
  Key: {
   "jotid": {S: ""+jotid},
   "userid": {S: ""+userid}
  }, 
  TableName: "jotter"
 };
 ddb.getItem(params, function(err, data) {
   if (err){
    console.log(err);
    callback(err)   
   }
   else{
    console.log(data);
     let jot = { 
      "userid": data.Item.userid.S,
      "title": data.Item.title.S,
      "body": data.Item.body.S,
      "jotid": data.Item.jotid.S
    }
    console.log(data);
    callback(null,jot)
   }
 });
};

Event object for test configuration: (get-jot)

{  
   "userid":"<32 digit uuid code>",
   "jotid":"<32 digit uuid code>",
}

delete-jot Lambda:

var AWS = require('aws-sdk');
var ddb = new AWS.DynamoDB({region: 'your table region', apiVersion: '2012-08-10'});

exports.handler = (event, context,callback) => {
 let jotid = event.jotid;
 let userid = event.userid;
 
 var params = {
  Key: {
   "jotid": {S: ""+jotid},
   "userid": {S: ""+userid}
  }, 
  TableName: "jotter"
 };
 
 ddb.deleteItem(params, function(err, data) {
  if (err) {
    console.log(err, err.stack);
    callback(err)
  }
  else {
    console.log(data);
    callback(null,data);
  }
});
};

Event object for test configuration: (delete-jot)

{  
   "userid":"<32 digit uuid code>",
   "jotid":"<32 digit uuid code>",
}

Test all the above Lambda functions, and see if the data you get in the response from callback is correct. For new jot check if the data is updated in the DynamoDB table. If not enjoy debugging the logs. We shall now see how we can invoke these functions from API gateway for each API/http-method call.

Working with API Gateway:

As we learned earlier, it makes it easy for developers to create, publish, manage, and secure APIs at any scale. It acts as a front door to backend services.

Before we start building let us understand our requirements and design our API accordingly.

• Get a list of all jots of a user.
• Post a new jot by a user.
• Get the full content of a jot of a user.
• Delete a jot of a user.

Accordingly our API request URIs & http methods could be something like this.

• GET <URL endpoint>/alljots?userid=<id>
• POST <URL endpoint>/newjot with jot content in body
• GET <URL endpoint>/jot?userid=<id>&jotid=<id>
• DELETE <URL endpoint>/jot?userid=<id>&jotid=<id>

Go to console > API Gateway > APIs > Create API

After creating the API, we could add any no. of resources(paths) to it. Let us add first resource.

Enable API Gateway CORS as it allows requests from different origins. Let us add the get method to it.

After adding the method, select the integration type to be Lambda function, fill in Lambda function details like region and name. When this URI is called this Lambda is invoked.

After saving, select GET, it opens method execution.

Understanding Method Execution:

1. Client requests are passed to Method Request.
2. Method Request acts like a gatekeeper it authenticates request, if not it rejects the request with 401 status code. Then it passes to Integration Request.
3. Integration Request controls what goes into action(Lambda in our case). Data mapping & shaping can be done here. 
4. Call the action with request data as params. 
5. When action calls back with response, Integration Response handles it.
6. Integration Response controls what comes out of action. It can also be used to map data to a required structure.
7. Method Response is used to define Integration Response. 
8. Method Response responds to the Client.

Adding a Data mapping template to Integration Request:

What mapping templates does is it maps data from request body or params into an object that is passed as event object to action (Lambda function). We do this only when we have to extract and get data into some specific structure our action needs.

Select Integration Request, open Mapping Templates.

After adding Data mapping template to Integration Request, let us test the API.

Testing the API:

Click on TEST in the Method Execution section. Enter the QueryStrings params as:

userid=”uuid”

Click on Test. You should be seeing all the jots of that userid in the Response Body.

Deploy API to a stage:

You could create different deployment stages, like dev, prod, testing etc. Everything we create or modify, for them to take effect, API has to be deployed to a stage. This gives us a URL endpoint, we could use to make API calls. For every change made, we have to deploy or the changes will be local to API Gateway and don’t reflect on the URL endpoint. Each stage has different URL endpoint with different resources.

Now we have successfully created an API with alljots path/resource.

Testing API using postman:

Launch postman, enter the URL, choose HTTP method, add appropriate headers and body if needed. Send the request.

Tada! API is working. Similarly add the remaining 3 URI’s in API Gateway and test them using postman.

Body mapping template for Delete jot and Get jot URI resources is the same, but we pass 2 arguments(userid, jotid) in query string.

Note: The reason why we passed data in query strings for get & delete methods is that these methods do not contain request body, the only means of passing data is by using path parameters or query strings. 

Data mapping template for Delete and Get jot URI resources:

Example for Get: (similarly for Delete)

For newjot post request, data has to be passed in the body in this format:

{  
   "userid":"<32 digit uuid code>",
   "jotid":"<32 digit uuid code>",
   "title":"title content",
   "body":"body content"
}

Example for Post:

Note: And again if things didn’t work properly, There can be many possibilities of why something isn’t working as we are dealing with 3 services here, my answer is start debugging the CloudWatch logs. 

We have successfully built and tested the AWS services part in our serverless architecture. We will look at how to integrate it with a react application, create Sign-in, Sign-up using Cognito and how to secure our API’s in API Gateway with Cognito and finally deploy the app on S3 cloud storage in Part2.

This story is authored by Koushik. He is a software engineer and a keen data science and machine learning enthusiast.