API Best Practices

Page Responsiveness is one of the most key indicators of the health of any product. Be it consumer centric product or business centric product , page responsiveness decides how quickly a user can get value out of any website.

Recently we started monitoring our page load times to see how does the user perceive and were completely shocked to see that some of our pages were taking around 20-30 seconds to load. After this , the next logical thing was to figure out why exactly this was the case. Slowly and steadily we figured out all the reasons for this slowness. In this whole process, we realised that the solutions to fix were much more generic and could have impacted any of your websites.

So in this blog post, we will go through these fixes made by us and also go about writing down the best practices for writing APIs. To make the subject matter easy and understandable , we will take the example of a simple fruits / vegetable website in which all the details related to all the different fruits are shown and people can click on a particular fruit element to get more details of the fruits.

Before proceeding ahead with the problems , I would like to talk about the design aspect of the APIs.

Designing APIs

If you are thinking that the frontend is the only thing consuming your APIs, you could not be more wrong. In this era, where the whole world is so integrated well together , if your application is not well integrated with other applications which people use day to day , then you are just digging your own grave. It is just a matter of time, another well integrated application will come and eat you for lunch. So the only way to save yourself from becoming obsolete is making your product as a platform which can be integrated with other such products.

Now to host your product as a platform, one of the core requirements is to design your APIs with specific requirements

API Endpoint should always be on a resource / resource-id

• Resource ID should not be part of the request body but it should be the part of the request endpoint
  
  • /fruits
    • CRUD /fruits/
    • CRUD /fruits/fruit-id/
      
  • /vegetables
    • CRUD /vegetables/
    • CRUD /vegetables/vegetable-id/
      .
• When performing any filtering of any kind on a collection, always follow this norm
  
  • Request Type: POST
  • Request Body: Json Object with all the filters applied
  • Endpoint: /fruits, /vegetables
    
    
• Your REST API Routes should closely resemble all the different logical entities which are present in your backend. If there are lesser or more number of logical entities in your backend when compared to route endpoints, then it means that there is still room for improvement
  
  • Let’ say that you have 2 logical business entities eg. Fruits and Vegetables so expected api routes in the platform would only be /fruits and /vegetables.
    
  • So if you have any top more level API routes in the platform like /good-fruits or /good-vegetables, then it means that there is still some room for improvement and routes for /good-fruits and /good-vegetables can still be translated to /fruits or /vegetables with filters in the POST body respectively.
    
  • Also in this principle we are talking about business entities like fruits and vegetables and not common any platform entities like /users or /groups

For summarising the different endpoints

GET	/collection	Get all the elements of the Collection
POST	/collection	Filter down to all the relevant elements of the Collection
PUT	/collection	Create a new Element of the Collection
PATCH	/collection/:resource_id	Update an existing Element of the Collection
DELETE	/collection/:resource_id	Delete an existing Element of the collection
POST	/collection/:resource_id/action	Doing an Action on a resource of a Collection
POST	/collection/action	Doing an Action on the Collection

Now that we have established common ground rules for API design, let’s come onto our next section in which we will talk about other API design fixes which we made to improve the page responsiveness

API Design Fixes

Before diving into each and every fix, I would like to remind each and every one of our reader that doing any computation on backend is far more preferred than doing any computation on the browser. Browsers should only act as a presentation layer which will just render the data coming from the API so that it is very easy for the user to communicate with the interface.

Problem # 1: Loading Resources / Models with Large Amount of Data in our Frontend

We had certain resources / models within our system which had huge amounts of data associated with it.

{
   "id" : "fruit-1"
   "name".    : "apple"
   "description" : "Apple is a wonderful fruit. An apple a day keeps the doctor away"
   "price"    : "$5 / kg" 
   "fruit_images" : blob1, blob2, blob3
}

As you can see in the above example, in the response for /fruits/fruit-1 , we are getting response of worth 2 MBs. This meant , we are taking valuable seconds by querying this from the DB and then taking taking valuable seconds by brining this payload to the UI.

However, in this above example, we were not at all loading those images on the interface which essentially meant that majority of the payload was not even being shown to the users. This led to make platform level changes to enhance our APIs to only return relevant content to the interface.

Fix # 1: Interface can now pass fields which would be required for loading of the resource.

So previously the API endpoint was

/fruits/fruit-id , returning payload of size ~ 2 MBs

{
   "id" : "fruit-1"
   "name".    : "apple"
   "description" : "Apple is a wonderful fruit. An apple a day keeps the doctor away"
   "price"    : "$5 / kg" 
   "fruit_images" : blob1, blob2, blob3
}

but now, UI interface is using following endpoint

/fruits/fruit-id?fields=id,name,description,price, was returning payload of size ~ 50 KB

{
   "id" : "fruit-1"
   "name".    : "apple"
   "description" : "Apple is a wonderful fruit. An apple a day keeps the doctor away"
   "price"    : "$5 / kg" 
}

So Resource APIs should support a way to define what all fields are needed for that resource in the response. This will ensure that the backend does not return the fields which are not needed by the interface or anything calling the API.

Problem # 2: Page Rendering was dependent on too many API calls

Loading of certain pages required too many API calls on the backend which meant that loading times of these pages was on the higher side. These API calls were all designed properly but the main issue was the fact that the page rendering was totally blocked on all these API calls

When we were loading the page for all the fruits, we were loading the data for all the filter values for fruits as well as other filters. This meant that we were waiting for response for APIs which were not directly impacting the view of the user for the page.

Fix # 2: Do not block the page rendering on the APIs which do not contribute to the view of the user for the page.

So we essentially made the API calls for getting the filter values for fruits and others non blocking for the page. This meant that the page will be loaded even if the response for the filter values has not even reached the interface. This improved our performance significantly. Also because the filter values were being loaded asynchronously so at the time user clicked on the filter, the user just needed to wait till the instant at which the API will get completed.

So if we were to put this generically, main page rendering should only and only be blocked on the APIs whose response will be truly be used by the interface for the page loading

Problem # 3: Making same API calls in quick successions

There were certain interactions within our product like searching where we were making too many API calls in quick sessions. So every-time user enters a character, we used to make a backend search for the resource-ids. This led to increased load on the backend which led to the increased page API latencies.

Fix #3: Debouncing

Debouncing is a programming practice in javascript which can be used to decrease the number of API calls which are being made in the backend

Debouncing in JavaScript is a practice used to improve browser performance. There might be some functionality in a web page which requires time-consuming computations. If such a method is invoked frequently, it might greatly affect the performance of the browser, as JavaScript is a single threaded language. Debouncing is a programming practice used to ensure that time-consuming tasks do not fire so often, that it stalls the performance of the web page. In other words, it limits the rate at which a function gets invoked.

Problem # 4: Making same API calls on different pages

We were making same API calls repeatedly on the different pages. API calls for populating different filter values for different drop-downs were being populated for each and every paginated response. This led to increased load on the backend leading to increased page latencies.

• 

• 

As one can clearly see in the above 2 visualizations that we are making the same API call for loading the filter values for both the pages i.e. Page 1 and Page 2.

Fix #4: Layering UI Components Logically and Making API calls when those components load.

Each UI component needs to be defined in a relevant scope and if that scope is not changing then the component wont be reloaded. In our case, we had defined the filter values dropdown within the scope of the fruits page because of which anytime a user would navigate from page 1 to page 2, filter values used to be populated again from the backend.

So after the fix, the filter values dropdown was moved to a higher scope when compared to the main fruit listing so because of that whenever the fruits listing was navigated from page 1 to page 2 , the higher scope was still the same without being populated again.

Lessons

If you were to sum up this blog post, you just have to remember 2 very basic rules of achieving low latencies via APIs

• Always prefer computing on the backend. UI should not have any business logic whatsoever and hence no computation of any kind should be done on the frontend.

• Loading of a Page should only and only wait for the response which it will show to the user when the page is rendered. If there is any response page is waiting for which is not being displayed to the user when the page is rendered for the first time then it means either
  
  • You are blocked for the wrong set of APIs
  • You don’t have proper APIs and your APIs need refactoring.