SDC#31 - Introduction to Pre-caching

The secret technique to boost application performance...

High traffic to your application is a triple-edged sword.

What’s wrong with me? Such a sword doesn’t exist…I think. Probably.

But anyway, let’s run with the assumption for a second that such a sword exists and high traffic is an example of it.

• On one side, high traffic shows that people are probably liking your stuff.

• But, what if the traffic is just a bunch of bots trying to DDoS you out of existence?

• Third, if the high traffic crosses the danger level and crashes the website, you’re going to alienate a lot of users.

How do you deal with all these possibilities?

While there are many techniques to boost application performance, there’s one that is not as well known as the others.

Yes, I’m talking about pre-caching.

Before we go further, a warm welcome to the 496 new subscribers who have joined us since last week.

If you aren’t subscribed yet, join 7400+ curious Software Developers looking to expand their system design knowledge by subscribing to this newsletter.

What is Pre-caching?

Pre-caching is a technique used to proactively cache data in anticipation of future requests.

The idea is to cache in advance data that has a high probability of being accessed so that you can serve it faster when requested by a user.

See the below diagram that shows the concept of pre-caching.

You can play around with the diagram on Eraser.io

The great part about pre-caching is that you can do it on the client side (browser) as well as on the server side.

Why Pre-caching?

Does pre-caching sound like a lot of trouble?

In my experience, its advantages outweigh the difficulties.

Here are a few important benefits:

1 - Performance Improvement

When you pre-cache data, you reduce the load time for your application.

This can be a shot in the arm for an application with a high volume of traffic.

2 - Better User Experience

No one likes websites where accessing a page feels like exorcising a demon.

Faster load time resulting from pre-caching improves the user experience. It also improves the availability of the system.

3 - Cost Savings

Pre-caching helps keep more money in the bank or invest elsewhere.

For example, by pre-caching data on a CDN, you are reducing the load on the origin server. This saves bandwidth and reduces the server costs.

4 - Reliability

Pre-caching can help blunt the impact of Denial of Service (DoS) attacks since the application won’t have to serve pre-cached resources from the origin server.

It’s an indirect benefit, but ultimately pre-caching improves the overall reliability and security of the application.

Steps to implement pre-caching

There are 4 main steps when it comes to implementing pre-caching.

You can play around with the diagram on Eraser.io

👉 STEP 1 - Identify frequently accessed data or resources. These resources are good candidates for pre-caching.

For example, most popular blog posts, bestseller product lists and so on. You can also include images, JS files and stylesheets.

👉 STEP 2 - Decide on the caching system to store the pre-cached data.

You could go for a local cache on the user’s device or even a distributed cache running on multiple servers.

👉 STEP 3 - Pre-populate the cache with the identified resources. You can configure the system to perform this step automatically during the system initialization phase.

Alternatively, you can also populate the cache on an on-demand basis as the data is accessed by the users. Remember, pre-caching is all about being proactive.

👉 STEP 4 - Analyze the access patterns and improve the pre-caching algorithm.

This is important to maintain a good cache performance.

Pre-caching Demo with Node.js

While there can be various implementations of pre-caching as a solution, the below piece of code shows a basic approach using Node.js and Express.

const express = require('express');
const nodecache = require('node-cache');
require('isomorphic-fetch');

// Setting up Express
const app = express();

// Creating the node-cache instance
const cache = new nodecache({stdTTL: 10})

// We are using the fake API available at <https://jsonplaceholder.typicode.com/>
const baseURL = '<https://jsonplaceholder.typicode.com/posts/>';

// Pre-caching Popular Posts
[1, 2, 3].map(async (id) => {
    const fakeAPIURL = baseURL + id
    const data = await fetch(fakeAPIURL).then((response) => response.json());
    cache.set(id, data);
    console.log(`Post Id ${id} cached`);
})

// API Endpoint to demonstrate caching
app.get('/posts/:id', async (req, res) => {
    const id = req.params.id;
    if (cache.has(id)) {
        console.log('Fetching data from the Node Cache');
        res.send(cache.get(id));
    }
    else {
        const fakeAPIURL = baseURL + id
        const data = await fetch(fakeAPIURL).then((response) => response.json());

        cache.set(req.params.id, data);
        console.log('Fetching Data from the API');
        res.send(data);
    }
})

// Starting the server
app.listen(3000, () => {
    console.log('The server is listening on port 3000')
})

The above example uses the node-cache library to create an in-memory cache.

To simulate how pre-caching works, the program simply assumes that posts with IDs 1, 2 and 3 are extremely popular ones and suitable candidates for pre-caching.

The data for these posts are pre-fetched during the application startup process and stored in the cache object. When a request is made for these specific posts, the application fetches the data directly from the cache instead of calling the source API.

Note that this example isn’t perfect.

We don’t get into solutions for stale data or maintaining cache consistency. The goal is to simply show a simple example of pre-caching.

Server Side Pre-caching

As I said earlier, pre-caching can work both on the client and the server.

But since we focus more on the backend side of things, let’s look at a couple of ways to perform server-side pre-caching.

Content Delivery Networks

In this method, you store a copy of the data on special servers that are distributed worldwide.

The geographic relevance of data plays a key role in determining what data to store where.

For example, a streaming platform like Netflix might choose to pre-cache different movies in CDNs located in North America and the Asia Pacific regions since user preferences will inevitably vary from region to region.

CDNs can perform on-demand caching as well as pre-cache results for future requests.

See the below diagram that shows the concept of a CDN.

You can play around with the diagram on Eraser.io

When a user requests some data, it’s delivered from the server that’s closest to the user’s location.

This in turn reduces the time it takes to serve the request and creates a better experience for the user.

Caching Proxy Server

In this approach, a server sits in front of the origin server and doubles up as a caching layer that stores a copy of the data.

When a request comes from the user, the proxy server delivers the data directly without having to request the origin server.

The below diagram shows this setup:

You can play around with the diagram on Eraser.io

Popular web servers such as Nginx can play the role of a caching proxy server. All you need to do is write the proper configuration.

For example, here’s the configuration to cache all files with the extension jpeg/jpg, png, CSS, and js for 60 minutes.

# configure the proxy server to cache all assets for 1 hour
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=static_cache:20m inactive=60m;

# set the cache control header to a max-age of 1 hour
add_header Cache-Control "public, max-age=3600";

# cache all assets
location ~* \\\\.(jpg|jpeg|png|css|js)$ {
    proxy_cache static_cache;
    proxy_cache_valid 200 60m;
    proxy_pass http://origin_server;
}

---

👉 So - what do you think about pre-caching and have you used it in your application in some form?

---

Roundup

Here are 3 interesting posts I read this week:

• Engineering is more about people than tech by

  Gregor Ojstersek
  : An eye-opening article on why developers need to build people skills.

• Design a Distributed Priority Queue by

  Eugene Shulga
  : A nice explainer on designing a distributed priority queue.

• Track your growth as an engineer by

  Fran Soto
  : We track metrics for our projects and systems. But what about our career?

---

That’s it for today! ☀️

Enjoyed this issue of the newsletter?

Share with your friends and colleagues.

See you later with another value-packed edition — Saurabh.