Stateless Architecture - What's the Deal?
And 3 techniques to build stateless services...
“Build a stateless service.”
This advice comes up quite frequently in modern application development.
But what does “state” exactly mean?
In programming, “state” is the condition of a system or an application component at a particular point in time.
For example, if you are shopping on Amazon, your login status on the website is a piece of state. Similarly, if you have added the latest iPhone device or that Playstation you always wanted to the shopping cart, that’s also an example of state.
To make things clearer, the state represents the data that helps us understand the current status of the application.
Going down to the level of code, you can have stateless and stateful classes. For example, this is a stateless class.
On the other hand, the below class can be considered stateful.
What is Stateless Architecture?
The traditional approach to building services was mostly stateful.
In a stateful service, the server maintains the state of the client’s session across multiple requests. It means that the server keeps track of the client’s data and context, often storing it in memory. Each subsequent request from the client relies on the state stored during previous iterations.
In simple terms, this is similar to a restaurant where only one waiter handles your order. This waiter keeps all the information about your order in his notepad. If you call any other waiter to check the order status or change something, they won’t have any context about your order.
On the other hand, stateless services don’t maintain any server-side state between requests.
Taking the restaurant example, this implies that all the waiters (including the one who took your order) are completely forgetful. They won’t remember anything about your order when you call them again.
Isn’t that bad?
No - because they will instead keep a note of your order at a separate centralized location. This centralized location will be accessible to all the waiters, which means that any waiter can handle your request.
The Advantage of Stateless Design
The main advantage of stateless design is that you can horizontally scale your service layer.
Coming back to the restaurant example, let’s say you went with the stateful approach.
While it may be easy to implement such an approach in a small restaurant with not many customers, consider the problems that may arise as things scale.
What if one of the waiters, who is responsible for a bunch of orders, needs to leave due to an emergency?
It means that the information stored in that waiter’s notepad is also gone. If not, you’ve to make sure that the information is handed over to another waiter before the first waiter leaves.
Ultimately, there’s a disruption in customer service while all this happens.
The same scenario happens in the case of a stateful design where a particular server instance is responsible for storing data related to a particular user.
If a particular server goes down, all the users tied to that server are bound to receive a negative user experience. For example, it might be the user suddenly finding themselves logged out of the website or their shopping cart data vanishing.
Stateless design solves these issues.
You can scale your services by adding more server instances if traffic increases.
Since each server instance is similar, you can route the requests to any of the instances. Even if some server instances go down, there is no impact on availability.
Deployment and maintenance of stateless services is simpler. You can perform rolling updates without service disruptions. Also, such services work well with containerization and serverless approaches.
Techniques to Build Stateless Services
When designing stateless services, the main goal is to eliminate server-side state and make sure that each request contains all the necessary information for processing.
First, you need to identify and extract the state from the services.
Second, you employ strategies to externalize and manage that state effectively.
Identification of State
A few considerations for identifying the state:
Session Data: Identify any session data that is stored on the server side such as user authentication information, shopping cart content, or user preferences. This data needs to be managed independently of the service.
In-Memory State: Examine the service’s codebase for any in-memory state, such as caches, global variables, or static data structures.
Logging and Auditing: Review logging and auditing mechanisms to identify any state that is captured or persisted on the server side. Think of a way this data can be managed separately from the service instance.
Strategies to Externalize State
There are three main strategies around this:
1 - Databases and Persistent Storage
A common technique to externalize the application state is to store it in a database.
See the below diagram for reference:
If the state data is structured and requires ACID, you can use relational databases like MySQL and PostgreSQL.
For more unstructured data (for example, shopping cart info), you can also go for key-value stores. This includes NoSQL options as well as distributed caches.
Lastly, for storing and retrieving large data objects such as files and images, you should use object storage such as Amazon S3 or Azure Blob Storage.
2 - Design Stateless APIs
The second technique to build stateless services is using RESTful principles to design APIs.
The API operations can be idempotent and use appropriate HTTP methods. These services should implement proper error handling and rollback mechanisms to maintain state consistency.
3 - Handling Session State and User Context
One of the common reasons for using state is to store the authentication status for a user. However, with stateless services, you should not store this information on the server instances.
A few ways to manage are as follows:
Implement stateless authentication mechanisms such as JSON Web Tokens to securely transmit user identity and claims. Validate and verify JWTs on each API request to ensure the authenticity of the user context.
Use client-side storage mechanisms such as cookies, local storage, or session storage to store specific data. Of course, security considerations need to be taken up.
Implement token-based session handling where a session token is generated and sent to the client. The session information is stored in shared storage or database for future reference.
The below diagram shows the session approach and the JWT approach to handle authentication.
Challenges with Stateless Services
There’s no such thing as a free lunch.
While stateless services have a lot of advantages such as scalability, fault tolerance, and easier deployment, they also create some challenges.
Stateless services often lead to increased network overhead because each request needs to include all the necessary data and context.
Stateless services result in your data getting distributed to external storage systems. You run into challenges with replication, partitioning, and eventual consistency. It also introduces additional latency.
More moving parts means greater operational complexity and monitoring needs. Also, higher infrastructure costs and data storage costs.
You also have to deal with security challenges such as maintaining the integrity of authentication tokens and ensuring that the communication channels are secure.
So - what do you think about building stateless services? Do you find it a good strategy or a bad one?
Launch of Eraser AI (Affiliate)
This week Eraser launched Eraser AI - a copilot for technical design.
They have rebuilt the AI feature from the ground up and it now includes diagram generation, diagram editing, and doc outline generation.
Here’s an awesome video that explains how Eraser AI can help you in your daily workflow.
You can try Eraser AI for free. But if you are looking for more usage and additional features, you need to opt for the Pro Plan.
Head over to Eraser and at the time of checkout, use the promo code “CODEX” to get a discount of $12 or a 1-month free trial on the Pro Plan.
Interesting Articles
Here are a few interesting articles I’ve read this week:
How to build credibility in the engineering industry by
Make every penny count and every byte secure by
That’s it for today! ☀️
Enjoyed this issue of the newsletter?
Share with your friends and colleagues.
See you later with another value-packed edition — Saurabh.
This was great, Saurabh! One thing I can think of to push people more in the stateless direction is exactly what some libraries are doing.
For example, express-session, a simple session middleware for express, states in the Readme that the default implementation is an in-memory story that doesn't scale past a single process, and you want to save your session data in the db in production. This is how I learned many years ago about stateless design and how to spin up multiple servers to serve users.
Stateful is suitable for development purposes, but we went stateless even for apps where we only had a single instance because it wasn't more complicated. However, it offered the benefits of scaling horizontally if we had to.