SCALE FROM ZERO TO MILLIONS OF USERS [Part 1]

Designing an application to cater larger audience is an evolutionary journey. Let’s start this voyage together. We’ll begin with a Single user system and gradually evolve to million users.

Single Server setup

Single server setup

1. User access website through Domain name (i.e. www.example.com)
2. DNS returns the IP address to the browser
3. Browser routes the HTTP request to the web server through the IP address
4. webserver returns an HTML page or response

With the growth of the user base, a single server won’t be sufficient to handle the load. So, separating the web tier and DB server allows them to scale independently.

Which DB to use?

There are two kinds of major DBs:-

1. Relational DB (MySQL, SQL Server, Oracle, PostgreSQL etc.)

Relational DB are well suited for the below use cases:-

1. Data are structure
2. Transactions are more important
3. Normalization is required

· Non-Relational DB (Mongo, couchDB, Cosmos, DynamoDB, Casandra, HBase, Neo4J etc)

NoSql are well suited for the below use cases

1. Data are unstructured
2. we need to store a humongous amount of data

This setup looks fine with an initial set of users. our business is doing great in the market. since many users are accessing the application, and with this setup, they are facing some performance and outage issues. So the solution is to scale the infra at a certain level, maybe Vertically or Horizontally.

Vertical scaling vs Horizontal Scaling

1. vertical scaling (Scale-up) — it refers to add more power (CPU, RAM ) to a server instance.

• vertical scaling is a good solution in case of low traffic, but there are some limitations as well
• It is impossible to add infinite CPU and RAM to a server
• It is a single point of failure. Failover can’t be possible with this setup and ultimately results in an application outage
• The solution to this problem is Horizontal Scaling

2. Horizontal scaling(Scale Out) — It refers to adding multiple server instances

• Since we have multiple servers we can cater to multiple requests in w.r.t Single server setup.
• But still, we have a challenges
• How to decide on which server we should route the requests to, keeping in mind that servers are used significantly?
• How to keep track of each request -response?
• The solution to this challenge is the Load balancer

Load balancer (Layer 7)

It efficiently distributes incoming traffic among the backend servers with the help of certain algorithms (i.e. Round Robin, Least Connects, Least time, IP hash etc..)

- Here, we have achieved horizontal scaling
- we have added an LB(Load Balancer) and a backend server, now with this setup we have resolved the failover issue and improved System Availability
- If traffic grows and 2 backend servers are not enough to handle efficiently, we can add more backend servers to the web server pool and LB will automatically start to route the request

Congratulation !! we have successfully called our application from 1K to 10K users

With the above setup, the web application is scaled, but since we are using only one DB there could be the possibility of a DB outage😐.

The solution to this problem is DB replication🕺

Reference Materials

1. System Design Interview an Insider Guide
2. Hypertext_Transfer_Protocol
3. https://netflixtechblog.com/active-active-for-multi-regional-resiliency-c47719f6685b
4. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html
5. https://learn.microsoft.com/en-us/azure/azure-monitor/autoscale/autoscale-overview
6. https://www.akamai.com/our-thinking/cdn/what-is-a-cdn