10K to 10M Users: Would You Start With Microservices?

Introduction#

Imagine you are designing a new platform.

Today, you expect:

10,000 users

But the business says:

"We want this platform to support 10 million users eventually."

The first architecture question usually appears immediately:

Should we start with microservices so the system can scale later?

For most systems, the answer is:

No.

I would usually start with a well-structured modular monolith, design clear domain boundaries, keep the application stateless where possible, and build the infrastructure so individual components can be extracted later.

The reason is simple:

User count does not determine architecture.

A system with 10 million users might comfortably run on a monolith.

Another system with only 100,000 users might require microservices because of organizational complexity, independent workloads, or strict isolation requirements.

The real question is not:

How many users do we have?

The better question is:

What architectural problems are we actually trying to solve?

Why This Matters#

Microservices are often presented as the architecture used by companies operating at massive scale.

You hear about architectures from companies like:

Netflix
Amazon
Uber
Google
Spotify

And the conclusion becomes:

Large company
    ↓
Millions of users
    ↓
Microservices

But that skips an important part of the story.

Large companies usually adopt distributed architectures because they have:

• Hundreds or thousands of engineers
• Many independent business domains
• Different scaling requirements
• Independent deployment requirements
• Large infrastructure teams
• Complex reliability requirements
• Different technology stacks
• Multiple geographical regions

Microservices solve many of those problems.

But they also introduce entirely new problems.

If your application does not have those problems yet, starting with microservices can mean paying the complexity cost before receiving the benefits.

Problem#

Suppose we are building an e-commerce platform.

The system contains:

Users
Products
Orders
Payments
Inventory
Notifications
Analytics

Someone proposes this architecture immediately:

                    API Gateway
                         |
        ---------------------------------
        |        |       |       |      |
      User    Product   Order  Payment Inventory
     Service  Service  Service Service  Service
        |        |       |       |      |
       DB       DB      DB      DB     DB
 
                         |
                   Kafka / RabbitMQ
                         |
                    Notification
                       Service

This looks scalable.

But we have created several distributed-system problems before the product has even acquired users.

Now we need to handle:

• Service discovery
• Network failures
• Distributed tracing
• API gateways
• Message brokers
• Event schemas
• Event versioning
• Retry strategies
• Dead-letter queues
• Idempotency
• Distributed transactions
• Eventual consistency
• Service authentication
• Deployment orchestration
• Multiple databases
• Observability across services
• Cross-service debugging
• Infrastructure automation

Instead of solving:

"How do we build the product?"

we are now solving:

"How do we operate a distributed system?"

That can be a very expensive trade-off for an early-stage system.

Scalability Is Not the Same as Microservices#

One of the most important system design concepts to understand is:

Microservices are not required for horizontal scaling.

A monolithic application can also scale horizontally.

Suppose we have one Node.js application:

Frontend
    |
Load Balancer
    |
-----------------------------
|            |             |
App 1        App 2         App 3
|            |             |
-----------------------------
             |
          Database

If the application is stateless, we can run multiple instances behind a load balancer.

For example:

             Nginx / Load Balancer
                      |
       --------------------------------
       |              |               |
    Node #1         Node #2         Node #3
       |              |               |
       --------------------------------
                      |
                   PostgreSQL
                      |
                    Redis

Traffic can be distributed across all application instances.

If traffic doubles:

3 instances
    ↓
6 instances

If traffic grows again:

6 instances
    ↓
12 instances

Nothing about this requires microservices.

Start With Capacity, Not User Count#

Another mistake is using total registered users as a scalability metric.

Suppose the product has:

10,000,000 registered users

That does not mean 10 million users are hitting your backend simultaneously.

Maybe only:

1,000,000 monthly active users
200,000 daily active users
20,000 peak concurrent users
2,000 requests/second

Your architecture should be based on workload.

Important measurements include:

For example:

10M users

could generate less traffic than:

500K users

on a real-time trading platform.

So architecture should follow traffic patterns, not vanity numbers.

Solution#

Instead of starting with microservices, I would usually start with this:

                 Client Applications
                        |
                  Load Balancer
                        |
             -----------------------
             |         |           |
           App 1     App 2       App 3
             |         |           |
             -----------------------
                        |
                 Modular Monolith
                        |
        -----------------------------------
        |          |        |      |      |
       Auth      Orders   Payment Product Inventory
        |          |        |      |      |
        -----------------------------------
                        |
                  PostgreSQL
                        |
                      Redis

Internally, the application is divided into business modules.

For example:

src/
├── auth/
├── users/
├── products/
├── orders/
├── payments/
├── inventory/
├── notifications/
└── analytics/

Each module should own its business logic.

The important part is that modules should not become tightly coupled spaghetti.

Modular Monolith#

A modular monolith is still deployed as one application.

But internally it has strong boundaries.

Instead of this:

Orders
  ↓
directly modifies payment tables
  ↓
directly modifies inventory tables
  ↓
directly sends emails

prefer:

Order Module
     |
     ├── Inventory Module
     |
     ├── Payment Module
     |
     └── Notification Module

Each module exposes a clear interface.

For example:

interface InventoryService {
  reserveProduct(productId: string, quantity: number): Promise<void>;
}

The order module should not know how inventory storage works.

It only knows:

reserve inventory

This makes future extraction much easier.

Think in Domain Boundaries#

One of the best ways to prepare for future microservices is to design around business capabilities.

For example:

Identity
Catalog
Ordering
Payments
Inventory
Shipping
Notifications

These could eventually become bounded contexts.

Initially:

Monolith
 
├── Identity
├── Catalog
├── Ordering
├── Payments
├── Inventory
├── Shipping
└── Notifications

Later:

                        API Gateway
                             |
             --------------------------------
             |               |              |
         Monolith        Payment Service  Notification
             |               |             Service
             |
      -----------------
      |       |       |
    Users   Orders  Catalog

You do not need to extract everything simultaneously.

Scale Vertically First#

Early in the system's life, the easiest solution may simply be using a larger machine.

For example:

2 CPU / 4 GB RAM
        ↓
4 CPU / 8 GB RAM
        ↓
8 CPU / 16 GB RAM

This is called vertical scaling.

Vertical scaling has limits, but it is incredibly simple.

There is no shame in using it when it solves the problem.

Architecture should optimize for the current stage of the product.

Then Scale Horizontally#

Once a single application instance becomes insufficient, run multiple instances.

                  Load Balancer
                       |
       --------------------------------
       |              |               |
    Instance 1     Instance 2      Instance 3
       |              |               |
       --------------------------------
                       |
                    Database

This requires your application to remain mostly stateless.

Avoid storing session state like this:

const sessions = new Map();

because:

User request 1 → Server A
 
User request 2 → Server B

Server B would not know about the session stored in Server A.

Instead use:

JWT

or shared session storage:

Redis

Now every application instance can process the request.

Add Caching#

At scale, your database often becomes the bottleneck before the application itself.

Suppose the same product information is requested thousands of times:

SELECT *
FROM products
WHERE id = 123;

Instead of hitting PostgreSQL repeatedly:

Request
   |
 Redis
   |
 Cache hit?
 /       \
Yes       No
 |         |
Return    Database
           |
         Cache
           |
         Return

Example:

async function getProduct(productId: string) {
  const key = `product:${productId}`;
 
  const cached = await redis.get(key);
 
  if (cached) {
    return JSON.parse(cached);
  }
 
  const product = await db.product.findUnique({
    where: { id: productId },
  });
 
  await redis.set(key, JSON.stringify(product), "EX", 300);
 
  return product;
}

This is a simple cache-aside strategy.

It can dramatically reduce database pressure without introducing microservices. The caching architecture guide explains the invalidation, stampede, hot-key, and failure decisions that appear once this simple pattern carries production traffic.

Optimize the Database#

Before breaking an application into services, make sure the database is actually designed correctly.

A slow query like:

SELECT *
FROM orders
WHERE user_id = 123
ORDER BY created_at DESC;

may not require a new service.

It might simply need an index:

CREATE INDEX idx_orders_user_created
ON orders(user_id, created_at DESC);

You should investigate:

EXPLAIN ANALYZE

Check:

• Sequential scans
• Index scans
• Query execution time
• Rows scanned
• Join strategies
• Sort operations

Many supposed "architecture problems" are really database optimization problems. Use EXPLAIN ANALYZE to measure the work PostgreSQL actually performs before introducing a larger architectural change.

Add Read Replicas When Needed#

As traffic becomes read-heavy:

                    Application
                         |
              ---------------------
              |                   |
            Writes              Reads
              |                   |
          Primary DB       Read Replica(s)

For example:

Create order
    ↓
Primary database
 
View products
    ↓
Read replica

This increases database read capacity without splitting the application. It also changes the freshness guarantee; the guide to stale reads after replica scaling covers read-your-writes strategies and replication-lag trade-offs.

Add CDN for Static Content#

Do not serve everything from your backend.

Images, videos, CSS, JavaScript, and downloadable content should often be served through a CDN.

User
 |
CDN
 |
Cached Asset

Instead of:

User
 |
Application Server
 |
Storage

A CDN can handle enormous amounts of traffic while reducing pressure on your application infrastructure.

Use Asynchronous Processing#

Another common bottleneck is doing too much work during HTTP requests.

Suppose an order API does this:

Create Order
    ↓
Charge Payment
    ↓
Generate Invoice
    ↓
Send Email
    ↓
Update Analytics
    ↓
Send Notification
    ↓
Return Response

The user waits for everything.

Instead:

Create Order
    ↓
Store Order
    ↓
Queue Background Work
    ↓
Return Response

Then:

                   Queue
                     |
        ---------------------------
        |            |            |
      Email       Analytics    Notification
      Worker       Worker         Worker

With Node.js you might use:

BullMQ + Redis

or later:

Kafka
RabbitMQ
SQS

depending on the requirements.

Again, this improves scalability without forcing the entire system into microservices.

A Practical Growth Path#

A realistic architecture evolution might look like this.

Stage 1: 10K Users#

Keep things simple.

Client
   |
Node.js Application
   |
PostgreSQL

Maybe:

1-2 application instances

Focus on:

• Product development
• Clean code
• Database indexes
• Logging
• Monitoring
• Backups
• Security

Stage 2: 100K Users#

Traffic starts growing.

Add:

Load Balancer
Redis
CDN
Multiple App Instances

Architecture:

                  CDN
                   |
              Load Balancer
                   |
          ---------------------
          |         |         |
        App 1     App 2     App 3
          |         |         |
          ---------------------
                   |
              PostgreSQL
                   |
                 Redis

Stage 3: 1M Users#

Now workload becomes more serious.

Add:

Background workers
Queue
Read replicas
Better observability
Autoscaling

Architecture:

                         CDN
                          |
                    Load Balancer
                          |
              ------------------------
              |          |           |
            App 1      App 2       App N
              |          |           |
              ------------------------
                          |
              ------------------------
              |                      |
           PostgreSQL              Redis
              |
        ----------------
        |              |
      Primary       Replicas
 
                          |
                        Queue
                          |
                 ----------------
                 |              |
               Worker         Worker

Stage 4: 10M Users#

At this point, some parts of the system may show very different scaling characteristics.

Maybe:

Orders = moderate traffic

while:

Notifications = massive traffic

or:

Search = massive read traffic

or:

Payments = strict reliability requirements

Now extracting specific services starts making sense.

For example:

                        API Gateway
                             |
          -----------------------------------------
          |                    |                  |
      Core Monolith       Search Service     Payment Service
          |                    |                  |
      PostgreSQL          Elasticsearch       Payment DB
          |
        Redis
 
                             |
                           Kafka
                             |
                    Notification Service

Notice something important.

We still did not necessarily convert everything into microservices.

That is perfectly fine.

When Should You Extract a Microservice?#

A service should normally be extracted because a real boundary has appeared.

Independent Scaling#

Suppose notification traffic grows dramatically.

Order API:
 
500 requests/sec

but notifications require:

50,000 events/sec

Scaling the entire monolith just for notifications becomes wasteful.

Extracting:

Notification Service

now makes sense.

Independent Deployment#

Imagine the payment team releases frequently.

Every payment deployment currently requires redeploying:

Users
Orders
Products
Inventory
Notifications

That increases risk.

A separate payment service allows:

Payment deployment

without touching the rest of the system.

Failure Isolation#

Suppose analytics processing crashes because of high memory usage.

In a monolith:

Analytics failure
      ↓
Entire application affected

With isolation:

Analytics Service failure
      ↓
Core ordering system keeps running

That may justify separation.

Different Technology Requirements#

Perhaps your core backend is Node.js.

But search requires:

Elasticsearch

machine learning requires:

Python

and heavy data processing needs:

Spark

Separate services may allow each workload to use the most suitable technology.

Team Ownership#

Imagine 100 developers are working in one repository.

Teams constantly interfere with each other's code.

You might have:

Team A → Payments
Team B → Orders
Team C → Search
Team D → Notifications

Clear service ownership can reduce organizational coupling.

This is where Conway's Law becomes relevant:

Software architecture often reflects the communication structure of the organization building it.

When Starting With Microservices Can Make Sense#

There are situations where microservices from the beginning can be reasonable.

For example:

Existing Large Organization#

You already have:

20 independent engineering teams

Each team needs independent deployment and ownership.

Clearly Independent Domains#

Suppose you are building:

Banking Core
Fraud Detection
Notification Engine
Analytics Platform

These may have very different reliability, security, and scaling requirements.

Regulatory Isolation#

Financial or healthcare systems may require strict isolation between certain workloads or data.

Extreme Scaling Requirements From Day One#

Imagine a service expected to handle:

hundreds of thousands of requests per second

immediately after launch.

The architecture might need workload separation from the beginning.

Existing Platform Infrastructure#

If your company already has:

Kubernetes
Service Mesh
Central Logging
Distributed Tracing
Kafka
CI/CD Platform
Service Templates
Observability
Infrastructure Team

then creating another microservice is much cheaper than it would be for a startup.

The infrastructure cost has already been paid.

The Distributed Monolith Trap#

One of the worst outcomes is creating microservices that are not actually independent.

Imagine:

Order Service
     |
     ↓ synchronous call
Inventory Service
     |
     ↓ synchronous call
Payment Service
     |
     ↓ synchronous call
User Service

Now processing an order requires four services to be available.

If one fails:

entire request fails

You now have:

Monolith-level coupling
+
Network failures
+
Deployment complexity
+
Distributed tracing
+
Latency

This is called a distributed monolith.

It is often worse than a normal monolith.

Database Per Service#

A true microservice generally owns its data.

For example:

Order Service
     |
  Order DB
 
Payment Service
     |
 Payment DB
 
Inventory Service
     |
 Inventory DB

Avoid this:

Order Service ----\
Payment Service ---+---- Shared Database
Inventory Service -/

because the services are still tightly coupled through the database.

For example, if Order Service directly queries Payment tables:

SELECT *
FROM payments
WHERE order_id = ?;

then Payment Service does not really own its data.

Changing its schema may break other services.

But Database Per Service Creates New Problems#

Suppose the business asks:

Give me an order with its payment and inventory status.

Previously:

SELECT ...
FROM orders
JOIN payments
JOIN inventory;

Easy.

Now:

Order Service
     |
     ├── Payment Service
     |
     └── Inventory Service

Or perhaps:

Domain Events
     ↓
Kafka
     ↓
Read Model

Microservices remove some problems while creating others.

That is why you should not adopt them casually.

Distributed Transactions Become Hard#

Inside a monolith:

await db.transaction(async (tx) => {
  await tx.orders.create(...);
  await tx.inventory.update(...);
  await tx.payments.create(...);
});

Everything succeeds:

COMMIT

or everything fails:

ROLLBACK

With separate services:

Order Service
Inventory Service
Payment Service

there is no simple database transaction across all three.

Now you may need concepts such as:

Saga Pattern
Outbox Pattern
Compensating Transactions
Idempotency
Eventual Consistency

For example:

Order Created
     ↓
Inventory Reserved
     ↓
Payment Failed
     ↓
Release Inventory
     ↓
Cancel Order

The architecture becomes much more complex.

Network Calls Can Fail#

Inside a monolith:

paymentService.processPayment();

This is a local function call.

It is extremely fast and reliable.

With microservices:

Order Service
     |
    HTTP
     |
Payment Service

Now you need to think about:

Timeouts
Retries
Circuit breakers
Partial failures
DNS failures
Connection failures
Service availability

Every network call should be treated as something that can fail.

Observability Becomes Critical#

In a monolith, debugging might look like:

Request
   ↓
Controller
   ↓
Service
   ↓
Database

The logs are in one application.

With microservices:

Client
  ↓
API Gateway
  ↓
Order Service
  ↓
Payment Service
  ↓
Kafka
  ↓
Notification Service

A single user request may cross five systems.

Now you need:

Correlation IDs
Distributed tracing
Centralized logging
Metrics
Service dashboards
OpenTelemetry
Prometheus
Grafana

Without observability, debugging production issues becomes painful.

Deployment Complexity#

Monolith:

git push
   ↓
CI/CD
   ↓
Deploy application

Microservices:

User Service
Product Service
Order Service
Payment Service
Inventory Service
Notification Service
Search Service

Each service needs:

Build pipeline
Docker image
Deployment configuration
Environment variables
Secrets
Monitoring
Health checks
Autoscaling rules
Rollback strategy

Infrastructure complexity grows quickly.

A Better Architecture Principle#

Instead of asking:

Should we use microservices?

Ask:

Where do we need independent boundaries?

Think about:

Scaling boundary
 
Deployment boundary
 
Failure boundary
 
Data ownership boundary
 
Team ownership boundary
 
Security boundary

When several of those boundaries align, a microservice becomes much more justified.

The Extraction Strategy#

Suppose our modular monolith looks like:

Application
├── Auth
├── Users
├── Products
├── Orders
├── Payments
└── Notifications

Later notifications become extremely heavy.

We can extract them.

Before:

Order Module
     |
Notification Module

After:

Order Module
     |
     ↓
OrderCreated Event
     |
     ↓
Message Broker
     |
     ↓
Notification Service

For example:

await eventBus.publish({
  type: "OrderCreated",
  orderId: order.id,
  userId: order.userId,
});

Notification service consumes it:

consumer.on("OrderCreated", async (event) => {
  await sendOrderConfirmation(event.userId, event.orderId);
});

The important idea is:

Design boundaries early.
Distribute them late.

Architecture Evolution#

Your architecture might evolve like this:

Stage 1
 
Monolith
   |
Database

Then:

Stage 2
 
Load Balancer
     |
----------------
|      |       |
App   App     App
     |
Database
     |
Redis

Then:

Stage 3
 
Load Balancer
     |
Applications
     |
-----------------------
|          |          |
Database  Redis      Queue
                      |
                    Workers

Then:

Stage 4
 
              API Gateway
                   |
        -----------------------
        |                     |
   Core Monolith        Search Service
        |
       Kafka
        |
  ---------------------
  |                   |
Notification      Analytics
 Service           Service

Architecture should evolve because the system has earned complexity.

Common Mistakes#

Choosing Microservices Because of User Count#

Bad reasoning:

We will have 10 million users.
 
Therefore:
 
Microservices.

Better reasoning:

What is our expected RPS?
 
What are our busiest workloads?
 
Which components need independent scaling?
 
Which components require isolation?
 
How many engineering teams do we have?

Premature Database Sharding#

Do not immediately create:

Shard 1
Shard 2
Shard 3
Shard 4

when a properly indexed PostgreSQL database can handle your workload.

Try simpler options first:

Indexes
Query optimization
Connection pooling
Caching
Read replicas
Partitioning

Then shard when data or write throughput actually requires it.

Making Everything Asynchronous#

Event-driven architecture is powerful.

But this:

UserCreated
     ↓
Kafka
     ↓
ProfileCreated
     ↓
Kafka
     ↓
SettingsCreated

is not automatically better than:

await createUser();
await createProfile();
await createSettings();

Use asynchronous communication when you actually need loose coupling, background processing, or independent consumers.

Too Many Tiny Services#

Avoid creating services like:

CreateUserService
 
UpdateUserService
 
DeleteUserService
 
UserEmailService
 
UserPasswordService

That is usually not meaningful domain separation.

Microservices should represent business capabilities, not individual endpoints.

Decision Framework#

Before creating a microservice, ask these questions.

But if the only answer is:

"We might have lots of users someday."

that is usually not enough.

Best Practices#

• Start with the simplest architecture that satisfies current requirements
• Prefer a modular monolith for many new systems
• Define strong domain boundaries from the beginning
• Keep your application stateless where possible
• Use horizontal scaling before assuming you need microservices
• Add caching where measurements show repeated expensive reads
• Optimize database queries and indexes before adding architectural complexity
• Use CDN for static and cacheable content
• Move slow background work to queues and workers
• Add observability before the system becomes difficult to debug
• Measure P95, P99, throughput, CPU, memory, database load, and error rate
• Extract services around genuine business and operational boundaries
• Avoid shared databases between independent microservices
• Expect eventual consistency when distributing data
• Use idempotency for retried asynchronous operations
• Use timeouts and circuit breakers for remote calls
• Prefer evolutionary architecture instead of predicting every future requirement

Final Architecture Thinking#

Think about the evolution like this:

10K USERS
 
Simple architecture
      |
      ↓
Modular Monolith
      |
      ↓
Measure
      |
      ↓
Horizontal Scaling
      |
      ↓
Caching
      |
      ↓
Database Optimization
      |
      ↓
Queues / Workers
      |
      ↓
Read Replicas
      |
      ↓
Identify Hotspots
      |
      ↓
Extract Services
      |
      ↓
10M USERS

Not:

10K users
   ↓
Microservices
   ↓
Kubernetes
   ↓
Kafka
   ↓
Service Mesh
   ↓
20 Databases
   ↓
Distributed System Problems

Complexity should be introduced because it solves a measured problem.

Not because it looks scalable on an architecture diagram.

Once services are extracted, independent deployment is not the same as failure isolation. You still need bounded concurrency and admission control to keep slow dependencies from overwhelming the whole request path, plus a retry policy that cannot turn a partial outage into a retry storm.

Conclusion#

If I were building a system expected to grow from 10,000 to 10 million users, I would usually not start with microservices.

I would start with:

Modular Monolith
+
Clear Domain Boundaries
+
Stateless Application
+
Well-Designed Database
+
Observability

Then scale gradually:

Vertical Scaling
        ↓
Horizontal Scaling
        ↓
Redis
        ↓
CDN
        ↓
Queues
        ↓
Workers
        ↓
Read Replicas
        ↓
Service Extraction

The goal is not to avoid microservices forever.

The goal is to introduce them when their benefits become greater than their operational cost.

Good system design is not about choosing the most advanced architecture.

It is about choosing the simplest architecture that satisfies today's requirements while leaving a clean path for tomorrow's scale.

Remember this line:

Design the boundaries early. Distribute the system when you actually need to.

References#

• Decompose by business capability — AWS Prescriptive Guidance
• Handling overload — Google Site Reliability Engineering