Server

Horizontal scaling

Horizontal scaling is handling more load by running more copies of the app behind a load balancer. Capacity grows by adding boxes, not by making one box larger.

How it is measured

Measure it as requests per second per instance and the number of instances. If doubling the instances from 4 to 8 takes you from 400 to 780 requests per second at the same p95, scaling is close to linear. If it gives 520, something shared is in the way.

Check for shared bottlenecks as you add copies: the database connection count, a single Redis, a shared file store, or session state held in memory on each node.

Worked example

A Magento store runs 3 app servers at 55 percent CPU and p95 of 640 ms. The team adds 3 more before a sale. Throughput rises from 210 to 395 requests per second, not 420, because sessions stored on local disk force sticky routing and some servers get hotter than others.

Moving sessions to Redis evens the load, and the next sale ran six servers each at 41 percent CPU. The database became the new ceiling at about 480 requests per second.

How it differs

Horizontal scaling adds more machines or containers. Vertical scaling gives one machine more CPU and RAM. Horizontal excludes a single point of capacity but requires a stateless app and a balancer. Vertical excludes that complexity but stops at the largest box you can rent, and a restart is needed to resize.

Common errors

Adding copies while sessions live in local memory. Scaling the app and leaving a single database. Counting copies without watching per-instance load. Forgetting each copy opens its own connections. Assuming it fixes one slow request, since more copies do not make any single request faster.

In practice

Before you need it, run two copies of your app behind a balancer in staging. Move sessions, uploads and cache out of the box, and see what breaks. The things that break are your real scaling limits.

See also

Vertical scaling, Load balancer

Sources

Count this on a real site.

Watch my website