Load Shedding
When 30k req/s hit a box that can only do 10k, extra servers take time and the database may not scale with you. Drop the overflow — keep checkout and payment, shed recommendations — via concurrency limits, bounded queues, health checks, or priority.
Application server
10000 req/sec
30000 req/sec
1. What about time of horizontal scaling? 2. What About you dont have that much resources 3. What about db scalling
Load Shedding
but i will serve 10000 req/sec
Smart Load Shedding
Imagine e-commerce app:
Checkout → HIGH priority
Payment → HIGH priority
Recommendations → LOW priority
Analytics → LOW priority
How to do Load Shedding
1. By concurreny limit 2. Bounded Queues 3. System Health 4. Priority Queue Shedding
Practically load shedding 4 ways se kar sakte ho
1. Concurrency limit
if activeRequests >= MAX:
2. Bounded Queue
Suppose worker 1000 requests process kar raha hai.
Queue capacity = 500
Queue bhi full?
New request reject.
Workers FULL
New Request ❌