Imagine a popular nightclub with a strict fire-safety limit of 100 people. At the door stands a bouncer with a clicker. Every time someone walks in, click. When the count hits 100, nobody else gets in until people leave.
A rate limiter is that bouncer, but for software. I built one called Sentinel, and this post explains what it does and the tricky problems I had to solve, without assuming you write code.
Why would a company need a bouncer?
Modern apps lean on outside services. A fintech app might call a bank's API to check balances. A delivery app calls a logistics API for tracking. An AI product calls a model provider. These outside services almost always come with a quota: "you may make 100 requests per minute." Go over it and you get blocked, charged extra, or in the case of some banking partners, penalised.
Now picture a company with ten different internal services, all calling the same bank API. None of them knows what the others are doing. Each one thinks it's being careful, but together they blow past the limit.
The fix is to put one bouncer in front of the door. Before any service calls the bank, it asks Sentinel: "Am I allowed?" Sentinel answers yes or no in about a millisecond, and keeps the official count for everyone.
Problem 1: Two people at the door at the same time
Here's the trap that catches most simple rate limiters.
Say the count is at 99 out of 100. Two requests arrive at the exact same instant. Both look at the counter, both see 99, both think "there's room for one more," and both walk in. Now you're at 101. At high traffic this doesn't happen once; it happens constantly.
In real life, you'd solve this by only letting one person talk to the bouncer at a time. Sentinel does the same thing using a small script that runs inside Redis, a very fast database that keeps data in memory. Redis runs each script from start to finish without letting anything else interrupt it. So "look at the count, decide, and add one" happens as a single, unbreakable step. Two people can never read the same "99."
I didn't just trust this; I tested it. The test fires 150 requests at exactly the same time against a limit of 100, spread across three copies of the service. It passes only if exactly 100 are allowed and 50 are refused. Not 101. Not 99.
Problem 2: The "midnight rush"
A naive limiter resets its count at fixed times, like every minute on the minute. That creates a loophole: someone can send 100 requests at 11:59:59 and another 100 at 12:00:00, squeezing 200 requests into two seconds.
Sentinel uses a sliding window instead. Rather than asking "how many requests since the minute started?", it asks "how many requests in the last 60 seconds, right now?" It blends the current minute's count with a share of the previous minute's, based on how far into the minute we are. The door doesn't magically empty on the hour anymore.
Problem 3: What if the bouncer's clicker breaks?
Redis is where the official count lives. So what happens if Redis goes down?
There are two honest answers, and which one is right depends on the business:
- Keep the door open (fail-open). Let traffic keep flowing so customers aren't affected, and accept a small risk of going over the limit.
- Close the door (fail-closed). Refuse everything until the count is reliable again. This is the right call for something like a banking API where going over has real financial penalties.
Sentinel lets each client choose. For "fail-open" clients, each copy of the service switches to a small, cautious backup counter in its own memory. It's set to allow about half the normal quota across all copies combined, so business keeps moving but overshoot stays limited. For "fail-closed" clients, the answer is simply no.
It also doesn't waste time knocking on a broken door. After three failed attempts to reach Redis, Sentinel stops trying for a couple of seconds (this is called a circuit breaker, like the one in your house), then checks again. When Redis comes back, everything returns to normal on its own within about two seconds.
Problem 4: Keeping records without slowing down
Companies often need a record of every request for billing and analytics. But writing to a permanent database on every single request would slow the bouncer down.
So Sentinel takes quick notes and hands them off. Each decision is dropped into a fast queue, and a separate worker collects those notes in batches and saves them to a proper database later. The bouncer never waits for the paperwork.
The worker is also built so that if it crashes halfway through, it picks up where it left off without saving anything twice. I tested this directly too: send 40 requests, kill the worker mid-way, restart it, and confirm the database ends up with exactly 40 records. No gaps, and nobody gets billed twice.
A dashboard you can actually read
Finally, each client gets a dashboard showing their live usage against their quota, plus trends over the last 10, 15, or 30 days. Admins can change a client's limit and the change takes effect immediately, without restarting anything.
The takeaway
A rate limiter sounds like a small thing: count requests, say yes or no. The hard part is doing it correctly when lots of requests arrive at once, quickly enough that nobody notices it's there, and safely when parts of the system fail. Most of my time on Sentinel went into those three words, and into tests that prove it rather than hoping it works.