An API gateway that only routes requests isn't earning its place in the architecture — a load balancer does that for free. The gateway earns its keep when it centralizes the three things every downstream service would otherwise have to reimplement badly and inconsistently: who is this caller (OAuth2), how fast are they allowed to go (Redis), and what happened, durably, for everything downstream that cares (Kafka). This is the shape I've landed on architecting gateways for high-throughput banking and fintech platforms, and it holds up well beyond that domain.
OAuth2 at the gateway: verify once, trust downstream
The mistake I see most often is every microservice independently validating JWTs against the identity provider's JWKS endpoint. It works, but it means N services all doing the same cryptographic verification, all independently caching (or not caching) the signing keys, all independently getting token-expiry edge cases slightly wrong.
Push token validation to the gateway, once:
@Component
public class OAuth2GatewayFilter implements GlobalFilter, Ordered {
private final JwtDecoder jwtDecoder; // backed by a cached JWKS provider
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
String token = extractBearerToken(exchange.getRequest());
if (token == null) {
return unauthorized(exchange);
}
return Mono.fromCallable(() -> jwtDecoder.decode(token))
.onErrorResume(JwtException.class, e -> Mono.empty())
.flatMap(jwt -> {
ServerHttpRequest mutated = exchange.getRequest().mutate()
.header("X-User-Id", jwt.getSubject())
.header("X-Token-Scopes", String.join(",", jwt.getClaimAsStringList("scope")))
.build();
return chain.filter(exchange.mutate().request(mutated).build());
})
.switchIfEmpty(unauthorized(exchange));
}
@Override
public int getOrder() { return -100; } // run before rate limiting and routing
}Downstream services trust the gateway's X-User-Id/X-Token-Scopes headers instead of re-verifying a signature — which only works if the network between gateway and downstream is actually trusted (a private VPC/service mesh, not the public internet re-entering at each hop). Cache the JWKS response with a sane TTL (5–10 minutes is typical) and a background refresh; don't fetch it per-request, and don't let a slow JWKS fetch become a synchronous dependency on every single API call.
For service-to-service calls behind the gateway, client-credentials grant with short-lived tokens (5–15 minutes) beats static API keys — no key rotation runbook, no long-lived secret sitting in a config map waiting to leak.
Redis: two completely different jobs, don't conflate them
Rate limiting. A fixed-window counter is the wrong default — it lets a client burst 2x their limit across a window boundary. Sliding-window-log is accurate but expensive at scale. The practical middle ground is a token bucket, implemented as a single atomic Lua script so the check-and-decrement doesn't race under concurrent requests from the same client:
-- KEYS[1] = bucket key, ARGV[1] = capacity, ARGV[2] = refill rate/sec, ARGV[3] = now
local bucket = redis.call('HMGET', KEYS[1], 'tokens', 'last_refill')
local tokens = tonumber(bucket[1]) or tonumber(ARGV[1])
local last_refill = tonumber(bucket[2]) or tonumber(ARGV[3])
local elapsed = tonumber(ARGV[3]) - last_refill
local refilled = math.min(tonumber(ARGV[1]), tokens + elapsed * tonumber(ARGV[2]))
if refilled < 1 then
return 0
end
redis.call('HMSET', KEYS[1], 'tokens', refilled - 1, 'last_refill', ARGV[3])
redis.call('EXPIRE', KEYS[1], 3600)
return 1Key the bucket by client ID (from the OAuth2 token, not the IP — IPs are shared behind NAT and CGNAT far more often than teams expect), and run Redis in cluster mode with the bucket keys hashed across shards so no single node becomes the rate-limiting bottleneck for your highest-traffic client.
Response/session caching is a separate concern with a separate TTL strategy — cache idempotent GET responses keyed on the full request (path + query + relevant auth scope), invalidate explicitly on writes that affect that resource rather than relying purely on TTL expiry, and never let the rate-limit Redis instance and the response-cache Redis instance be the same cluster under load — a cache-heavy read spike shouldn't be able to degrade your ability to enforce rate limits.
Kafka: decoupling the gateway from everything that cares what happened
The gateway sees every request. That makes it the natural place to emit an event — not to synchronously call an audit service, a metrics service, and a fraud-detection service on the request's hot path, which just turns three optional downstream dependencies into three ways to make every single API call slower or fail outright.
Publish a lightweight gateway.request.completed event asynchronously after the response is already on its way back to the caller:
producer.send(new ProducerRecord<>("gateway.request.completed", clientId,
GatewayEvent.newBuilder()
.setClientId(clientId)
.setPath(path)
.setStatusCode(statusCode)
.setLatencyMs(latencyMs)
.setTimestamp(Instant.now())
.build()));Partition by client ID, not randomly — it keeps one client's event ordering intact for any downstream consumer building per-client usage timelines, and it means a hot client's volume is at least predictable per-partition rather than smeared unpredictably across the whole topic. Set acks=1 here, not acks=all: this is telemetry, not a financial transaction — losing an occasional audit event to a broker failure is an acceptable trade for not adding synchronous replication latency to every gateway request. (If you're also using Kafka to move the actual financial/business events — the payment or the state change, not the telemetry about the API call — that's a different topic, a different acks setting, and it deserves its own write-up.)
Give every consumer group its own dead-letter topic and a bounded retry count before parking a poison message there instead of blocking the partition indefinitely — one malformed event from a client with a slightly-off SDK version shouldn't be able to stall the entire audit pipeline behind it.
How it scales horizontally
Every piece of this is designed to add capacity by adding instances, not by growing any single node:
- Gateway instances are stateless — no local session, no local rate-limit counters (that's what Redis is for) — so they scale behind a load balancer with nothing more coordinated than a health check.
- Redis runs clustered, sharded by client ID for rate limiting so no single client's traffic can concentrate on one node.
- Kafka partitions scale independently of gateway instance count; add partitions (and consumer instances) as event volume grows, not as gateway replica count grows — they're not coupled.
The gateway ends up doing three things well instead of routing being the only thing it does at all: proving who's calling, keeping any one caller from drowning the rest, and giving everything downstream a durable, async record of what happened — without any of those three concerns becoming a synchronous dependency of the other two.