An API gateway that only routes requests is not earning its place in the architecture — a load balancer does that for free. The gateway earns its keep when it centralises the three things every downstream service would otherwise reimplement badly and inconsistently: who is this caller, how fast are they allowed to go, and what happened, durably, for everything downstream that cares.

What follows is a design I would defend rather than a project write-up. The gateway work I have actually shipped — 5M+ requests a day on Resilience4j, Redis and Spring Cloud Gateway at a consumer product, and a custom Azure gateway at a commerce platform — is where the operational opinions here come from. The OAuth2 layer specifically is architectural guidance, not a claim about a system I built: treat it as the shape I would reach for, with the reasoning attached, and check it against your own identity provider before adopting it.

OAuth2 at the gateway: verify once, trust downstream

The mistake I see most often is every microservice independently validating JWTs against the identity provider's JWKS endpoint. It works, but it means N services all doing the same cryptographic verification, all independently caching (or not caching) the signing keys, all independently getting token-expiry edge cases slightly wrong.

Push token validation to the gateway, once:

@Component
public class OAuth2GatewayFilter implements GlobalFilter, Ordered {

    private final JwtDecoder jwtDecoder; // backed by a cached JWKS provider

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String token = extractBearerToken(exchange.getRequest());
        if (token == null) {
            return unauthorized(exchange);
        }
        return Mono.fromCallable(() -> jwtDecoder.decode(token))
            .onErrorResume(JwtException.class, e -> Mono.empty())
            .flatMap(jwt -> {
                ServerHttpRequest mutated = exchange.getRequest().mutate()
                    .header("X-User-Id", jwt.getSubject())
                    .header("X-Token-Scopes", String.join(",", jwt.getClaimAsStringList("scope")))
                    .build();
                return chain.filter(exchange.mutate().request(mutated).build());
            })
            .switchIfEmpty(unauthorized(exchange));
    }

    @Override
    public int getOrder() { return -100; } // run before rate limiting and routing
}

Downstream services trust the gateway's X-User-Id/X-Token-Scopes headers instead of re-verifying a signature — which only works if the network between gateway and downstream is actually trusted (a private VPC or service mesh, not the public internet re-entering at each hop). Strip those headers from inbound requests at the edge, unconditionally, or you have built a header-spoofing bypass rather than an authentication layer.

Cache the JWKS response and refresh it in the background rather than fetching per request, and do not let a slow JWKS fetch become a synchronous dependency of every API call. On expiry, note that RFC 7519 defines exp as a rejection boundary with implementation-defined leeway for clock skew — which is exactly the edge case N services each get slightly differently, and a reason to decide it in one place.

For service-to-service calls behind the gateway, the client-credentials grant with short-lived tokens beats static API keys: no rotation runbook, no long-lived secret sitting in a config map waiting to leak.

Redis: two completely different jobs, do not conflate them

Rate limiting. A fixed-window counter is the wrong default — it lets a client burst to twice their limit across a window boundary. A sliding-window log is accurate and expensive at scale. The practical middle ground is a token bucket, implemented as a single atomic Lua script so the check-and-decrement cannot race under concurrent requests from the same client:

-- KEYS[1] = bucket key, ARGV[1] = capacity, ARGV[2] = refill rate/sec, ARGV[3] = now
local bucket = redis.call('HMGET', KEYS[1], 'tokens', 'last_refill')
local tokens = tonumber(bucket[1]) or tonumber(ARGV[1])
local last_refill = tonumber(bucket[2]) or tonumber(ARGV[3])
local elapsed = tonumber(ARGV[3]) - last_refill
local refilled = math.min(tonumber(ARGV[1]), tokens + elapsed * tonumber(ARGV[2]))

if refilled < 1 then
  return 0
end

redis.call('HMSET', KEYS[1], 'tokens', refilled - 1, 'last_refill', ARGV[3])
redis.call('EXPIRE', KEYS[1], 3600)
return 1

The atomicity is the whole point, and it is worth knowing why a script rather than a MULTI/EXEC transaction: a transaction queues commands and cannot branch on a value read inside it, so “read the bucket, compute the refill, then decide” is not expressible. A script is. Note also that EXPIRE on every accepted request is what stops idle client keys accumulating forever.

Key the bucket by client ID from the token, not by IP — IPs are shared behind NAT and CGNAT far more often than teams expect — and run Redis in cluster mode with bucket keys hashed across shards so no single node becomes the bottleneck for your highest-traffic client.

Response and session caching is a separate concern with a separate TTL strategy: cache idempotent GET responses keyed on the full request (path, query and the relevant auth scope), invalidate explicitly on writes that affect that resource rather than relying purely on TTL expiry, and keep the rate-limit instance and the response-cache instance on different clusters. A cache-heavy read spike should not be able to degrade your ability to enforce rate limits.

Kafka: decoupling the gateway from everything that cares what happened

The gateway sees every request. That makes it the natural place to emit an event — not to synchronously call an audit service, a metrics service and a fraud-detection service on the request's hot path, which turns three optional downstream dependencies into three ways to make every API call slower or fail outright.

Publish a lightweight gateway.request.completed event asynchronously, after the response is already on its way back to the caller:

producer.send(new ProducerRecord<>("gateway.request.completed", clientId,
    GatewayEvent.newBuilder()
        .setClientId(clientId)
        .setPath(path)
        .setStatusCode(statusCode)
        .setLatencyMs(latencyMs)
        .setTimestamp(Instant.now())
        .build()));

Partition by client ID rather than randomly. Kafka orders records within a partition and not across a topic, so keying by client keeps one client's event ordering intact for any consumer building per-client usage timelines, and it makes a hot client's volume predictable per partition rather than smeared across the whole topic.

Set acks=1 here rather than acks=all: the leader acknowledges without waiting for the full in-sync replica set, which is a real durability trade — a leader failure immediately after the ack can lose the record. For telemetry that is acceptable, and it avoids adding replication latency to every gateway request. If you are also moving actual financial or business events through Kafka, that is a different topic, a different acks setting, and it deserves its own write-up.

Give every consumer group a dead-letter topic and a bounded retry count before parking a poison message there, instead of blocking the partition indefinitely. One malformed event from a client with a slightly-off SDK version should not stall the entire audit pipeline behind it.

How it scales horizontally

Every piece is designed to add capacity by adding instances, not by growing any single node:

  • Gateway instances are stateless — no local session, no local rate-limit counters, which is what Redis is for — so they scale behind a load balancer with nothing more coordinated than a health check.
  • Redis runs clustered, sharded by client ID for rate limiting so no single client's traffic concentrates on one node.
  • Kafka partitions scale independently of gateway instance count. Add partitions and consumer instances as event volume grows, not as gateway replica count grows — they are not coupled.

The gateway ends up doing three things well rather than routing being the only thing it does at all: proving who is calling, keeping any one caller from drowning the rest, and giving everything downstream a durable async record of what happened — without any of those three becoming a synchronous dependency of the other two.

Related reading