API Gateway and BFF

Every platform grows an edge service: one public entry point that terminates authentication, applies limits, shields clients from failing backends and sometimes reshapes responses for a specific frontend (a backend for frontend, or BFF). You will build one with Spring Cloud Gateway in front of two small stub services. Writing it yourself demystifies what Nginx, Envoy and Kong do and forces you to reason about timeouts, retries, hop-by-hop headers and non-blocking I/O in one place.

Target behaviour

# routing + JWT: identity is passed to the upstream, the token is not
curl -i -H "Authorization: Bearer $JWT" localhost:8080/api/users/42
# 200  X-Correlation-Id: 7d1e...   (upstream saw X-User-Id: alice, no Authorization)

curl -i localhost:8080/api/users/42
# 401 WWW-Authenticate: Bearer

# rate limit per user
for i in $(seq 1 30); do curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $JWT" localhost:8080/api/users/42; done
# 200 200 200 ... 429 429

# a client-supplied correlation id is kept and returned
curl -i -H "X-Correlation-Id: order-77" -H "Authorization: Bearer $JWT" localhost:8080/api/users/42
# 200  X-Correlation-Id: order-77

# upstream down: circuit opens, the fallback answers fast
curl -i -H "Authorization: Bearer $JWT" localhost:8080/api/orders/1
# 503 {"error":"orders_unavailable","retryable":true}

# BFF: one call, three services in parallel, optional part degrades
curl -i -H "Authorization: Bearer $JWT" localhost:8080/bff/dashboard
# 200 X-Partial: feed
# {"user":{"name":"Alice"},"stats":{"orders":12},"feed":[]}

Architecture

client -> [correlation id] -> [JWT auth] -> [rate limit (Redis)] -> route match
                                                                       |
            /api/users/**  -> Retry -> CircuitBreaker(users)  -> users:8081
            /api/orders/** -> Retry -> CircuitBreaker(orders) -> orders:8082
            /bff/dashboard -> BffController --Mono.zip--> users | stats | feed(optional)
                                   (every hop carries X-Correlation-Id and X-User-Id)

Gateway is built on WebFlux and Netty, so a few event-loop threads multiplex thousands of connections; that fits a proxy, which only waits on I/O. The cost is that anything blocking inside a filter (a JDBC call, Thread.sleep, a synchronous HTTP client) stalls every connection on that thread. Authentication happens once at the edge and backends trust the identity headers, which is only safe if they are reachable from the gateway alone.

The core of it

Routes, timeouts, retry and breaker wiring live in YAML. Filters run in the order declared, so Retry listed first is the outer layer and every attempt passes through the breaker.

spring:
  cloud:
    gateway:
      httpclient: { connect-timeout: 1000, response-timeout: 3s }
      routes:
        - id: users
          uri: ${users.url:http://localhost:8081}
          predicates: [ Path=/api/users/** ]
          filters:
            - StripPrefix=1
            - name: Retry
              args: { retries: 2, methods: GET, statuses: BAD_GATEWAY,
                      backoff: { firstBackoff: 50ms, maxBackoff: 500ms, factor: 2 } }
            - name: CircuitBreaker
              args: { name: users, fallbackUri: "forward:/fallback/users" }
            - name: RequestRateLimiter
              args: { redis-rate-limiter.replenishRate: 10, redis-rate-limiter.burstCapacity: 20,
                      key-resolver: "#{@userKeyResolver}" }

Retrying only idempotent methods matters: retrying a POST that timed out after the upstream committed creates duplicates. Resilience4j reads its own settings; note that the time limiter defaults to one second, not your route timeout.

resilience4j:
  circuitbreaker:
    instances:
      users:
        slidingWindowSize: 20
        minimumNumberOfCalls: 10
        failureRateThreshold: 50
        waitDurationInOpenState: 20s
  timelimiter:
    instances:
      users: { timeoutDuration: 3s }

Validate JWTs with the standard resource-server support and leave only health endpoints open. The gateway needs no user database, only the issuer's public keys.

@Bean
SecurityWebFilterChain security(ServerHttpSecurity http) {
    return http.csrf(ServerHttpSecurity.CsrfSpec::disable)
        .authorizeExchange(a -> a
            .pathMatchers("/actuator/health/**", "/fallback/**").permitAll()
            .anyExchange().authenticated())
        .oauth2ResourceServer(o -> o.jwt(Customizer.withDefaults()))
        .build();   // spring.security.oauth2.resourceserver.jwt.jwk-set-uri in yaml
}

@Bean KeyResolver userKeyResolver() {
    return ex -> ex.getPrincipal().map(Principal::getName).defaultIfEmpty("anonymous");
}

The correlation id filter runs first. It keeps a well-formed client id, otherwise creates one, adds it to the forwarded request and echoes it in the response.

@Component
public class CorrelationIdFilter implements GlobalFilter, Ordered {
    public static final String HEADER = "X-Correlation-Id";

    @Override
    public Mono<Void> filter(ServerWebExchange ex, GatewayFilterChain chain) {
        String id = Optional.ofNullable(ex.getRequest().getHeaders().getFirst(HEADER))
                .filter(v -> v.matches("[A-Za-z0-9-]{8,64}"))
                .orElseGet(() -> UUID.randomUUID().toString());
        ServerWebExchange out = ex.mutate()
                .request(r -> r.headers(h -> h.set(HEADER, id))).build();
        out.getResponse().getHeaders().set(HEADER, id);
        return chain.filter(out);
    }

    @Override public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; }
}

A second global filter turns the validated token into trusted headers. It deletes any inbound X-User-Id first, otherwise a client could impersonate anyone.

@Component
public class IdentityFilter implements GlobalFilter, Ordered {
    @Override
    public Mono<Void> filter(ServerWebExchange ex, GatewayFilterChain chain) {
        ServerWebExchange clean = ex.mutate().request(r -> r.headers(h -> h.remove("X-User-Id"))).build();
        return ex.getPrincipal().ofType(JwtAuthenticationToken.class)
            .map(t -> clean.mutate().request(r -> r.headers(h -> {
                h.set("X-User-Id", t.getToken().getSubject());
                h.remove(HttpHeaders.AUTHORIZATION);          // terminate the token here
            })).build())
            .defaultIfEmpty(clean)
            .flatMap(chain::filter);
    }
    @Override public int getOrder() { return Ordered.LOWEST_PRECEDENCE - 10; }
}

The BFF endpoint calls the services concurrently, so latency is the slowest call rather than the sum. A required service failing fails the request; the optional feed degrades to an empty list and is reported in X-Partial.

@GetMapping("/bff/dashboard")
Mono<ResponseEntity<Map<String, Object>>> dashboard(@RequestHeader(CorrelationIdFilter.HEADER) String cid,
                                                     @AuthenticationPrincipal Jwt jwt) {
    Mono<JsonNode> user  = get("/v1/me", users, cid, jwt).timeout(Duration.ofSeconds(2));
    Mono<JsonNode> stats = get("/v1/summary", statsClient, cid, jwt).timeout(Duration.ofSeconds(2));
    Mono<Optional<JsonNode>> feed = get("/v1/recent", feedClient, cid, jwt)
            .timeout(Duration.ofMillis(800)).map(Optional::of).onErrorReturn(Optional.empty());
    return Mono.zip(user, stats, feed).map(t -> {
        var body = new LinkedHashMap<String, Object>();
        body.put("user", t.getT1());  body.put("stats", t.getT2());
        body.put("feed", t.getT3().orElse(JsonNodeFactory.instance.arrayNode()));
        var res = ResponseEntity.ok();
        if (t.getT3().isEmpty()) res.header("X-Partial", "feed");
        return res.body((Map<String, Object>) body);
    });
}

The get helper wraps WebClient, copies X-Correlation-Id and X-User-Id onto the outgoing call and ends with retrieve().bodyToMono(JsonNode.class).

Fallbacks are ordinary controller methods reached through forward:. FallbackController maps /fallback/{service} to a 503 with Retry-After: 10 and the JSON body {"error":"<service>_unavailable","retryable":true}: a fast, honest answer, not a fake success.

Observability: add spring-boot-starter-actuator, micrometer-registry-prometheus and micrometer-tracing-bridge-otel, expose health,prometheus,gateway, and the route-level spring_cloud_gateway_requests timer and Resilience4j resilience4j_circuitbreaker_state gauge appear on /actuator/prometheus.

Testing

Stub the upstreams with WireMock and drive the real gateway through WebTestClient. The aggregation test proves parallelism by timing.

@SpringBootTest(webEnvironment = RANDOM_PORT,
        properties = {"users.url=http://localhost:${wiremock.server.port}", "feed.url=http://localhost:${wiremock.server.port}"})
@AutoConfigureWireMock(port = 0)
@AutoConfigureWebTestClient
class GatewayIT {
    @Autowired WebTestClient web;

    @Test
    void aggregatesInParallelAndDegradesOptionalPart() {
        stubFor(get("/v1/me").willReturn(okJson("{\"name\":\"Alice\"}").withFixedDelay(400)));
        stubFor(get("/v1/summary").willReturn(okJson("{\"orders\":12}").withFixedDelay(400)));
        stubFor(get("/v1/recent").willReturn(serverError()));

        long start = System.nanoTime();
        web.mutateWith(mockJwt().jwt(j -> j.subject("alice"))).get().uri("/bff/dashboard")
            .header("X-Correlation-Id", "test-corr-1").exchange()
            .expectStatus().isOk()
            .expectHeader().valueEquals("X-Partial", "feed")
            .expectHeader().valueEquals("X-Correlation-Id", "test-corr-1")
            .expectBody().jsonPath("$.stats.orders").isEqualTo(12);
        assertThat(Duration.ofNanos(System.nanoTime() - start)).isLessThan(Duration.ofMillis(750));
        verify(getRequestedFor(urlEqualTo("/v1/me")).withHeader("X-User-Id", equalTo("alice")));
    }
}

Add a second test where the upstream returns 500 ten times, then assert the next call gets the 503 fallback without reaching WireMock.

Bugs you will hit

• Blocking code on the event loop. A JDBC call or RestTemplate inside a GlobalFilter freezes all requests on that Netty thread, and latency spikes appear under load only. Keep filters non-blocking; if you must block, wrap it in Mono.fromCallable(...).subscribeOn(Schedulers.boundedElastic()).
• Everything times out at one second. The Resilience4j TimeLimiter defaults to 1 s and cancels calls long before your response-timeout. Set resilience4j.timelimiter.instances.<name>.timeoutDuration at least as large as the route's response timeout.
• Client-forged identity headers. If you only add X-User-Id when a token exists, an unauthenticated or custom-header request can still pass its own value through. Always remove inbound identity headers first, and firewall the backends so only the gateway reaches them.
• Lost correlation id in logs. MDC is thread-local and reactive pipelines hop threads. Call Hooks.enableAutomaticContextPropagation() with io.micrometer:context-propagation on the classpath and register the correlation id as a ThreadLocalAccessor; otherwise only headers carry it, not log lines.
• Retries multiply the load. Retrying 3 times behind 3 layers of retries becomes 27 calls against a struggling service. Retry in one layer only, restrict to GET, use backoff, and let the circuit breaker open to cut the storm.

Interview angle

• API gateway versus BFF? A gateway is a generic edge: routing, auth, limits and resilience for all clients. A BFF is owned by one frontend team and shapes responses (aggregation, trimming fields) for that client. They often live in the same process at first and are split once frontends diverge.
• Why Spring Cloud Gateway over Nginx or Envoy? It lets you write filters in Java with your existing JWT, Micrometer and Resilience4j stack, and test them as code. Nginx and Envoy are faster, lighter, battle-tested at huge scale and configured declaratively, so they suit pure routing and TLS termination; Envoy adds dynamic xDS configuration and a service mesh role.
• How does a circuit breaker help, and what are its states? It stops calling a failing dependency so threads and connections are not wasted and the dependency can recover. Closed passes calls and counts failures; open fails fast; after the wait duration, half-open permits a few trial calls and closes on success or reopens on failure.
• Where do you validate authentication and authorization? Authenticate once at the edge (signature, expiry, issuer, audience), but backends still make authorization decisions on the resource and must not be reachable bypassing the gateway. The gateway is a single point of failure, so run several instances behind a load balancer and keep it stateless apart from Redis counters.