Order Service with Idempotency and Outbox
Build an order service that survives retries and concurrent updates using optimistic locking, idempotency keys and a transactional outbox.
Revise first: Transactions & PropagationHibernate & JPA InternalsTransactions & Isolation LevelsData Stores & Messaging
You will build an order service that creates orders, moves them through a state machine, and publishes OrderCreated events reliably. Three correctness problems drive the design: two writers updating the same order (optimistic locking), a client retrying a POST after a timeout (idempotency keys), and the dual-write problem of saving to the database and publishing to a broker (transactional outbox). Interviewers use this project to see whether you understand what @Transactional actually guarantees.
Project layout
src/main/java/com/example/orders/
api/ OrderController, CreateOrderRequest
domain/ Order, OrderStatus, OrderItem
idempotency/ IdempotencyRecord, IdempotencyRepository
outbox/ OutboxEvent, OutboxRepository, OutboxPublisher
service/ OrderService
src/main/resources/db/migration/V1__orders.sqlOptimistic locking with @Version
A @Version column makes Hibernate append AND version = ? to every UPDATE. If zero rows match, it throws OptimisticLockException, which Spring translates to ObjectOptimisticLockingFailureException. This prevents lost updates without holding row locks.
@Entity
@Table(name = "orders")
public class Order {
@Id @GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Version
private long version;
@Enumerated(EnumType.STRING)
private OrderStatus status = OrderStatus.NEW;
private BigDecimal total;
public void pay() {
if (status != OrderStatus.NEW) throw new IllegalStateException("Cannot pay from " + status);
status = OrderStatus.PAID;
}
}Map the failure to 409 Conflict in a @RestControllerAdvice and let the client re-read and retry. Do not blindly retry inside the service: the retry must reload the entity in a fresh transaction.
Idempotency keys
The client sends Idempotency-Key: <uuid> on POST /orders. Store the key with a unique constraint in the same transaction as the order, so a duplicate request either replays the stored response or loses the insert race.
CREATE TABLE idempotency_key (
key VARCHAR(64) PRIMARY KEY,
request_hash CHAR(64) NOT NULL,
order_id BIGINT REFERENCES orders(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);@Transactional
public Order create(String key, CreateOrderRequest req) {
String hash = Hashing.sha256(req);
Optional<IdempotencyRecord> existing = idem.findById(key);
if (existing.isPresent()) {
if (!existing.get().getRequestHash().equals(hash))
throw new IdempotencyKeyReuseException(key); // 422: same key, different body
return orders.getReferenceById(existing.get().getOrderId());
}
Order order = orders.save(Order.from(req));
idem.saveAndFlush(new IdempotencyRecord(key, hash, order.getId()));
return order;
}saveAndFlush forces the unique-constraint check now. Two concurrent first requests both pass the findById check; the loser gets DataIntegrityViolationException and rolls back, so catch it at the controller boundary, then re-run create to return the winner's order.
Transactional outbox
Calling kafkaTemplate.send inside a transaction is a dual write: if the commit fails after the send, consumers see an order that does not exist; if the send fails after the commit, the event is lost. Instead, insert an outbox row in the same transaction as the order, and publish from a separate poller.
@Transactional
public Order create(String key, CreateOrderRequest req) {
// ... idempotency check and order save as above ...
outbox.save(new OutboxEvent("order", order.getId().toString(),
"OrderCreated", json.write(order)));
return order;
}@Scheduled(fixedDelay = 500)
@Transactional
public void publish() {
List<OutboxEvent> batch = outbox.findUnsentForUpdateSkipLocked(PageRequest.of(0, 100));
for (OutboxEvent e : batch) {
kafka.send("orders", e.getAggregateId(), e.getPayload()).join();
e.markSent();
}
}@Query(value = """
SELECT * FROM outbox_event WHERE sent_at IS NULL
ORDER BY id LIMIT :n FOR UPDATE SKIP LOCKED
""", nativeQuery = true)
List<OutboxEvent> lockUnsent(@Param("n") int n);SKIP LOCKED lets several service instances poll without blocking each other. Delivery is at-least-once (a crash after send but before commit resends), so consumers must be idempotent; the event id is the dedupe key. Keying by aggregate id keeps events for one order in partition order.
Testing concurrency
Prove the version check works by loading the same row twice.
@Test
void secondStaleWriteFails() {
Long id = service.create("k1", sampleRequest()).getId();
Order a = orderRepo.findById(id).orElseThrow();
detach(a);
Order b = orderRepo.findById(id).orElseThrow();
b.pay();
orderRepo.saveAndFlush(b);
a.pay(); // stale version
assertThatThrownBy(() -> orderRepo.saveAndFlush(a))
.isInstanceOf(ObjectOptimisticLockingFailureException.class);
}Add a second test that fires 10 parallel create calls with the same key through an ExecutorService and asserts orderRepo.count() == 1.
Interview angle
- Why does
@Transactionalnot make the Kafka send safe? The broker is not enrolled in the database transaction; commit and publish are two independent steps that can each fail. The outbox turns it into one local transaction plus a retried relay. - Optimistic or pessimistic locking? Optimistic when conflicts are rare and you can retry or surface a 409; pessimistic (
SELECT ... FOR UPDATE) when contention is high or retries are expensive, such as stock decrement on a hot row. - How do you make a POST idempotent? Client-supplied key, unique constraint in the same transaction, and a request hash to reject key reuse with a different body. Keys expire after a retention window.
- Self-invocation of a
@Transactionalmethod? It bypasses the proxy, so no transaction starts. Call it through another bean or inject the proxy.
Milestones
Done when
- Two stale updates to one order cannot both succeed; the loser gets HTTP 409.
- Repeating a POST with the same key and body returns the original order without a second insert.
- The same key with a different body is rejected with 422.
- No order exists without its outbox row, and no outbox row exists without its order.
- Killing the service between
sendand commit results in a duplicate event, never a lost one. - Two running instances publish each outbox row at most once per poll cycle.
Stretch goals
- Replace polling with Debezium change data capture on the outbox table.
- Add a retention job that deletes sent outbox rows and expired idempotency keys.
- Add a
reserve stockstep as a saga with a compensating cancel. - Expose Micrometer metrics for outbox lag (oldest unsent row age) and 409 rate.