Razorpay

Common causes on Java 21: pinning from blocking inside synchronized (JDBC drivers, legacy libs), blocking native calls, a fixed-size pool of connections or a semaphore throttling everything, and ThreadLocal heavy caching (millions of copies). Diagnose with JFR event jdk.VirtualThreadPinned and -Djdk.tracePinnedThreads=full. Fixes: use ReentrantLock, upgrade the library, or Java 24+.

An operation is idempotent if repeating it has the same effect as doing it once. Networks time out and clients retry, so a "create payment" call must not charge twice. Use a client-supplied idempotency key enforced by a unique constraint.

CREATE TABLE payment (
  id bigserial PRIMARY KEY,
  idempotency_key text NOT NULL UNIQUE,
  amount numeric(12,2) NOT NULL,
  status text NOT NULL,
  response jsonb
);

INSERT INTO payment (idempotency_key, amount, status)
VALUES ('k-123', 50.00, 'PENDING')
ON CONFLICT (idempotency_key) DO NOTHING;   -- 0 rows => replay: return stored response

• Insert the key and perform the side effect in the same transaction where possible; for external calls (PSP) pass the key downstream too.
• Store the response so a retry returns the same result; reject same key with a different payload (422).
• Expire keys after a window (e.g. 24 h).
• MySQL equivalent: INSERT ... ON DUPLICATE KEY UPDATE or INSERT IGNORE.
• PUT and DELETE are idempotent by definition; POST is not and needs the key.

Accept payment requests with an idempotency key, orchestrate calls to a PSP through a state machine, and record every money movement in an append-only double-entry ledger with periodic reconciliation.

• Requirements: charge, refund, multiple payment methods, exactly-once effect, auditability, PCI compliance, high consistency.
• Estimates: 1M payments/day (~12/s, peak 100/s); low QPS, extreme correctness requirement.
• API: POST /v1/payments {orderId, amount, currency, method} with Idempotency-Key header; GET /payments/{id}; webhooks from PSP.
• Data model: payments(id, order, amount, state, idem_key UNIQUE), ledger_entries(id, txn_id, account, debit/credit, amount); amounts in minor units.

Checkout -> Payment API -> Payment DB (state) -> PSP Adapter -> PSP (Stripe/Adyen)
PSP webhook -> Webhook Handler (verify signature) -> update state -> Ledger -> Kafka events
Reconciliation Job: PSP settlement files vs ledger

• Idempotency: unique key stored with the request hash and response; replays return the stored result.
• State machine: CREATED -> PENDING -> SUCCEEDED/FAILED; timeouts are unknown outcomes, so query the PSP status before retrying and never blindly re-charge.
• Use the outbox pattern to publish events atomically with DB updates.
• Trade-offs: synchronous confirmation vs async webhook; never store raw card data (tokenize via PSP).
• Failure modes: PSP timeout (unknown state, poll and reconcile), duplicate webhooks (idempotent handler), ledger imbalance caught by reconciliation.