Skip to content
All writing
Software EngineeringJul 29, 2026·13 min read

Moving $10M a Month on Node.js and a Prayer

The unglamorous architecture that kept eight figures flowing every month: idempotency keys, double-entry thinking, and a nightly job that assumes everyone lied.

There's a specific feeling when you deploy code that moves real money. It's not excitement. It's the feeling of carrying someone else's wedding cake across a wet floor. The numbers on my work page — $10M+ a month, 99.9% uptime — look tidy in a metric card. This is the messy kitchen behind them.

None of what follows is clever. That's the point. Payments is the one domain where 'boring' isn't a compliment you give code — it's the requirement.

01

The stack was boring on purpose

Node.js services. MongoDB. Redis. A queue between anything that mattered. At a seed-stage company under deadline, you don't pick technology for the architecture diagram — you pick it for the 2am incident. Boring tech has known failure modes, and known failure modes have Stack Overflow answers. The exotic stuff fails in ways only its maintainers understand, and they're asleep in another timezone.

One rule mattered more than any framework choice: money math never touches floating point. Amounts live as integers in the smallest unit — paise, cents — from the API edge to the database. 0.1 + 0.2 not equaling 0.3 is a fun JavaScript quiz question right up until it's a customer's refund.

02

Idempotency: the only word that matters

Networks lie. Clients retry. Load balancers time out *after* the request succeeded. If your charge endpoint runs twice when called twice, you don't have a bug — you have a lawsuit generator. Every mutation that touches money carries a client-generated idempotency key, and the server treats a repeated key as 'return the previous answer, do nothing.'

charge.ts
// the key is the contract: same key, same outcome, exactly one side effect
async function charge(req: ChargeRequest) {
  const existing = await db.charges.findOne({ idempotencyKey: req.key });
  if (existing) return existing.response; // replay, not re-execute

  // reserve the key BEFORE side effects — a unique index makes the race safe
  await db.charges.insertOne({ idempotencyKey: req.key, status: 'pending' });

  const result = await provider.charge(req.amountMinor, req.currency);
  await db.charges.updateOne(
    { idempotencyKey: req.key },
    { $set: { status: 'done', response: result } }
  );
  return result;
}

Heads up

'It only ran once in testing' means nothing. Retries come from layers you don't control — mobile SDKs, proxies, the user's thumb. Design for the duplicate; it *will* arrive.

03

Double-entry thinking, yes, the 500-year-old kind

Venetian merchants figured this out before the printing press: money never appears or disappears, it only *moves*. Every transaction writes two ledger entries — a debit somewhere, a credit somewhere else — and the sum across the ledger is always zero. If it isn't, you don't have an accounting quirk, you have a bug with a dollar amount attached.

ledger.ts
// a payout is never 'update balance' — it's two entries that must cancel out
await ledger.write([
  { account: 'merchant:acme:payable', delta: -50_000, txId },
  { account: 'bank:settlement:outgoing', delta: +50_000, txId },
]);

// the invariant you check forever after:
// SUM(delta) over all entries === 0

The ledger is append-only. Nothing is ever edited — mistakes get *reversing entries*, like a real accountant would make. This felt ceremonial until the first dispute, when we could replay a merchant's entire history entry by entry and show exactly where every rupee went. That conversation lasted five minutes instead of five days.

04

Queues, not calls

A payment isn't one operation — it's a little pilgrimage: validate, reserve, charge, record, notify, settle. Chain those as direct calls and one slow provider takes the whole path down with it. Every step became a queue consumer instead: small, retryable, with its own dead-letter queue for the ones that failed three times and need a human.

Queues also give you backpressure for free. Provider having a bad day? The queue absorbs the spike, consumers grind through at whatever rate the provider tolerates, and nothing melts. Cron jobs batch problems into daily surprises; streams and queues hand them to you one at a time, while they're still small.

05

Reconciliation: assume everything lied

The most important code in the whole system ran at night and did nothing but check other code's homework. Pull the provider's settlement file, pull our ledger, diff them line by line. Every mismatch lands in a report a human reads over coffee. Most days the report was empty. The days it wasn't were the days reconciliation paid for itself — a webhook that never arrived, a double credit from a provider retry, a timezone bug that shifted a day's cutoff.

Trust your own system the way you trust a brilliant colleague who occasionally shows up without sleep: verify anyway.
06

What 99.9% actually cost

Three nines sounds humble next to the five-nines crowd, until you do the math: it's a budget of about 43 minutes of downtime a month, and every one of those minutes has a dollar figure attached. It cost us alerts tuned until pages meant something, deploys that always had a rollback, and a personal rule that no payment code ships on Friday. Not because of superstition — because Saturday on-call debugging a settlement mismatch is how you learn to hate your past self.

The full case study is on my work page under the fintech automation suite. But the honest summary fits in one line: the prayer was optional. The reconciliation job wasn't.

Key takeaways

  • 01Money math is integers in the smallest currency unit — floating point never touches an amount.
  • 02Every money mutation carries an idempotency key. Design for the duplicate request; it will arrive.
  • 03Append-only double-entry ledgers plus nightly reconciliation catch the failures you didn't predict.

FAQ

FintechArchitectureNode.js

Related reading