Deliverability Incident Response

Deliverability incidents come in exactly three shapes, and they do not overlap. Throttling: the provider defers with 4xx codes, your queue builds, and the right move is respectful retreat. Blocking: the provider rejects with 5xx codes, and the right move is to halt and remediate, because retrying through a block digs the hole. Bulking: the provider accepts everything with a clean 250 OK and quietly folders it, which means no bounce evidence at all, and the diagnosis lives in postmaster telemetry and engagement cohorts instead of logs. Classify first. Every hour spent remediating the wrong shape is an hour the incident compounds.

The first hour then runs in a fixed order: authentication and DNS before anything else, because a broken DKIM signature, SPF record, or PTR entry makes every other signal noise; blocklists second, queried through channels that actually answer; bounce-log classification third; provider telemetry fourth. The order exists because the classic misdiagnoses all come from skipping it. Teams burn a day rewriting subject lines to fix suspected content filtering while a DNS record sits broken. They read a wave of mailbox-full bounces as an IP block. On shared pools, they treat a noisy neighbor's listing as their own hygiene failure instead of escalating for a pool move. The checklist is boring, and boring is the point: the diagnosis order is the mechanism that survives the adrenaline.

I have run this under fire. During a DNS provider migration, five A records that looked like abandoned data-center entries were pruned from our zone. They were the forward half of forward-confirmed reverse DNS for the ESP's dedicated sending IPs, the reverse half living at the ESP where nobody could see the pairing. The next day, a holiday-weekend campaign to roughly 800,000 recipients bounced at just under 8 percent overnight, 550-class blocks at the largest provider and reverse-DNS-mismatch errors at the smaller ones. Working the triage order, the ESP's deliverability support confirmed broken FCrDNS within a day; recreating the forward records to match the existing reverse records exactly restored the round trip, and the whole thing resolved in roughly two days. It produced two rules I have kept ever since: before deleting any A record during DNS cleanup, look up the reverse DNS of its IP, because if the reverse points back at that record you are holding live sending infrastructure; and after any DNS migration, reconcile old zone against new, record by record, deliverability-critical entries first.

The other incident worth telling is public record, and it taught the response half. In January 2013, as the Senior Marketing Manager over Amazon's email channel, I watched two pre-staged congratulatory emails, one for each possible national championship winner, fire before the game even kicked off. A technical misfire sent both. Fans of both teams woke up congratulated for a title nobody had won, and ESPN and GeekWire had it within hours. I did not queue the sends; I was accountable for the channel, and I led the response. The follow-up went out the same day, opening with "Well this is embarrassing" and admitting we "completely fumbled our congratulation email for tonight's game." Candor and speed did what no spin could: the apology was received better than the mistake was resented. The postmortem over the following weeks produced mechanisms, not memos: mandatory double-verification for pre-scheduled high-visibility sends, system-level guardrails preventing simultaneous variant sends, and cross-functional training on both.

Remediation posture differs by provider, and knowing the doors matters as much as knocking. Google is algorithmic: the sender contact form feeds a model, nothing manual releases a block, and recovery follows corrected behavior. Microsoft is procedural: the exact NDR code dictates the channel, self-service portal for one range of codes, a manual delist mailbox for another, and using the wrong one guarantees failure. Yahoo sits between: support can review with an evidence dossier, no whitelisting exists. Blocklists are dataset-dependent, with self-service removal for some listings and manual evidence-driven review for others, and one operator policy is universal: fix the cause before requesting removal, because a re-listing after premature delisting burns the self-service option.

Recovery is warmup logic reapplied after trauma. And it is where stakeholder pressure peaks: marketing wants the volume back today, and pushing marginal traffic into a healing reputation resets the penalty clock. That conversation is Relevance over Frequency under fire, and the runbook exists partly so you can point at it instead of arguing from memory.

The lesson that has organized my incident thinking ever since: how you respond defines the experience more than the mistake itself.