{"term":"incident-review","results":[{"@id":"https://coco.clown.business/posts/multi-tank-ga","title":"Remediation complete: multi-tank helium is GA","description":"All four action items from the Hendricks PIR are closed. We now run two helium tanks in two separate minivans in an active-active configuration, with automated failover via my cousin Dale. Warm-standby poodles are pre-twisted at every event. The status page is live and currently shows all giraffes operational. Thanks to Giggles for being an extremely patient customer, and to the Hendricks family for agreeing to be a case study. Madison, 8, has asked to be cc'd on all future incident reviews. Approved.","image":null,"date":1791277708030,"postDate":1787842800000,"octothorpes":["balloon-art","incident-review","resilient-infrastructure"]},{"@id":"https://coco.clown.business/posts/hendricks-outage-review","title":"Post-incident review: the Hendricks 8th birthday balloon outage","description":"Blameless PIR for the August 8 incident. From 14:02 to 14:47 local time, balloon animal availability at the Hendricks 8th birthday was 0%. 23 attendees affected, plus one parent who asked for a sword. Giggles Gambino was on call and handled comms with professionalism and a kazoo. Root cause: July's helium consolidation left us with a single point of inflation. The tank's regulator froze during peak demand (the giraffe queue), and there was no failover. I own that decision. I said consolidation was maturity. It was a single tank. Action items: (1) multi-tank architecture, two tanks in two separate minivans; (2) a public status page; (3) pre-twisted warm-standby poodles at every event; (4) never again use the phrase “one source of truth” about a gas.","image":null,"date":1791277701590,"postDate":1786384800000,"octothorpes":["balloon-art","incident-review","resilient-infrastructure"]}]}