San Jose, CA (2018-03-29)
Video
Description
SRE’s are frequently tasked with being front and center in intense, highly demanding situations in the production environment that require clear lines of communication. Our systems fail not because of a lack of attention or laziness but due to cognitive dissonance between what we believe about our environments and the objective interactions both internal and external to them. In this talk, I’ll discuss how we can revisit our established beliefs surrounding failure scenarios with an emphasis not on the who in decision making but the why behind those decisions. With this mindset, we can encourage our teams to reject shallow explanations of human error for said failures, instead focusing on how we can gain greater understanding of these complexities. I’ll walk through the structure of post mortems used at large tech companies with real world examples of failure scenarios and debunk myths regularly attributed to failures. Through these discussions, you’ll learn how to incorporate open dialogue within and between teams to bridge these gaps in understanding.
Write ups
Tanya Reilly – Conference report: SRECon Americas Day 3
Tweets
Watching @wcgallego talk about post mortem. It’s about telling the story. #SREcon pic.twitter.com/A7uKfrqVNq
— Michael Rembetsy (@mrembetsy) March 29, 2018
Blame-aware postmortems > “blameless postmortems”
I agree. 😎#SRECon pic.twitter.com/UsgxzXxdL6
— J. Paul Reed (@jpaulreed) March 29, 2018
Likes to start off post-mortems with that question: why are we here? Post-mortems are about story sharing/exchanging/syncing. #SREcon
— Kate Taggart (@qkate) March 29, 2018
Will Gallego on “fixed mindset vs growth mindset” in his technical post mortems talk. Reminds me of a great pair of books:
Mindset, by Carol Dweck <— the O.G. of growth mindset
Change Your Questions, Change Your Life, by Marilee Adams#SREcon @wcgallego— Baron Schwartz (@xaprb) March 29, 2018
“You don’t have to be a domain expert in an area to hold a postmortem in that area. In fact, sometimes it can be useful if you aren’t.”-@wcgallego #SRECon
— J. Paul Reed (@jpaulreed) March 29, 2018
Next: Postmortems should be open invite.
Everyone should be invited (particularly actors), but no one should be forced to attend.#srecon
— Murali Suriar (@msuriar) March 29, 2018
“When running a postmortem, we’re here to learn. Fixing things can be a nice side-effect.”-@wcgallego
Postmortems👏aren’t👏about👏filing👏Jira👏remediation👏tickets!
(Yes, I know it feels weird.)#SREcon
— J. Paul Reed (@jpaulreed) March 29, 2018
“I’m just gonna leave this slide up here for a minute…”-@wcgallego
😂😂😂#SREcon(Yes, it was @jasonhand and I clapping for this at the front of the room…) pic.twitter.com/klS9vdgRAQ
— J. Paul Reed (@jpaulreed) March 29, 2018
Keys to a good post-mortem: getting people to open up, question everything, say out loud even things they think are common knowledge. [Ed: oh man that last one is so important!] #SREcon
— Kate Taggart (@qkate) March 29, 2018
People often push back against this claim, to which I ask “heard of Knight Capital?”#SRECon pic.twitter.com/nnhK7Ygmqb
— J. Paul Reed (@jpaulreed) March 29, 2018