I want to help folks. That’s one of my driving forces in software development. I don’t want to simply build for the sake of building, my code nothing more than a digital ouroboros. There needs to be a sense of purpose to what I’m investing my time in, something greater than “hey, look at this thing I made, aren’t I oh-so-clever?”. I’ve always found deep satisfaction in finding out when my work can truly help others. Otherwise, I’m writing code for the sake of writing code, and that way leads to petty arguments, a territorial “us” vs “them” attitude, rather than what does the most good.

I’ve workshopped this philosophy, in some nebulous form or another, as my approach to engineering for several years. It overlaps heavily with DevOps styles and can definitely take SRE approaches as an influencer. I’m not looking to coin a phrase – they quickly rise and fall with misinterpretations anyways. I will say that being without a term for it, it can be easy to hand wave exactly what I mean. This past weekend I feel like it has solidified in my head with a codification of the phrase Support Driven Engineering (SDE).

“How can I help?”

I love feeling useful. I think that’s a big reason I become a software engineer in the first place. At the core of everything I do, I keep a mental bread crumb trail back to the good I’m looking to accomplish. If I don’t know who I’m helping and knowing how I’m helping them, it’s not a project worth starting, maybe even worth quitting mid stream. The most beautifully designed architecture in the world doesn’t do us any good without the first principle of knowing the problem statement you’re trying to solve, and more specifically who you’re trying to solve it for. Otherwise, it’s self serving and self congratulatory. Not every line of code written has to go towards curing cancer, mind you. But there are enough projects out there looking to design for the sake of designing, to tear someone down, to prove one’s own worth. Poorly understood goals are often a major contributing factor to failed projects.

A quick tangent: I love games. I love board games and table top games, and I’ve been playing video games since the late 80’s, when I was finally old enough to pick up an Atari controller. Without a doubt my favorites are collaborative games, and I always fall into support roles. Whatever your team’s task – drive a payload against a competing team in an online FPS or maybe eradicating viruses in Pandemic, I tend towards positions that help other people in my crew do their jobs more effectively. For those Overwatch fans, I’m a Mercy main, primarily healing teammates and buffing their skills. When I playing D&D, I’m typically organizing them as the DM because I love creating a world for other people to play around in. Not everyone can, or should, looking for homeruns all the time.

Pulling back from metaphors, what I want to highlight is that there are among us folks who don’t need to be the mythical ninja-wizard-jedi engineer to be great at our jobs and be critical to the process. Some of us want to be racking up assists or laying the groundwork to build from, a safety net should something go wrong or need patching up. There’s satisfaction in improving the lives of those around us, being the best at what we do so others can be better at what they do. In short, I love playing the support role.

Years ago, I caught an amazing talk from Alice Goldfuss at Velocity Santa Clara called “Rock Stars, Builders, and Janitors: You’re doing it wrong” (take some time to watch that video, it’ll do you a lot of good). This was by far my favorite talk at that conference and one I reflect on years later. We overemphasize the rock stars who want to build the next latest-and-greatest project, only to see it wilt in production once the shine has left the apple. Often, others are required to clean up after the rock star leaves a mess. We simply don’t give enough credit to those who maintain/repair/iterate existing technologies (builders) nor those who organize our environments to make them much more sane to live in (the janitors). It’s hard to convince people to eagerly dive into work that gets less praise, more toil, and generally underappreciated.

And that’s where in my mind the rubber meets the road. We need an effective system for those of us who want to do this critical work.

Just Culture and Engineering

It’s easy enough to say you want to do good. The road to poorly maintained systems is paved with good intentions, after all. My desire to “just
be productive wasn’t enough, though, and trying to gain some direction from this was difficult.

During my time at Etsy, I was fortunate enough to work with a staggering number of compassionate individuals. I was made to feel welcome and given breadth to make mistakes. Those failures were never weaponized against me, but turned into lessons to grow. There’s an expectation of every day failures, great and small, that each of us would make but more often than not are innocuous. Why punish us for coming up short if we were trying our best? This rejection of Retributive culture (“rules are paramount – never break them or pay the consequences!”) for a more healthy Restorative Culture (aka Just Culture – “Who can we help?”) helped to solidify a lot of my thinking around all of this.

Just Culture has at its core three principles:

Who was hurt?
What do they need?
Whose obligation is it to meet that need?

I love so much about this, especially the requirement of empathy at a foundation for everything we do. It’s a constant challenge to push back on a learned response to failure, where we’re taught to bark orders and condescend to others as to how they broke a sacrosanct rule. It’s critical to pull that apart, understand how a set of rules might be insufficient (“No deploying without running the tests – ever!”) and see how we can understand the system we’ve put in place is insufficient for the needs of those around us.

I have two concerns as to where Just Culture comes up short: the focus on it as a means of a foundation for retrospectives/post mortems/incident reviews and who can meet the need. So much of this work is reactive, waiting for things to blow up or fail before we investigate what choices people are facing and how we might go about understanding their needs. Now, there are certainly important concepts for forward thinking work (game days, chaos experiments, etc.) and I’m incredibly hopeful for their progress in our industry. The field of resilience engineering is at its infancy and we’re looking at low hanging fruit. Unfortunately, that means we focus too much on the tooling and not enough on what peoples’ needs are. Likewise, the obligation to meet those needs makes it easy to sidestep work by saying “well, that sounds like a you problem and not a me problem”. That certainly isn’t inherent to Just Culture (and any good practitioners would push back on that), but it’s easy to slip in that direction.

What I want to see in resilience engineering is asking before an outage “What do you need? Who do you look to for that need? And how can I help bridge that gap?”.

Gallego as a Verb

When I kick off a new project, work my way around a problem, or even start a given work day, I’ll rely on these three questions to ground me. Sometimes that means going out of my way a bit to look for problems, a dangerous pastime to engage in. At Etsy, my name turned into a verb: “to Gallego oneself”. To do so is to talk yourself into taking on a task that you passively signed up for. An example:

Me: “Hey, the graph on this dashboard is looking a bit down week over week this quarter and the data team had some questions about it. Anyone know who owns this and why that might be? I think it might be related to this system…”

Coworker: “Yeah, we don’t have a maintainer for that, but it sounds like it could use some love. Why don’t you start checking it out!”

And just like that I have a new project.

Now, often that would be met with jeers and laughter, getting saddled with another project that was passed over because it was a bear to work on or low on glamor. In all honesty, I have to wonder how many of these projects turned yak shaves became burdensome to getting recognized for promotions. Picking your assignments, when applicable, is an art I have yet to fully master.

What I will say is that I built a reputation through this and one I’m damn proud of. If I’m asking these questions, it’s because I believe they need answering. And if I’m working on something, it’s because I wanted to make people’s lives better. I can be relied upon to get things done and soon after the tech debt we’re swimming in didn’t seem quite so deep. Was I standing a little taller to touch the floor, or was the water receding back a bit? Maybe a little of both. The point is I could make a dent, if only small, in setting expectations of what engineering could be. It doesn’t always have to be a raging tire fire we’re forced to abide.

Relating to DevOps and SRE

DevOps and SRE are about breaking down barriers to getting things done. In that sense, SDE is extending these principles in yet another proactive manner.

Part of DevOps is about avoiding the throwing-over-the-wall principle between teams and dismantling silos that reinforce bureaucracy and mistrust amongst engineers. A classic example of this older thinking in engineering is the following:

“I can’t work on this SQL query, that’s a job for our DBA’s”

This is, unfortunately, still more prevalent today than many would like. Specialization is great, but empathetic knowledge of the tasks required of other individuals is critical for cross team projects. With a DevOps mindset, not only should I have a strong desire to see a project through and communicate with others, I should have a strong desire to learn about the deeper requirements beyond the surface area required by my day to day. SDE takes this concept one step further. Not only do I need to see how your job works, I need to understand the context surrounding it. Maybe I can avoid future roadblocks by making an API more extensible or gain clearer vision of your timeline. Sometimes all that’s needed is a more caring attitude in knowing how much work others have laid out before them.

Likewise, the SRE philosophy centers around treating ops work as software problems. How are we dealing with toil, what are the issues we’re dealing with, and how can we build better tools to avoid these pain points. I’ve heard SRE described as a specific implementation of DevOps (I’m still very confused as to why some folks pit one against the other), so it’s natural that there’s alignment here as well to SDE. I come from a software engineering background before doing more infrastructure work and then later ops related work, so this approach in a lot of ways is natural. The continuation is therefore how can we make this traversal between the ops related work and the non-ops related work seamless. To do that, you have to make those empathetic bonds with the work of teams and teammates you may not have direct involvement with, but you possess either knowledge, skill, time, or energy to pass long. If I can ask what needs you have in Ops/Infra as a Dev, I bet I can build some really amazing stuff to help you in your day to day! Vice-versa, in my work in Ops/Infra, I can better understand the needs of say the product team and reduce friction to getting your great work out to production in stable and scalable system.

Core Concepts to SDE

I’ve described a lot of the foundational work others have developed that have deeply influenced my thinking on engineering, but haven’t yet succinctly described my own principles on SDE. So enough preamble, here’s what I believe:

People need help. This is universal, as no single individual can accomplish everything given constraints.
Help is hard to ask for, easier to give. Pull models to ask for support are good. Push models to offer it are way more effective and feel less burdensome for everyone involved.
Problems are easier to understand together. Collaboration can untangle problems better than a lone wolf approach.
Support builds trust, reduces conflict. Asking questions begets understanding. Understanding breeds empathy. Empathy reduces conflict.
Help can’t be forced. You can inspire others to care about one another, but you can impose it upon them. In other words, you can give people reasons to care but you can’t make them.
Cooperation is contagious. Compassionate people make others want to reciprocate, to repeat, and to replicate elsewhere.

Is this complete? No. These topics can be expanded upon and the list extended. Counterintuitively, I’d love to reduce the list into fewer, more potent bites as well. As you can tell by this post, brevity isn’t a particular strength of mine.

Isn’t this ripe for abuse?

Oh God yes. Both externally and internally, this can go off the rails big time. If you’re the grunt of your team/org/company who gets all the toil, it’s very easy for bad actors to abuse your best intentions. I’d love to say we live in a world where good people are treated well and we’ve filtered out malicious and self-interested folks, but it’s unfortunately not true. I’ve been stuck out on the proverbial ledge many times because I wanted to do good when reporting to managers who abused that trust.

Likewise, that abuse can be self inflicted! Feel like you’re biting off more than you can chew or you’re working longer hours to keep promises you made? You’re almost certainly overcommitting to work. If your process of reaching out to to others means taking on their workload (sometimes emotional toil), then you’re on a quick path to burn out.

To quickly answer (and leave it open ended for more later), it’s critical to remember three things:

Supporting abusers in a toxic relationship will not magically fix them. Offering help can even perpetuate toxicity. The behavior can be externally highlighted, but like caring for others, must be self driven.
Self care is essential. That means being reflective not just of what others are going through, but sometimes you need to ask yourself what you can do to better your own situation. You can’t be effective at helping others if you’re unable to help yourself.
SDE is a balancing act. Sometimes you’ll go too far by asking for more than you can handle. Often I’ll find colleagues who just need someone to rubber duck with rather than pick up the task to completion. Other times, it’s a yak shave that can completely derail my own projects. There is no silver bullet to solving this, but frequent reflection helps.

There’s without a doubt privilege baked into this as well. I’m a straight white dude compensated well enough to live comfortably and currently hold a fantastic job. The fact that I’m free and able enough to help carry the burden is proof. I’d argue that underrepresented groups have been practicing this for a long time – from improving hiring processes and resetting the balances in power throughout an organization to small things like taking notes in meetings and a billion other aspects I’m not actively aware of because of my own previously stated privilege. I’ll add to this by saying “Yes, and…”, that those of us with this privilege need to embrace these concepts because we’re in a place to do so. The advantages we possess morally oblige us to pay it forward. Specifically, the more senior, the more knowledgeable, and in general the more advantaged we are, the greater our obligation to extend support to those who need it.

Conclusion

This requires so much more expansion, discussing everything from examples of implementation, disadvantages of the approach and pitfalls to be wary of, and ways to meditate on this throughout the day. It’s emotionally driven, and that’s not always a bad thing, though it costs more than doing the bare minimum. I firmly believe there is a better way to do our work, one that asks more of a person than “What do I get out of this?”. It’s not perfect and should be scrutinized. I wouldn’t approach a code review by burying it, but exposing it so that it can be refined and improved upon.

It’s likely I’ll make a part two to extend these ideas, but considering the word count as it currently stands, I’ve put enough down for now. More reflection over some of these topics would do a lot of good as well. That said, I’d love to hear feedback to this and comparative approaches (lots of “Compassionate Coding” and the like seem to fall within this same realm). Fire away.

Support Driven Engineering (SDE)