As engineering leaders, our job is to know how we can set up our organization for success and how to deliver impact to our end customers. This means being able to juggle multiple areas within the organization. From my chats with hundreds of engineering leaders, I see five topics that are consistently on their minds: Organization, Velocity, Quality, Outcome, and Impact. I believe these are the pillars of great engineering leadership.
Today I will walk you through the five building blocks that I live by and the questions I suggest continuously asking yourself as an engineering leader.
Some of these questions come from an engineering leader I respect a lot, Paul Popper, Head of Engineering at Faire, one of the fastest-growing companies of all time. I have found the answers to the questions are essential for building a road to success, and they will also help engineering teams draw a line from their work to customer impact.
🎯 These questions are relevant for every level of engineering leadership, but some will be explored differently based on your role. Remember this: Great engineering leaders are not dropping any of these. They are dedicating thinking cycles to them and more than anything, they understand that they need to improve continuously.
This is part 2 of my series on a Great Engineering Leadership. Make sure you read the 7 Mental Models For Great Engineering Leadership and The Engineering Leader's Process For Continuous Improvement for a more complete view of this topic.
The Questions You Should Continuously Ask Yourself as an Engineering Leader
🏛️ Organization
All organizational questions stem from “are we set up as an organization for success?” You should be asking yourself this question all the time, but mainly as you grow. Other questions, such as “is there too much interdependency between teams?” start at the top, because as a VP of Engineering, your job is all about dependency management. However, one question is universal: “Are the engineers happy?”
🏎️ Velocity
"How fast do we deliver?"; "Where are the bottlenecks?"; "How do we improve?" No matter if you're an engineering manager, team lead, or CTO, these questions need to be on your mind. However, they require different granularities at different job levels.
⭐ Quality
"What's the quality of the software that we deliver to our end user?" Everything that impacts the experience for the end-user: From bugs reported and how fast they're fixed to your uptime, performance, and reliability. This topic is key for all engineering leadership roles - at a team level scope for engineering leaders and more aggregate at the CTO level.
🎁 Outcome
"What are we actually delivering?"; "Where are we investing our time?"; "Are we dedicating 20% to paying down tech debt?". At the individual team level, the team and the engineering manager should focus on "what are the different types of work that we're delivering?" and "How are our initiatives ongoing?". You might be looking at the same questions at the executive level but a much more aggregated version.
💥 Impact
"What is the business impact of what we deliver?" - This is relevant for all engineering leaders, but usually, the amount of control that you have on impact gets smaller the closer you are to being an individual team member.
I will go deeper into this topic because I've seen the best organizations struggle with quantifying business impact for engineering.
If we talk about impact as a set of metrics, impact is elusive. It's tricky to measure. However, there are some parts of an engineering organization where you can very clearly see the impact of what you're doing: "We're AB testing this new feature that we're shipping. Is it going to move the needle on the product metric?" Very straightforward.
It becomes a little less clear when you're building a team that's developing the microservice responsible for something in the backend. The important thing is that when you sit at the top of the engineering org, you can tell whether this is meaningfully impacting the business.
Impact questions are harder to answer, but at the end of the day, any engineering team should be able to draw a line between what we're working on and shipping out.
💭 There is a question that should follow ALL of the topics above: "... and how do we improve?" You need to continuously ask yourself how to improve for all of these pillars. How do you improve velocity? How can you improve quality, impact, outcome? One of the biggest challenges for engineering is maintaining and increasing productivity/velocity as the team grows - keeping these questions in check will help with this.
Here's a quick cheat sheet on all of these:
.png)
How to think about these questions at different stages of growth
In the early stage, it’s all about shipping an application as quickly as possible and testing things. So at this stage, you should be spending most of your time thinking about velocity, “what can allow us to move faster?”
The moment you start getting to some stage of maturity and you have more (and bigger) customers, quality starts becoming more important. Organization questions are going to take up more of your time when you start undergoing hyper-growth,
Outcome is something that should always be looked at. However, when you're five people, you have a better understanding of where you're investing your efforts on. But when you have 100 engineers that becomes fuzzier. Same with impact. At an early stage, impact is easy to understand, later on, it becomes more difficult.
👉 When to ask: The cadence depends on your role and the context. Your quarterly and yearly planning meetings are a great time to go through the five pillars, as are your 1:1’s and team retrospectives. Your priorities should also guide how often and how many of these questions you ask. One thing should ALWAYS be top of mind: “How can we improve this.” It’s an easy one to remember and will ensure continuous improvement throughout your engineering organization.
What happens when you don’t ask these questions?
As an engineering leader, you are probably already asking yourself some variation of the questions above. After all, it is your job to answer most of these to keep our teams aligned and working towards a clear goal. So what does it look like when you’re not looking at those pillars?
If you’re never asking, “are we set up for success?” or “how fast are we going?” or “What’s the quality of what we deliver?” you’re essentially being led by a mythical hand, often a product leader just throwing tickets your way. You become an organization that just moves tickets from left to right. And that’s not what engineering is.
An engineer org that isn’t critically reflecting on itself isn’t leading itself. It leads to terrible developer experience and the inability to deliver on your planning and the software. Because they are not asking the right questions, these organizations face an all too familiar consequence: not scaling linearly as they grow. In fact, they often start slowing down as the number of engineers grows.
