9 Metrics Questions Engineering Leaders Need To Ask
Why are metrics important to software engineering leaders?
Since performance metrics have become readily available to software engineering leaders, they use them to identify, track and communicate issues to their team. This allows them to increase the team’s productivity; while ensuring an effective project management and the prioritization of problems.
In their essence, performance metrics provide software engineering leaders with the necessary tools to:
- Manage current and predicted workloads,
- Reduce opportunity cost and overtime,
- Identify areas of improvement.
Are metrics only important to leaders?
No. Metrics are also important for the software development team as a whole. Engineers can use live metrics to communicate the current status of a project. Teams will benefit from this because it reduces the necessity to schedule progress-based meetings, which consume time and create an opportunity cost.
In addition, when software engineering teams are given autonomy they can use metrics to address any arising issues on their own. This allows software engineering leaders to focus on matters of company alignment, long-term visions and building organizational capabilities.
What questions should I ask myself to improve the performance of my software engineering team?
1. Are there any bottlenecks that are reducing the throughput of my team?
When you quantify your bottlenecks you can start measuring your improvement as a team. By identifying where work is getting stuck you can modify your process and improve continuously as a team. As a software engineering leader you need to understand in which stage of the software delivery pipeline work is getting stuck. This will allow you to:
- Understand the impact of changes,
- Unblock your team,
- Spot early warning signs,
- Understand where to invest your time in to improve.
2. Are pull requests too large and causing increasing code churn?
When a software engineering team faces big pull requests this becomes problematic.
- A reduced review quality: Reviewing thousands of lines of code, written in a different style by someone else, is difficult. It becomes hard to spot problems, suggest improvements and understand which parts of the code were affected.
- An increase for potential review errors: Many lines of code have many side effects and moving parts. This makes it difficult to test even with automated tests, given the many possible corner cases.
- A decrease in the quality of collaboration: When creating a large piece of code you naturally will become attached to it. This will create a personal resistance towards suggestions or improvements that would aim at improving the code.
- A delayed review: To review a large piece of code will take away time from other tasks, so reviewers would rather postpone it.
Therefore, if pull requests are too big they need to be reduced. With Athenian you can analyze how pull request size is affecting the lead time of your team and make appropriate decisions. This will reduce the team’s opportunity cost and increase its performance.
3. Is the assignment scope too large per software engineering team?
Understanding whether the amount of work per software engineering team is correctly distributed is important. If the amount of work delivered by your team does not meet expectations it could indicate two things: Either, (1) inefficiencies in the process, or (2) an insufficient backlog.
4. How can I improve the quality of communication with my team?
Communication between the team leads and the software development team depends on their ability to share information. By having access to live information on key performance indicators (KPIs) and performance metrics, teams can use data to support their communication on why, how and where to improve. When communication is inefficient and not transparent, it leads to contested opinions instead of data driven discussions. This harms the team’s velocity and increases its cost over time.
5. Is the team’s velocity decreasing?
Nowadays software development teams have a lot of pressure to become faster. But, if not enough attention is paid to quality your velocity will decrease, since velocity is not only speed, but speed in the right direction. Therefore, it is important to find the correct balance between the right velocity, while keeping up the quality of your team’s work.
If your team’s velocity is decreasing, it could be linked to hidden productivity issues.
6. What process in the software delivery pipeline represents the biggest proportion of our total cycle time?
Cycle time is defined as the time spent between beginning work and delivering it to the end customer. Understanding your cycle time, and what comprises it, is very important. It highlights inefficiencies within your software delivery pipeline and points out problems before they occur. By using our dashboard, you will be able to explore the 4 different stages comprising the commit-to-release pipeline. This will allow you to pin-point which stage is increasing your cycle time, and which stage requires most attention to reduce inefficiencies.
7. How can we increase deployment frequency?
A company’s deployment frequency provides evidence on the organization’s DevOps health, indicating a company’s actual speed. When a software engineering team deploys code they are delivering value to the customers by discovering customer needs and providing solutions to their problems. If your deployment frequency is low you can increase it by limiting the lifespan and size of pull requests. This will decrease the time spent in the review stage and reduce the total lead time. If the lifespan and pull request size is already small, you can introduce feature flags to increase deployment frequency. Feature flags are used to show and hide features in a solution; even before they are complete and ready for release. These encourage software engineers to ship work upon completion, increasing deployment frequency.
8. Has the lead time for changes increased?
The lead time for changes is defined as the time taken to get changes, from being comitted, to running in production. If your lead time to implement changes to your product is high, customers won’t receive value for a long period of time. This will reduce the customer’s satisfaction and encourage them to look for other solutions. If this is the case, you might want to implement a continuous delivery strategy to reduce the lead time of your software development team. This is a software development practice that allows code changes to be automatically prepared for a release to production by automating their testing before releasing it to customers.
9. Has the change failure rate increased over the past month?
The change failure rate is the percentage of deployments that failed, out of the total number of deployments over a certain period. These faulty deployments will require revision before they can be released and make the product work again. This will reduce the team’s velocity and cause an opportunity cost.
How Athenian approaches metrics
Athenian understands the value that metrics have for software engineering leaders. We believe that:
- They allow a deeper insight into current and historical performance.
- They give feedback on implemented practices and their effect on the team.
- They help you identify possible problems before they arise.
To access this valuable information you can use the Athenian Software Delivery dashboard. It will give you the ability to:
- Spot which segment of the pipeline increases your total cycle time the most.
- Identify bottlenecks and potential problems before they arise.
- See the impact of your decisions on the team’s performance over time.
- Check the deployment frequency of your team and what affects it.
- See how the workload and pull request size is affecting throughput.
- Increase the transparency and communication with your team.
