Growth Frameworks

Crafting Effective Metrics: A Guide for Technology Leaders and Beyond

Tim Kohn is a Technologist-in-Residence at Summit Partners, bringing more than 25 years of experience in building high-scale software and web services for consumers and the enterprise. During his career, Tim played a pivotal role in Amazon’s rapid expansion, leading teams of over 1,000 people during critical phases of the company’s growth, helping to build Prime Video, Amazon Web Services (AWS) and e-Commerce capabilities on Amazon.com.

Across functions and industries, metrics are vital to measuring and understanding the health of your business and the productivity of your teams. Not all metrics are created equal, however. Over the course of a career focused on building great teams and great products, I’ve spent a lot of time thinking about the kinds of metrics that have been most effective in achieving those goals. In my experience, the most effective metrics are those that help teams empathize with customers and inspire measurable results.

During my time at Amazon Prime Video, one of the key metrics our engineering and product teams aligned around was one we called “Time Spent Waiting,” which measured the length of time between when a user first opens the app and when their streaming content appears on-screen. Time Spent Waiting provided a tangible metric against which our teams could independently make – and measure – progress, and it aligned with our organization’s overarching goal of delivering the best streaming experience to our customers.

Despite its apparent simplicity, it was transformative for our organization. So, what made this an effective metric? Below, I break it down into a few key components to consider when developing your own metrics.

Make Your Metrics Intuitive

A great metric should be readily understood by employees, stakeholders and decision-makers – something that a junior engineer joining the team would understand on their first day and quickly grasp what it measures, why it matters, and how they can contribute to its improvement. As teams scale and organizations grow rapidly, this clarity becomes even more important. While “it can be hard to make it look easy,” investing the effort to distill complex observability data into a concise metric is crucial.

At Prime Video, we had massive streams of information from millions of devices. Without the right focus, the challenge of improving the streaming experience felt unbounded and overwhelming. By focusing on Time Spent Waiting, we were able to distill this data into something that was easily understood by both technical team members and executive leaders, and, importantly, could help guide action and measure progress across a 100+ person engineering team.

Focus on the Inputs

While output metrics – those that measure outcomes achieved, such as revenue, retention rate or gross margin – provide important insights into business health, they are often beyond our direct control. In contrast, input metrics allow us to measure the actions we take to achieve outcomes, and we can actively influence them. When a metric directly correlates with the efforts undertaken, it provides actionable insights. For instance, if an engineering team improves code quality (work done), they can expect to see a positive effect on system stability or customer satisfaction (the metric).

With the Time Spent Waiting metric, we were able to drill down on even more digestible inputs such as latency contributions (e.g., fetching images, rendering cover art, validating credentials and rendering controls) and work to pinpoint precisely where time was being lost. This approach helped guide the efforts of our engineering organization, allowing team members to focus on specific work to enhance the customer experience, ultimately driving positive changes in output metrics like app engagement, customer retention and overall revenue growth.

Measure Strategic Priorities

Connecting your team's metrics to broader organizational goals may seem obvious, yet it's surprising how often teams – engineering teams, in particular – operate only with available metrics simply because they can, rather than aligning with what truly matters to senior leadership and the overall strategy.

When metrics reflect strategic priorities, your team gains a clear understanding of and appreciation for their opportunity to impact the organization's success. Many of the most effective metrics I’ve worked with blend customer usage and tech measures, which was certainly the case with Time Spent Waiting.  This alignment can also foster meaningful cross-functional collaboration, allowing different departments to work toward shared goals while tailoring metrics to their specific functions.

Effective metrics transcend mere data points. They can shape organizational culture, help teams empathize with customers and inspire measurable results. For metrics to have the most impact, it’s important for leaders to foster cultural buy-in, ensuring teams recognize the value of measurement. Visual representations of progress – or lack of progress -- can help here, motivating teams and enhancing understanding, while testing and measuring guide iterative improvements. Regular live reviews with engaged stakeholders keep metrics relevant and actionable.

While my focus here has been through the lens of a technical leader, I believe these best practices can be applied to many functions across your organization:

Growth Timeline

No items found.

Don't delete this element! Use it to style the player! :)

Cae Keys
Truemuzic
Thumbnail
Play
0:00
0:00
https://interests.summitpartners.com/assets/DHCP_EP9_FutureHealthCare_DarrenBlack-2.mp3

Related Experience

The content herein reflects the views of Summit Partners and is intended for executives and operators considering partnering with Summit Partners.

Stories from the Climb

At Summit, it’s the stories that inspire us – the problems being solved and the different paths each team takes to grow a business. Stories from the Climb is a series dedicated to celebrating and sharing the challenges of building a growth company. For more Stories and other Summit perspectives, please visit our Growth Company Resource Center.

Get the Latest from Summit Partners

Subscribe to our newsletter to stay up to date on our partners, portfolio, and more.