Growth Frameworks

How to Embrace Experimentation and Make Sense of Your Data

Amidst unprecedented volatility in markets and industries, some businesses are experiencing steep declines and others are seeing step function growth. In our experience, the ability to quickly and effectively respond and react in new ways often lies in the power of experimentation. Here are three ways to help your team better anticipate challenges and recognize opportunity by understanding the value of data-informed trial and validation.

Nearly every business around the world has experienced rapid, dramatic changes in consumer and business behavior over the last few years. A decade or more of relative stability was abruptly interrupted by a pandemic, followed more recently by inflation and ongoing concerns about the possibility of a prolonged economic downturn. Historical data on sales, marketing, product usage, or other operational metrics becomes an imperfect guide to the future in times of significant turbulence, and this may be even more pronounced when the immediate past includes a large (hopefully) non-recurring event. Even in the best of times, attributing business results to the actions of your teams is challenging. So, how do you make sense of data as you work to navigate changing conditions?

1. Beware of Outliers

Recent years have been anything but predictable. Often, to evaluate the success of a change, companies rely on historical business data and compare results – sales, bookings, churn – before and after a product update, a new customer success initiative, a marketing campaign or another operational adjustment. However, history can be an unreliable comparison, particularly in the wake of a pandemic or in the face of volatile, inflationary, or recessionary conditions. Data in a volatile economic climate is especially challenging as it can only accurately represent a single moment in time. Even under normal circumstances, external “outlier” events such as unexpected press coverage, changing customer demographics, or seasonality can temporarily skew your analysis. Straightforward before-and-after comparisons will inevitably lead to incorrect conclusions if qualitative factors are not considered. Most businesses need a more sophisticated analytical toolkit to account for the unpredictable and unexpected.

2. Experimentation Leads to Enlightenment

Experimenting or trying, failing, and then trying again, is not a new concept. Yet, testing an unknown result can still feel risky to businesses. In the early 17th century Age of Enlightenment, French philosopher, and creator of the first encyclopedia, Denis Diderot outlined the three principles of knowledge: observation to collect the facts, reflection to combine them and most importantly, experimentation to test and verify them.  Any data-driven culture should be grounded in experimentation: testing changes and letting the data guide you. We see this approach regularly applied to marketing. You may have heard about – or even tried! – A/B testing on website button colors or email headlines. But the power of experimentation also matters in product development, customer support, operations, and in countless areas across your business. With experimentation, some customers receive a new experience while others receive the existing “control” experience. Results between the two groups can be compared because they occur under the exact same external conditions – allowing you to see the true effect. The ability to understand what is currently working and why will help your team find signal through the noise so they can invest their time and resources more efficiently.

3. Validate, Invest, Succeed

Change is inevitable in any economic environment. The value of experimentation is in understanding that change and steering it toward growth. Resources, whether time or budget, are scarce even in the best of times. As businesses continue to navigate changing market conditions, they need to be highly conscious of how resources are allocated. Here too, experimentation is key.

While teams often consider experimentation in the context of optimizing success – picking the option customers like best – A/B testing can also be used to avoid failure; quantifying the impact of product, process and other changes that can challenge results. When was the last time someone in your product organization told you, “We changed something, we think customers like it, but we don’t know how much sales have changed?” Or, “it was only a back-end change, customers won’t even see it!” Experimentation can eliminate risk and be an early identifier of the difference in conversion, sales or any other metric used to measure the new experience against the control group or status quo. Informed by the data, your team can invest talent and budget in ideas and experiences that work and drop the ones that don’t.

As we witness dramatic changes in how people work, learn, shop, and interact, adaptable, innovative companies will not only survive, but thrive.

Understanding the value of experimentation and how crucial it is to be building a durable business will help companies quickly adjust, offer new products, services, and delivery methods to meet customers’ changing needs during this uncertain time. Importantly, experimentation done right not only fuels innovation and growth, it also serves to reduce risk by measuring outcomes to inform decisions. A culture of experimentation can solicit new ideas and solutions and unleash the creativity of employees in every part of your business. Even the ideas that don’t work out can inspire your team to think differently about customers and what they need.

Growth Timeline

No items found.

Don't delete this element! Use it to style the player! :)

Cae Keys

Related Experience

The content herein reflects the views of Summit Partners and is intended for executives and operators considering partnering with Summit Partners. For a complete list of Summit investments, please click here.

Stories from the Climb

At Summit, it’s the stories that inspire us – the problems being solved and the different paths each team takes to grow a business. Stories from the Climb is a series dedicated to celebrating and sharing the challenges of building a growth company. For more Stories and other Summit perspectives, please visit our Growth Company Resource Center.

Get the Latest from Summit Partners

Subscribe to our newsletter to stay up to date on our partners, portfolio, and more.