Point: Fast-cycle experiments let companies create the best product/service offering with the least risk.

Story: At the World Innovation Forum, Scott Cook, co-founder of Intuit, described his company’s culture of high-velocity experimentation. Intuit uses an experiment-driven decision-making process throughout the organization. Rather than expect executives and managers to know all the answers, Intuit uses large numbers of low-cost experiments to test new product, service, and marketing ideas.

To illustrate how high-velocity experimentation works in organizations, Cook described how the concept works in Intuit’s 20-person online TurboTax unit.  In the past, this unit ran about 7 experiments during the annual three-month tax filing season.  Now they run 140 experiments.  Not only do they run more experiments, but they run each experiment on a fast cycle so that they can accumulate results and grow more knowledge during each season. Intuit created a weekly cycle for developing, testing, and analyzing experiments that lets each experiment create new information that feeds into the next set of experiments the next week.

Fast experimentation also improves employees’ sense of engagement and ownership.  In the past when the 20-person team did only seven experiments per year, the average team member might only have one of “their” experiments run once every three years.  But with new high-velocity approach, each employee creates and tests a new idea once every two weeks.

Paradoxially, running more experiments and getting more failures lowers the fear and cost of failure.  When a company runs only a few experiments — and every change or new product really is an experiment — then each experiment matters a lot more to the company and to the employees working on that experiment.  If an employee works for more than a year on a single big experiment, then the failure of that experiment surely has an impact on the employee’s career, even if the company professes to permit failure.  But if an employee works on many quick experiments that steadily improve organizational performance, then the success or failure of individual experiments matters little.

In Intuit’s case, 89% of experiments fail, and yet the online TurboTax unit increased conversions by 50% and successfully created a widely-lauded smartphone app that lets people do their entire tax preparation — from taking photos of tax documents, to automatically putting numbers in the right boxes, to electronically filing their forms — all on a smartphone.  The hundreds of tiny experiments let the TurboTax unit tweak and test many variations to come up with the best offering.

Action

• Trust experiments rather than experts to find the truth in a dynamic and volatile business environment.
• Teach individuals how to perform cheap experiments by finding and testing the key assumptions behind every new idea rather than building one big new product.
• Use large numbers of low-cost experiments to both grow knowledge and improve employee engagement.
• Create an accelerated cycle of experiments to reduce time-to-knowledge and time-to-market.
• Coach executives to become experiment-seekers — to use experimentation as a key decision-making tool.

Tags: Case study, experiments, failure, Innovation, Intuit, speed, World Innovation Forum