Recommendation 3

Embrace a Culture of Experimentation

Proposal: Encourage evidence-based, outcome-driven policies and experimentation by (a) testing multiple strategies simultaneously and using evidence to assess them; (b) testing different operational approaches in real time to accelerate learning; (c) creating opportunities to incorporate diverse viewpoints into decision-making processes; (d) offering bonuses, recognition, awards, and other incentives for managers who promote innovation and experimentation, give employees greater voice, and encourage creativity and divergent views.

Comment: The Board observed several teams in the DoD that practiced evidence-based, outcome-driven, and experimental methods, such as rapid prototyping. Instead of being guided only by anecdotes or intuition, or relying solely on process and procedure, the leaders of these teams focus on the effects of their practices and test various possibilities of employing different practices to seek out empirical evidence indicating which products or approaches are optimal. Their approach is rapid, iterative, and risk-tolerant. Instead of giving processes pride of place, they focus on outcomes, and how to get there most efficiently. These practices should be generalized, and not only to products and services, but potentially to strategies and operations as well.

The Board observed that the predominant culture in the Department values authority, consensus, tradition, and an extreme form of professionalism sometimes described as a “zero defect mentality.” These are admirable qualities, but they also tend to make employees risk-averse when it comes to creativity, experimentation, and dissent; and it makes nearly impossible for an ethos of experimentation to flourish. Experimentation – including embracing calculated risk-taking and learning from failure – is vital to improving decision-making and promoting innovation in the workforce. DoD leaders must find ways to embrace both sets of virtues; one way of doing that is to celebrate, reward, and provide incentives to those who take risks, learn from failures, and offer dissent. It is obvious there are situations unique to the military when the “no fail” attitude and authoritarian culture are the correct response to life-threatening situations or mission critical orders, but there are numerous circumstances where life and limb are not at risk, and an inflexible mindset is actually the greater risk to the overall mission.

Background: Experimentation and failure are built into the culture of Silicon Valley, and in the latter’s case, it is not only accepted, but often encouraged. Perhaps the most famous example in Silicon Valley history around companies empowering employees to experiment is Google’s 20% rule, which, while utilized less as an official policy today, allowed employees to spend 20% of their time on outside projects they believe would benefit the company. This idea – meant to harness employees’ creativity and entrepreneurship – led to the creation of Google News, Gmail, and AdSense.

Google’s R&D lab, known as X, specifically rewards employees who experiment and fail because not doing so diminishes the possibility that employees will take risks and discover key breakthroughs.

Facebook has posted on YouTube segments of its all-day or all-night internal sessions, which have taken place every several months for the past several years and during which employees work on projects outside their usual scope of work. These projects are not just technological in nature, but can also include efforts such as improving or making more efficient the use and delivery of office amenities.

Beyond Silicon Valley, experimentation is becoming a core competency for leaders and managers across a variety of industries and functions. Experiments are necessary for innovation—there is evidence that half of all patents are the result of serendipitous, unplanned events.  It is through tinkering, iterating, and making mistakes that creative ideas often emerge. Management researchers have gone so far as to argue that “failure is an essential prerequisite for effective organizational learning.” Indeed, when governments and private companies attempt to launch rockets into orbit, the more they have failed in the past, the greater their odds of succeeding in the future.

Experiments are unlikely to happen without psychological safety—the belief that it is safe to take risks. Psychological safety has been identified as the single most important driver of team effectiveness at Google and as a key factor in organizational effectiveness in other settings. Leaders establish psychological safety when they invite critical feedback, show vulnerability, and admit fallibility. Research has shown that pilots foster psychological safety when they announce to cockpit crews that they are open to being challenged, stressing that everyone’s first responsibility is not to respect authority but to land the plane safely. Physicians create psychological safety when they seek and value the input of every member of a healthcare delivery team. Manufacturing team leaders cultivate psychological safety when they frame mistake as learning opportunities—which in turn encourages people to run experiments.

Companies such as Autodesk, Microsoft, and Capital One encourage their employees to experiment in different ways:

• Autodesk prioritizes training employees not how to think differently – because the company recognizes that they already generate good ideas on their own – but rather how to operationalize them. Employees are encouraged to pitch business ideas and are trained to show why Autodesk is the right company to implement them for the benefit of the industry and consumers.

• Microsoft revamped its performance and evaluation metrics to allow employees to lead new innovation challenges for which they would not have been rewarded under the previous evaluation system.

• Capital One provides flexibility to teams to find and cultivate innovation champions among middle management – which usually focuses on core business processes and often rejects innovation activities they view as irrelevant to said processes – that allow more room for employees to test out new ideas.

High-tech companies have relied on split testing for years to understand their consumers’ behavior, evaluate their products, and determine how to optimize the products and services they deliver. Digital companies routinely use experimentation to optimize performance. For example: Amazon and eBay customize search results for individual customers based on their searches to understand the behavioral patterns of purchasers; Google runs analytics on how different users behave online based on the number and type of search results listed on one page; and Netflix evaluates what viewers click on to create more personalized homepages for them. Even small discoveries can have a massive impact on these companies' revenue, which explains why they devote significant resources to conduct this kind of split testing and analysis. It is central to how these companies operate, and it should be just as central for DoD to improve its extensive operations. Applying these principles to logistics, contracting, computing, maintenance, training, recruiting, and even strategy and operations could yield improvements in performance.