​No, not the warriors who fly aircraft into combat, DoD needs many, many more of them.

What I’m talking about is pilot programs where a set of programs experiment with a new approach.

The DoD bureaucracy is massively complex. To change any enterprise process often involves many stakeholders, large budgets, and long timelines.

What is often proposed is a pilot program, whereby a select few programs can adopt a novel approach, to see what works and what doesn’t, and assess if and how to scale to the rest of the enterprise. This is a logical approach.

The challenge with many pilot programs in DoD, often Congressionally directed, is the burdens outweigh the output.

Consider this scenario:

There’s broad consensus that the X process isn’t working.

Someone proposes a pilot program whereby a few programs can pilot the new X process. It may offer greater flexibility and/or exemptions from key bureaucratic elements to achieve a desired outcome.

Congress supports the idea and includes it in the NDAA.

DoD identifies a lead to implement the X Pilot Program.

The lead sends out a call to the Services and Agencies to identify potential X pilot programs.

Keep in mind that most program offices are overworked, under-resourced, and focused on trying to navigate their programs through a byzantine gauntlet of processes, reviews, documentation, and oversight with unstable funding and competing priorities. But in this one area, they have the opportunity to do things easier.

Most program managers will look down at their shoes, not wanting to be a pathfinder, but rather focus on executing their programs.

A few will step forward and volunteer. Some are bold innovators seeking to be trailblazers. Others are naïve of the road ahead, while some view this as a lifeline to save a troubled program or career.

Leadership will select the cohort of X pilots and they’ll work through what that really means and how to go out and execute it.

When the X pilot programs move out with novel approaches, the bureaucratic elements will emerge to resist or sabotage the change.

PMs will claim: “But I’m an X Pilot program, here to experiment with novel approaches. I have a get out jail card, exempting me from this cumbersome process.”

Yet the resistance will have their means to force compliance, as exemptions pose a threat to their world order, purpose in life, or revenue stream (for some contractors). Many of the various oversight and functional organizations may be unaware of the pilot program and objectives. These organizations will often then insert themselves into the processes and reviews, to ensure due diligence, which negates the exemption from the cumbersome process of questionable value.

The DoD lead for implementing the pilot program is required to report to DoD leadership and Congress on the progress and effects of the pilots. They will regularly inquire with the pilot programs on the status, issues, metrics/data, and insights on their experience.

If the lead maintains a light touch approach, they risk not gaining sufficient insight and data to report. If they take a heavy-handed approach, the pilots grow resentful for the constant inquiries and reporting.

The other challenge is isolating the variables to measure the impact. Consider, in science experiments we have three variables:

• Controls: Variables to hold constant
• Independent: Variables you change – what the pilot experiments with.
• Dependent: Variables affected by the change – which you measure

There are 100s of moving parts in an acquisition program to include strategies, staff, funding, costs, requirements, contracts, contractors, stakeholders, technologies, testing, oversight, and much more. To effectively measure the impact of the pilot program, one would need to baseline the program before the pilot effort – and by baseline I don’t mean an APB. Then hold everything constant, except for the independent variable being piloted, and run the pilot for a year or two.

Given we can’t do that, it is difficult to isolate the effect the piloted change had on the program. Pilot efforts are required to offer some data, but mostly provide anecdotal evidence. If the wrong pilot programs were selected (troubled programs or not a good fits for pilot), that will often spoil the results.

This is followed by some in Congress who were supportive of change and sponsored the pilot frustrated by the lack of data and insights from the pilot program. Others in Congress and DoD who were resistant to the change point to the lack of hard evidence as insufficient justification to change operations.

We’re left with a few years passed, most involved in the process are frustrated, and no meaningful reform is enacted to scale beyond a few pilots. This has been the downfall of many pilot programs across DoD and government.

Is there a better alternative?

1. Bring the key stakeholder groups together and debate the issues and proposed reforms.
2. Assemble small teams from diverse groups to lay out what the new approach could be or a few courses of action.
3. Leadership decide on the change.
4. For major enterprise-wide changes, instead of a pilot program, identify a subset of organizations, portfolios, or programs that would be a good place to start with the new approach.
5. Issue a directive (or statute) that going forward they are either exempt from key bureaucratic elements and/or authorized to use the new, evolving model. This isn’t for a short trial period, but their new approach going forward.
6. Include a communication campaign to inform key stakeholders and the broader workforce of the purpose, scope, and details of the reform effort.
7. Require some status checks, change management practices, and continuous improvement with key stakeholders involved.
8. Leaders must make bold decisions to remove the non-value-added processes, organizations, or people (including saboteurs) to enable meaningful progress.
9. Then based on the results and insights, iteratively scale for the rest of the enterprise.

Disclaimer:  The opinions expressed here are those of the authors only and do not represent the positions of the MITRE Corporation or its sponsors.