Competitive Prototyping

Prototyping plays an essential role in developing tangible, deployable, valuable functionality, but also has other uses. Real innovation always includes a risk of failure, and continuous prototyping gives program managers and users a powerful resource to better refine and understand requirements, reduce risk, quickly integrate new technologies, and identify innovative solutions that leverage existing programs as well as GOTS and COTS products. For example, the program might wish to create a prototype to assess whether a COTS product can perform a given function. In other cases, a prototype can evolve into the final product or be used in a future release.

Prototypes can enable the DoD to quickly eliminate ineffective approaches and focus on effective ones. They also provide a way to demonstrate functionality to help solidify requirements for the final design. An ongoing technology demonstration capability creates an environment for prototyping several different implementations of a design feature to benchmark the resulting performance and analyze the tradeoffs of each approach. This can reduce risk, drive better design decisions, and save time during the development process.

Agile development inherently involves continuously refining capabilities and requirements by adding to previous work or refactoring (cleaning up code) when necessary to meet changing priorities. Teams can reprioritize requirements based on the information learned during prototyping activities. Product Owners and users may also identify new requirements, refine existing requirements, or even eliminate obsolete requirements (e.g., “we don’t really need X…”) based on prototyping efforts

“I have not failed. I have successfully discovered 10,000 ways to NOT make a light bulb.” – Thomas Edison

Some prototyping practices to consider in an Agile environment are:

• Institute rapid prototyping that allows developers and government R&D organizations to quickly demonstrate potential solutions that meet urgent requirements, bring technologies to maturity and integrate them into the solution space, and highlight advantages over alternative options.
• Establish an environment that includes processes and properly configured DoD test platforms to rapidly prototype capabilities that extend beyond the next increment or project. This creates opportunities for innovative and dynamic solutions. A portfolio-level multiple award Indefinite Delivery Indefinite Quantity (IDIQ) contract is the best vehicle to support this type of continuous prototyping environment which can also provide opportunities for innovative small businesses to enter the DoD market.
• Involve multiple vendors during the TMRR phase to develop one or a few releases. This gives the government insights into different Agile approaches, vendor strengths, and experience going through release and sprint activities. Involving all stakeholder groups, including users, testers, and certifiers, enables early identification of risks, issues, and opportunities. This, the government becomes better informed and has more experience to shape program structure and strategies and select the right vendor(s) for development.

From Defense Acquisition Guidebook Sect 4.2.4

“Competitive prototyping (CP) has as a primary objective to acquire more innovative solutions at better value by ensuring competition. At this point in the life cycle, the CP strategy should focus on mitigating key technical risk areas. The program office should have a clear understanding of technical, engineering, and integration risks at Milestone A. Current policy does not require full-up system prototypes; therefore, competitive prototyping may include prototyping critical technologies, system elements, integration of system elements, or full-up prototypes. Because a primary objective of this type of prototyping is to support a follow-on award choice between developers, contract incentives should be aligned to CP strategy goals. Contract goals should require the solutions demonstrated during CP be used in the subsequent PDR/CDR designs. The CP strategy should be identified in the SEP and TDS, tasks specified in RFPs/Task Orders, technically managed by the program office, and included in the Test and Evaluation Strategy with specific test objectives.”

Implementation Guidance From Better Buying Power 3.0

Increase the use of prototyping and experimentation. The intent of this initiative is to reinvigorate the use of prototyping and experimentation for the purposes of rapid fielding of technologically advanced weapons systems, providing Warfighters with the opportunity to explore novel operational concepts, supporting key elements of the industrial base, and hedging against threat developments or surprises by advancing technology and reducing the lead time to develop and field new capabilities.

Prototypes are preliminary versions of a system or major sub-system assembled to resolve some area of risk and/or to explore operational potential. In this context, prototyping occurs prior to making a substantial commitment of resources for engineering and manufacturing and development or production and does not require programming or budgeting for follow-on activities. Importantly, during tight budgets these projects are low cost compared to full scale development and production.

Experimentation puts prototypes into the Warfighter’s hands for assessment in an operational context. Experimentation capabilities span use in the field by military personnel, wargaming, simulation, Service/Combatant Command exercises, and government/industry live, virtual, and constructive environments. Prototyping and experimentation activities contribute to the requirements definition process; aid reducing technical, schedule and cost risk; help refine the manufacturing processes; introduce new tactics, techniques, and procedures; help reveal unanticipated vulnerabilities; and aid retention of critical defense-related skills in the industrial base.

References

• Competitive Prototyping, MITRE Systems Engineering Guide
• The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle, National Academies of Science, 2016
• AT&L Memo on Prototyping and Competition, Sep 2007