In implementing OTB and assessing OTB performance, the program office must distinguish the composition of its technical baseline from how it demonstrates deep ownership of the baseline.  The composition and scope of a technical baseline varies according to the type of system being acquired. System factors drive the kind of knowledge base that program offices must attain to establish tailored deep ownership. These factors include system type, technology maturity, implementation approach, scope of manufacturing, complexity of system security, level of mission assurance, integration complexity, system interdependencies, and scope of product support.

Identification of deep ownership areas and the outcome of the OTB BCA, in alignment with pending acquisition events and milestones, will help target what analyses to perform, and when.  The process of analytic performance (versus reading another organization’s analysis) is critical both to the formation and evolution of intellectual parity, and to the leveraging of an independent knowledge base in the context of volatile, real-time decision making. 

Model-Based System Engineering (MBSE) and end-to-end performance modeling are recommended practices to establish an independent knowledge base and form common ground for constructive interaction with contractors and external stakeholders.  To reap the benefits of these, the program office often must implement an overarching System Integration Lab (SIL) / Reference Implementation Lab (RIL) infrastructure strategy, aligned to generate information and findings relevant to critical decision making.  The program office infrastructure must also support well-functioning knowledge management.

The diagram below shows three technical baseline ownership models: controlled, collaborative, or influenced. These models are distinguished by the treatment of data rights, how systems integration labs and other facilities are used, how the program should allocate staffing, and the timing of OTB related activities over the course of the program. The model employed may change based on the business case analysis and the resources available to the program.  It is likely that in a complex program, the Government will be implementing two or all three models simultaneously, reflecting the tailoring of OTB.

The ultimate goal of OTB is acquisition success.   It is imperative that the program office not just establish a focused knowledge base, but that such knowledge is generated and applied to proactively support effective acquisition decision making.   We have cited personnel, infrastructure, and information elements of OTB.  A fourth element, processes, is important to analytic feedback and robustness, and it is essential to leveraging technical expertise in the programmatic space.  For example, cost/schedule processes must be anchored to technically sound inputs and trade-offs to form realistic, risk-informed budgets and schedules, which enable a program to be executable.   In the guidance, we cite key OTB-enabling processes.  To achieve effective management, there must be consistently engaged technical presence in all of them.

In this light, we have recently identified that forming an engagement framework is invaluable to establishing the right amount of engineering/SME staffing (not just the right skill and experience mix) and ensuring decisioning processes are supported by salient, comprehensive information and guidance.  The framework cites areas requiring significant technical presence and organizes a time-phased staffing profile for each and for the technical/SME team.  Typical engagement areas include the following:

• Design, populate, certify, administer, and use SIL/RIL infrastructure
• Self-generate key analyses and information
• Actively support internal engagements such as processes, working groups, and boards
• Foster peer-peer, informal contractor relationships
• Perform cross-interface technical management
• Oversee external engagements

With multiple elements, critical resource expenditure, and the quality of decision making at stake, an undertaking like OTB requires progress measurement.  From considerable reference material, we have distilled seven progress criteria broken out as follows: 

Foundational criteria

1. OTB Business Case: Effective identification and evolution of deep ownership areas
2. Proper Resourcing, Including People, Infrastructure, and Information: the means to accomplish OTB

Technical Focus Criteria

3. Performance Achievability Assessment: Independent from contractors, and pre-contract award
4. Ongoing Industry/Contractor Engagement Enabled by Organizational Transparency: informal peer relationships with contractors aligned with deep ownership areas. Not to tell the industry partner how to design, but to (a) bring external information (the contractor could not have) to influence trades and analyses and (b) bring back information to the program office team, to help guide independent decision making (e.g., how best to cope with a change in requirements). 
5. Clear, Selective, Government-Managed Interfaces: to support the ability to reuse of capabilities within a system, to promote competition for upgrading capabilities, and to help reduce “ripple effect” impacts of changes through the system. Also, especially in the early stages of the program, to allow the government to isolate risk within areas of the system.

Impact Criteria

6. Engineering Rigor and Engagement in Planning, Decision Making, and Implementation: not just support, but effective, proactive, consistent engagement.
7. Technically Sound Transitions between Phases and Organizations: Engineering foresight at times of greatest leverage (such as formation of the RFP before development contract award) to enable smooth transition at times of great impact (e.g., from development/test to product support).

The criteria are delineated by five measurable levels of increasing maturity.  This approach to progress measurement supports dashboard-based progress assessment reporting. 

Consistent with the tailoring of OTB, each program will have different targets for these categories, and they should vary across subsystems and enterprise-level ownership areas.  In addition, programs will typically be at lower levels of OTB readiness at the earlier acquisition phases.