“The twenty-first century provides an exciting opportunity for systems engineering. Increases in technological complexity result in new challenges in architecture, networks, hardware and software engineering, and human systems integration.” – MITRE Systems Engineering Guide

The International Conference on Systems Engineering (INCOSE) chartered an “Agile Working Group” to refine the ISO/IEC/IEEE 15288-2015 systems engineering processes to reflect Agile principles. The developments, guidance, and best practices from that effort will be made available and couched in the DoD context here.

In an Agile environment, systems engineering requires tailored methods and processes to deliver incremental capabilities, and therefore demands a disciplined approach to coordinating parallel requirements elaboration and prioritization, technical development, operations, and sustainment activities. Systems engineers play an essential role in operational, technical, and programmatic integration, as expressed in the core Agile software development tenet of active collaboration among developers, users, and other stakeholders. Program leaders must encourage systems engineers to engage developers, testers, users, and other stakeholders in their disciplined engineering processes.

To enable faster, smaller capability deliveries, Agile development requires proactive integration among enterprise architectures, platform architectures, and related development efforts, where each of the stakeholder groups are contributing concerns and opportunities from their constituency for the good of the successful release and system delivery. To find the right balance between structure and the flexibility necessary to deliver usable capability aligned with user needs, programs should conduct continuous interdisciplinary systems engineering reviews in accordance with DoDI 5000.02 requirements. However, as illustrated in the figure below, programs should replace comprehensive Preliminary Design Reviews (PDRs) and Critical Design Reviews (CDRs) with more frequent and incremental design reviews during the release planning phases. To demonstrate functionality and provide insight into the program’s progress, these reviews should focus on the relatively small scope of a release and how it aligns to the enterprise architecture. Similar technical reviews can be decomposed to the release level.

Transitioning Large Formal Technical Reviews to Smaller, More Frequent, Iterative Reviews

While Agile systems engineering involves frequent informal technical and programmatic reviews, this less formal approach does not equate to less rigor. Instead, greater frequency allows key decision makers and other stakeholders to become more familiar and comfortable with processes, system and operational considerations in the Agile environment, which enables a more collaborative and productive review process. As emphasized in the sections above, full participation by key decision makers and users is fundamental to the Agile approach.

Some key systems engineering practices to consider in an Agile environment include:

• Provide information to all key stakeholders on a consistent, regularly scheduled basis, either through design reviews or program reviews and the use of an Integrated Data Environment shared by the stakeholders such as the development contractor(s) and the government. This improves government transparency, increases team understanding, and reduces the need for formal contract data deliverables.
• Use the release planning and sprint demonstrations as opportunities to bring users, developers, and stakeholders together in face-to-face sessions to drive collaboration and strengthen teaming arrangements.
• Ensure that once a clear architecture is in place, systems engineers continue to refine it as they learn more from the development sprints and releases. The team is responsible for developing, updating, and collaborating the architecture and shared system assumptions continuously throughout the Agile process.
• Independent of releases, hold periodic technical reviews for reviewing larger epics, creating strategic plans, grooming the program backlog, managing resources, and shaping the program’s direction.
• Capture systems engineering analytic and technical management processes and related content in portfolio-level systems engineering plans for broad use. Individual releases can capture the scope details in small appendices approved at a low level.

“Defines the scope of the technical effort required to develop, field, and sustain the system as well as providing critical quantitative inputs to program planning and life-cycle cost estimates. It provides the Program Manager and Systems Engineer with a framework to accomplish the technical activities that collectively increase product maturity and knowledge and reduce technical risks.” It serves as the basis of the Systems Engineering Plan and supports development of other planning and cost documents.
– Defense Acquisition Guidebook

In an Agile environment, technical planning is a continual, incremental process throughout the acquisition lifecycle. Technical planning in the early phases of a program provides a framework with greater level of detail on the priority requirements to be accomplished in the first few releases. Agile programs do NOT spend considerable time developing large technical plans, but rather provide a foundation to begin developing releases, often including multiple vendors as risk reduction efforts during the Technical Maturation and Risk Reduction (TMRR) phase. Each release and sprint begins with plans that lay out how many user stories will be accomplished based on available time and resources. For example, if the team is working on configurations for a COTS collaboration system, the user stories can identify desired functionality such as:

As a end-user, I want the ability to share data instantaneously with everyone in the online meeting.

Daily stand-up meetings review what each team member accomplished on the previous day, plans to do in the current day, and any roadblocks encountered. Sprint and release reviews allow the team to reflect on what worked and did not work to enable continuous improvement going forward. Continual grooming of the team and sprint backlogs helps prioritize and refine the development. Early and active risk management plays a key role in the technical planning process.

Technical planning occurs at several levels and timeframes within the development process. For example, the development team normally conducts technical planning on a frequent basis, usually during the daily standup meetings. The development team usually works with the Product Owner on a monthly basis, or at the start of each sprint, to conduct technical planning at the sprint level. The Product Owner usually works with the release team, to conduct technical planning at the release level, which should be aligned with the release timeframe every 6–12 months. In general, breaking-up technical planning into smaller, more frequent engagements at all levels will better support the Agile development process to ensure it aligns with evolving priorities, user feedback, and program-level changes.

“The Decision Analysis process transforms a broadly stated decision opportunity into a traceable, defendable, and actionable plan. It encompasses one or more discrete analyses at one or more lower (e.g., system element) levels and aggregates them into a higher-level view (e.g., system “scorecard” presentation) relevant to the decision maker and other stakeholders.”

– Defense Acquisition Guidebook

Decision analysis for Agile occurs at key points in development when issues arise and the development team must make decisions about development steps and design based on an understanding of risks and options. This can be challenging, since an optimum design and implementation solution is typically unknown, and in many cases “unknowable” due to inherent complexity, at the outset of a program. The team should resist the urge to over-engineer or design an optimal solution in advance; instead, the team should proceed based on a preferred direction, and then assesses progress and course correct based on analysis of specific experiences. Typical Agile decisions center on adjusting the grouping of user stories, determining which sprints to release to users, resizing and splitting design and implementation into separate stories, and the impact of removing requirements from the requirements backlog.

“The Technical Assessment process allows the Systems Engineer to compare achieved results against defined criteria to provide a fact-based understanding of the current level of product knowledge, technical maturity, program status, and technical risk. This assessment results in a better understanding of the health and maturity of the program, giving the Program Manager a sound technical basis upon which to make program decisions.” – Defense Acquisition Guidebook.

Technical assessment in an Agile environment resembles the process used in other programs: it evaluates progress based on a set of information and metrics. In programs using Agile methods, metrics reflect the key elements of the development approach and typically focus on user stories, including the number of user stories executed by the development team, and how quickly they were completed. Input may include ‘burn down charts’ showing progress in implementation of user stories and reduction in the requirements backlog or ‘velocity’—the rate at which user teams are implementing user stories. Systems engineers, instead of only attending major program reviews, will now actively participate in sprint reviews to assess progress, demonstrations, and upcoming plans. Use of tools that automatically generate burn-down charts and other metrics assessments can lead to significant time savings and provide even more information for technical assessment.

The Enterprise Architecture is the explicit description of the current and desired relationships among business and management process and IT. It describes the “target” situation which the agency wishes to create and maintain by managing its IT portfolio. – Franklin D. Raines, Former OMB Director

In today’s environment, IT programs can no longer operate independently; instead, they must interface and communicate with many other systems across the DoD. As the DoD continues to evolve toward a common operating environment with enterprise platforms, IT programs will deliver capabilities via a series of applications or web services. IT systems must therefore be designed, developed, and maintained in concert with enterprise architectures. The enterprise architecture is central to orchestrating technical integration and envisioning how a system should operate in the future.

An effective architecture is one where the environment is clearly understood and stakeholders understand how they connect to the enterprise. The architecture should highlight both the technical and operational processes and structures across the enterprise. It should take both existing and planned infrastructure and systems into account – from the capabilities they currently provide to the gaps that must be filled and the required interfaces to other systems.

Enterprise architectures capture vast amounts of information; however, the central guideline in developing and maintaining architectures in an Agile environment is to keep it simple. The value of the architecture decreases as its complexity increases, with volumes of documents and artifacts making it difficult for participants to comprehend.

Throughout the IT acquisition lifecycle, the enterprise architects serve as essential members of the program team, helping to effectively scope, structure, and analyze alternatives for delivering capabilities to the warfighter. They ensure the architecture captures the operational and technology environments as well as current processes, components, interfaces, and other features. As the entire program team learns how the required capabilities will achieve military objectives, the team members must also understand how the program will align with existing and planned efforts across the enterprise, and must provide the architects with updates to the program so they can continually manage and update the architecture. Enterprise architects must ensure that the program effectively digests and integrates additions to the architecture and archives expired elements. Frequent collaboration is critical to success.

When the program develops new software – for instance a new application, or customization of an existing COTS product – it must establish an initial architecture to guide the design and integration aspects of the project. The architecture will help to identify how to prioritize the requirements to deliver initial functionality.

In programs that employ Agile, the initial design of the architecture and platform infrastructure is usually created separately from the software development portion. Under a Scrum methodology, this upfront architecture and infrastructure design and build are referred to as “Sprint 0.”

Interfaces with external systems and changes driven by the enterprise architecture are often managed as stories on the program backlog. For example, if a new business process is introduced into the enterprise architecture, then the appropriate user stories would need to be added to the impacted backlogs as a response to these changes. Many interfaces must be broken up into many stories, but be managed as a theme or epic. Experts in Agile methods recommend the use of common (ideally open sourced) platforms, standards, interfaces, and application program interfaces (APIs) over costly point-to-point interfaces.

Additional References:

• Agile Enterprise Architecture
• Enterprise Architecture, Disciplined Agile Delivery

In most large acquisitions, many of the systems engineering processes are executed by the contractor(s) building the solution. These processes overlap with government responsibilities in some cases, so it is important to understand whether these activities are accomplished effectively. According to the INCOSE Systems Engineering Handbook, 4th Edition, some of these processes include):

• Technical Processes:
  • Design definition—the overall system design will, and should, evolve over time, but it is important that all stakeholders are involved in design decisions.
  • System Analysis—this is an ongoing process to ensure that incremental development of the solution remains stable, coherent, and aligned with the stakeholders needs.
  • Verification and Validation—the release process must continuously verify that what is built works – verification and meets the needs – validation (i.e., satisfies the user story).
  • Transition—moving incremental solutions into operations must be accomplished with care to avoid “update fatigue” on the part of the user while still allowing for the solution to evolve.
  • Operation—in an Agile environment, development continues while initial releases are being using in actual operations. It is important to monitor and measure the performance of the delivered increments and to have a mechanism to respond to “real user” feedback.
  • Maintenance—fielded systems may break. Maintenance activities should be fed into the product backlog and prioritized accordingly. Subsequent releases are technically “upgrades” to the existing capability and should be managed through the users’ maintenance processes, as well.
  • Disposal—as new functionality is added to a fielded, incrementally developed system, legacy systems may eventually need to be removed from service, and sometime early functionality of the new system may need to be retired when replaced by an improved functionality or no longer needed.
• Technical Management Processes
  • Quality Assurance—more than just testing, good software engineering practices are important to maintain the portability of code and better allow for future unanticipated changes to the fielded system.