Systems Engineering

Requirements

During the early stages, the program works with its operational sponsors to refine and structure the program’s requirements. To set the stage, the team should have a clear understanding of the users’ Concept of Operations (CONOPS), performance objectives, and operational environment. The requirements are prioritized to support an Analysis of Alternatives (AoA), tradeoffs, and the advantages and disadvantages of structuring development via releases. As requirements mature, frequent collaboration with end users is critical to ensure the requirements capture users’ priority needs and that all stakeholders have a common understanding of current and future CONOPS. Collaboration with other stakeholders (e.g., engineers, testers, and enterprise architects) is also critical during the shaping of the requirements. The program identifies a Product Owner, structures the requirements documents, and establishes the requirements management processes.Once a contractor is selected, the development team refines the program and release backlogs with the Product Owner and PMO.

Identify a Product Owner

The product owner fulfills one of the most important roles in the Agile team. In general, the product owner/Product Owner’s responsibilities across the lifecycle include:

  • Establishes a clear vision for what the program is to accomplish
  • Conveys the concepts of operations to the Program Management Office (PMO) and development team
  • Manages the program and release backlogs to ensure the requirements/user stories are current, prioritized, technically feasible, and have a clear “definition of done” acceptance criteria
  • Works with the development team to convert requirements into user stories
  • Adds/deletes/refines user stories and acceptance criteria with inputs from multiple stakeholders
  • Coordinates with senior operational sponsors, expert stakeholders, and a broad pool of end users to capture their requirements, priorities, and feedback while sharing insights, and providing demos on the program’s progress
  • Answers questions from the development team on user stories, functionality, and user interface
  • Coordinate capability demonstrations with operational sponsors and end users after each sprint to provide feedback to the development team.
  • Provides reguarly feedback to development team on interim developments
  • Identifies active participant in release and sprint planning and retrospectives
  • Identifies active participant in modeling, mock-ups, and testing
  • Coordinates with other product owners of related programs the product will interface or integrate with
  • Coordinates deployment of releases.

The more closely the Product Owner can work with the development team and the PMO during development, the better the chances for success. If co-location with the development team or PMO is not achievable, there should be frequent synchronous collaboration sessions, such as face-to-face meetings, video/audio teleconferences, and online collaboration/demonstrations. A Product Owner should be available to provide timely information and decisions. Asynchronous collaboration involving tools such as JIRA can also enable the Product Owner to work closely with the development team.

The program must select someone who has the operational experience, communication skills, and vision necessary to shape the program. The program should designate and empower product owner(s) who can convey the program vision, serve as the voice of the customer, and manage the program backlog. This could be done by a single leader or a small team of leaders from the operational community who perform day-to-day requirements management.

Product Owner References:

Structure Requirements Documents

Approval of the Information System-Initial Capabilities Document (IS-ICD) at the MDD empowers the program to use the IT Box model of the JCIDS Manual, to tailor subsequent requirements documents per approval of their designated flag-level requirements oversight board. Programs may decompose requirements into Requirements Definition Packages (RDPs), Capability Drops (CDs), and/or alternative document structure. The figure below shows a potential document structure as outlined in the JCIDS Manual. In this example, RDPs and CDs are incrementally developed and approved over time. RDPs are the primary documents that capture a subset of the ICD requirements. CDs can either cover a small subset of an RDP or a standalone item such as a small application.
IT BoxThe IT Box model offers a significant flexibility advantage over the traditional JCIDS documents, but an Agile environment will need even more flexibility in order to dynamically manage requirements. Agile prioritizes working software over documentation; it is critical to not invest too much time and energy creating and approving lengthy requirements documents ahead of time. Agile relies on continual user engagement and requirements refinement throughout the program. The flag-level requirements oversight board identified in the IS-ICD and the Product Owner should share a clear understanding of expectations and authorities.

Establish Requirements Management Process

A clear, rigorous requirements management process is critical to align stakeholders toward common outcomes. One approach is to ask the requirements board to approve a succinct RDP as the initial set of high-level requirements, capabilities, and/or user stories for a release or set of releases to support planning. Managing requirements in a traditional acquisition program includes a requirements owner to develop a large requirements document and collaborate with the user community.  In an Agile environment, a similar role helps to capture requirements (via backlogs) and collaborate with end users, but also join the program team as a permanent member to regularly iterate on the requirements.  During the planning and execution of the release, the Product Owner should be entrusted to add, delete, change, and refine requirements or user stories per an established Agile requirements process. Because Agile releases and sprints are time boxed, the schedule is locked while the scope is flexible. Similarly, based on development, demonstrations, and feedback on early sprints, the requirements or user stories for the release will likely change. The Product Owner should keep the requirements board informed of major changes during a release for transparency and common expectations. The development team defines and agrees on the scope of each sprint and finalizes it via a sprint planning session that takes a day or less. Given the rapid timelines, it would be too cumbersome to coordinate approval of CDs as a subset of an RDP via a flag officer-level requirements board. The requirements board should empower the Product Owner should to approve the scope of each sprint and inform key stakeholders. This is best accomplished via a set of user stories and acceptance criteria, whether captured in a CD or ideally via collaborative Agile development tools. A small stand-alone capability independent of a release may be captured in a few user stories or CD and, depending on the requirements board’s expectations, approved by the board or the Product Owner.

Requirements Best Practices from Early Adopters
  • Operational environments support small, frequent capability deliveries.
  • Requirements are clearly decomposed into small tasks.
  • End-users can engage throughout requirements and development to share CONOPS insights and provide immediate feedback from demonstrations.
  • Managed a program backlog via half page work packages with a rough government estimate, design context, and technical interfaces
  • Empowered Product Owners – Single/multiple based on user community’s size/diversity
Initial Development of a Product Backlog

Requirements in an Agile environment are usually managed via program, release, and sprint backlogs rather than through formal requirements documents. Backlogs could take the form of databases, Excel spreadsheets, or Agile-based software tools. The Product Owner actively manages (grooms) program and release backlogs, working with the user community and other stakeholders to identify the greatest level of detail needed for the highest priority requirements.

The figure below shows the relationships among the program, release, and sprint backlogs. The program backlog contains all desired functionality and requirements. A release backlog typically comprises the highest priority requirements from a program backlog that a team can complete within the established timeframe. Each sprint consists of the highest priority requirements from the release backlog. Once the development team commits to the scope of work for a sprint, that scope is locked. Sprint demonstrations conducted by the contractor at the end of a sprint may identify new features or defects that the team would add to the release or program backlogs.

The Product Owner, in collaboration with operational sponsors, requirements organizations, legacy system operators, a broad user base, architects, systems engineers, enterprise architects, and other stakeholders, captures, integrates, refines, and prioritizes items on a program backlog. Items can include:

  • Epics or themes of major requirements elements that span multiple releases
  • Interfaces with other systems
  • Infrastructure requirements/interfaces

requirements backlog

Interaction between Program, Release, and Sprint Backlogs

Many commercial Agile development tools are available to capture program requirements via user stories with acceptance criteria. These user stories are maintained in a database to further refine, prioritize, and estimate the complexity, time, and cost for each element. During the early phases of the program, this is often done by government personnel such as the Product Owner with support from the PMO prior to contractor selection during the EMD phase.

Government personnel, to include FFRDC and support contractors with Agile development expertise, draft the user stories based on end-user requirements; these user stories and estimates are refined once a development contractor is selected. User stories must be clearly presented in operational terms using an established format and include acceptance criteria. The government captures draft estimates of the complexity and time required to complete the necessary activities for the top priority stories to support structuring releases and estimating costs and schedules. The details of the lower priority stories will be worked out in the future.

Manage Program Backlogs

Once a contractor is selected, the development team reviews and refines the program and release backlogs with the Product Owner and PMO. This team is best suited to estimate the complexity and schedules to complete each user story. Over time, as the team develops releases, the fidelity of its estimates will improve as the members gain a better understanding of the team’s velocity, refinement of the backlogs, and estimation techniques.

The Product Owner regularly grooms the program backlog with the help of many stakeholders. Periodic meetings independent of the ongoing release may be required to refine the priority user stories with the users, engineers, developers, architects, testers, and other stakeholders. The use of collaborative online tools will ensure stakeholders have insight into the backlog details and provide inputs and feedback. As with many areas of Agile, a small, empowered, high-performing team will have better success than efforts that involve many people in daily operations. While five to nine is a common team size cited in Agile forums, the best size depends on the program environment.

The Product Owner, actively collaborating with users and stakeholders, is responsible for grooming the backlog to ensure the content and priorities remain current as teams receive feedback and learn more from developments and external factors. Users and development teams may add requirements to the program or release backlog or shift requirements between them. The release and development teams advise the Product Owner on the development impacts of these decisions, while users advise the release team about the operational priorities and impacts. Dependencies must be understood because the ability to develop capability to address a specific user story may depend on externally delivered capabilities (e.g., from other systems), internally developed capabilities already developed or planned. Some programs may use a Change Control Board to aid in some of the larger backlog grooming decisions.

In an Agile environment, users often translate requirements into epics and user stories to concisely define the desired system functions and provide the foundation for Agile estimation and planning. These stories and epics describe what the users want to accomplish with the resulting system. User stories help ensure that users, acquirers, developers, testers, and other stakeholders have a clear and agreed-upon understanding of the desired functions. They offer a far more dynamic approach to managing requirements than large requirements documents. Development teams periodically review the stories on the backlog to ensure the fidelity of details and estimates. Engineers may also write user stories to cover underlying characteristics of security, technical performance, supportability, training, or quality. Interfaces with other systems are usually captured as user stories.

User stories require little maintenance; they can be written on something as simple as an index card. A common format for a user story is:

“As a [user role], I want to [goal], so I can [reason].

For example, “As a registered user, I want to log in so I can access subscriber-only content.” User stories should have the following characteristics:

  • Concise, written descriptions of a capability valuable to a user
  • High-level description of features
  • Written in user language, not technical jargon
  • Contain information to help teams estimate level of effort
  • Worded to enable a testable result
  • Traceable to overarching mission threads

Each user story should be associated with defined acceptance criteria to confirm when the story is completed and working as intended. This requires the stakeholders to have a clear “definition of done” to ensure common expectations. Acceptance criteria consist of a set of pass-fail statements that specify the essential functional requirements in verifiable (measurable) terms. Defining acceptance testing during the planning stages enables developers to test interim capabilities frequently and rework them until they achieve the desired result. This approach also streamlines independent testing following development of a release. Examples include:

“A registered user can log-in 95% of the time.”

“While logged in subscriber-only content is available 100% of the time to registered users.”

The development team, users, and other stakeholders may use storyboards and mockups to help visualize the system use and features.

storyboard

Teams update the backlogs based on what the users and developers learn from demonstrated and fielded capabilities. The new items may include making fixes to software or generating new ideas. As operations change, teams may add or change requirements and user stories, both in content and priority. For example, the team may add integration items to the backlog as the program interfaces with other systems. Systems engineers and enterprise architects may add items that support the release integrity or technical underpinnings of capability delivery and information assurance. Ideally, teams should address issues discovered by government and contractor testers within the next sprint or release, but they may add those issues to a backlog based on the scope of the fix.

Additional references for user stories:

Active User Involvement in Development

A close partnership between users and materiel developers is critical to the success of defense acquisition programs and is a key tenet of Agile. Users must remain actively involved throughout the development process to ensure a mutual understanding across the acquisition and user communities. While most users maintain operational responsibilities associated with their day job, the more actively they can engage in the development, the better chances for success. Operational commanders must make a commitment o to allocate time for users to engage in development activities.

Users share the vision and details of the concepts of operations (CONOPS) and the desired effects of the intended capabilities. Through ongoing discussions, the program office and developers gain a better understanding of the operational environment, identify alternatives, and explore solutions. Users can then describe and validate the requirements, user stories, and acceptance criteria. The program office must make certain that the requirements can be put on contract and are affordable given funding, schedule, and technological constraints. Testers should also take an active part in these discussions to ensure common expectations and tests of performance.

In situations where primary users are not available to engage with the Agile team on a regular, ongoing basis, the end users can designate representatives to speak on behalf of the primary users.

User Forums

User forums enhance collaboration and ensure that all stakeholders understand and agree on the priorities and objectives of the program. The forums can serve as a valuable mechanism for gathering the full community of stakeholders and fostering collaboration. They give users an opportunity to familiarize developers with their operational requirements and CONOPS and to communicate their expectations for how the system would support these needs. Continuous engagement of users, developers, acquirers, testers, trainers, and the many other stakeholders at these forums also enables responsive updates and a consistent understanding of the program definition.

Suggestions for successful user forums include:

  • Hold regularly scheduled user forums and fund travel by stakeholders across the user community; alternatively, or in addition, provide for virtual participation.
  • Arrange for developers to demonstrate existing capabilities, prototypes, and emerging technologies. These demonstrations give users invaluable insight into the art of the possible and the capabilities currently available. User feedback, in turn, guides developers and acquirers in shaping the program and R&D investments.
  • Allow the full community to contribute to the program’s future by holding discussions on the strategic vision, program status, issues, and industry trends. Program managers should not rely on one-way presentations.
  • Give stakeholders the opportunity to convey expectations and obtain informed feedback.
  • Establish working groups that meet regularly to tackle user-generated actions.
  • Hold training sessions and provide educational opportunities to stakeholders.

Often large-scale enterprise Agile systems may require the involvement of additional independent teams of testers that execute system-/enterprise-level testing and integration. Please refer to contract preparation for details on contracting for outsourced testing.

Key Questions to Validate Requirements Management
  • What requirements documents does the program have?
  • What requirements oversight board exists and what does it review/approve?
  • Who is responsible for managing the program backlog?
  • What is the process for adding, editing, and prioritizing elements on the backlog?
  • How are requirements translated into user stories, tasks, acceptance criteria, and test cases?
  • Are there clear “definitions of done” that the team and stakeholders agree upon?
  • Are developers involved in scoping the next sprint (or equivalent)?
  • Does the backlog include technical requirements from systems engineers, developers, and testers along with the user requirements?
  • How frequently are the backlogs groomed?
  • Is there an established change control process for the requirements backlog?
  • Are architecture or system-of-systems requirements managed on the backlog?
  • Are requirements/user stories/backlogs maintained in a central repository accessible by a broad user base, and is content shared at user forums?
  • Does the product owner regularly communicate with the development team about the CONOPS?
  • Are testers actively involved in the requirements process to ensure requirements are testable?
  • Is there a mapping among requirements, user stories, tasks, and related items?
  • Is there a mapping between release/sprint requirements and higher level program/enterprise requirements?
  • Does the program have a dynamic process to regularly update, refine, and reprioritize requirements?
  • Does each requirement/user story planned for the release/sprint have associated cost and acceptance/performance criteria?
  • Are users actively engaged throughout the design, development, and testing process to provide feedback for each sprint?
  • Is integration with lifecycle support processes and tools iteratively refined and evaluated as part of each sprint?

Additional references:

3 Comments

  1. Mike Pelkey

    You propose one model for the ICD above, using the IT Box model. But at the IT Box model link (https://aida.mitre.org/it-box/), you note that this model is NOT appropriate for requirements for DBS (Defense Business System) capabilities. So, what requirements management model do you recommend for Agile acquisition of DBS (or do you not recommend Agile for DBS)?

    Reply
    • Pete Modigliani

      Mike,
      Per the JCIDS Manual, IT Box is not applicable to Defense Business Systems (DBS). DBS should embrace adoption of Agile principles to iteratively build out its capabilities working closely with business users. DoDI 5000.75 lays out the DBS requirements process. OSD/AT&L, DCMO, and DoD CIO are iterating on the supporting guidance for the BCAC processes within the DODI 5000.75.
      Pete

      Reply
  2. Arnie Rosenthal

    Lots of good material. But based on a scan of the titles (I didn’t have time to read all the documents; perhaps some go deeper, but this belongs at the top level), it gives little prominence to the implemented system, or to a Tools strategy. It seems to offer little to prevent systems engineering artifacts’ becoming shelfware after initial release.
    The typical practice of keeping documentation in MS Office does little to automate parts of implementation, Without a strategy to offer end to end tools (to support model based systems engineering), developers have little incentive to keep the systems engineering artifacts up to date and in synch with the implemented system. Mandates are evaded or become box-checking.
    Arnie Rosenthal arnie@mitre.org

    Since 70% of costs are for evolution afterwards, this seems a major omission.

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *