On Being Agile

It's certainly no secret that the federal government is among the most rigid of all institutions. Bureaucratic gridlock extends far beyond Capitol Hill and finds its place in most aspects of federal business processes. As a result, agencies, contractors and consultants have built an industry that caters to strict project and program management guidelines to deliver services and information technologies that make government work.

This approach is not always the best methodology to develop technology, however. There is no shortage of examples of federal contracts gone awry due to mismanagement and miscommunication. From the much publicized failure of the FBI's Sentinel program, a network designed to allow agents worldwide to share evidence and tips about investigations -- which replaced the previously abandoned Virtual Case File system -- to the Custom and Border Patrol's SBInet initiative, designed to use ground sensors, infrared cameras and radar to spot illegal immigrants crossing the U.S.-Mexico border, millions of taxpayers' dollars have been wasted because the government and its contractors haven't adapted the way they develop projects to the realities of changing requirements needs.

Properly employing agile management in the federal government increases the probability of success for IT projects. In place of a predictive model, where methods are developed to execute against the expected requirements to minimize change, agile methodologies allow government officials and developers to use a process focused on value and adapting to project realities. The process promotes timely delivery of high-quality software that aligns with agencies' needs and goals. An increasing number of federal IT managers, contractors and consultants are turning to agile to deliver value.

What Is Agile?

Agile refers to a set of software and technology development methodologies that follow an iterative development cycle -- a system where goals, requirements and solutions are adapted throughout the development process. Agile aims to rapidly deliver software and technology solutions tailored to customers' specific needs, but be flexible enough to meet changing and varied market demands.

In addition, agile encourages agencies to follow a disciplined approach to project management that fosters the core concepts of inspection and adaptation throughout the process. Other key principles include teamwork, self-organization and accountability. In contrast to strict waterfall project management methodologies, the agile process values the clients and their needs over processes and specific tools, the development of working software to comprehensive documentation, collaboration with customers over contract negotiations, and responding to potential changes in goals or tools.

Agile development evolved in the 1990s as a reaction to the highly regulated and micromanaged methods managers followed in traditional projects. The new practices gave innovators a more flexible, streamlined, customer-centric way to develop IT solutions. Breaking down the life cycle of a project into focused iterations (or in some cases, units of work) allows developers to deliver working technologies on schedule, within budget and at a sustainable pace. This minimizes risk and lets developers adapt to changes.

With agile, developers and customers can overcome issues common to traditional project management: lack of communication, missed deadlines and opportunities for midcourse adjustments. Documentation is produced as needed, eliminating the process focus and bureaucratic bottlenecks common in federal government.

In practice, the agile model follows an adaptive pattern. It sets short iterations with frequent checkpoints. Planning is most detailed for immediate work and more high level for tasks that come later. The agile model builds in frequent inspection and opportunities to correct the course of progression, offering the ability to adapt to change instead of trying to minimize it.

Step by step, within an IT project, the "product" owner creates a backlog (a prioritized list of user stories or features) that the project should implement. The highest value items are at the top of the backlog for immediate attention. Along with this, a high-level roadmap is created that lays out product releases on a calendar and provides an expansive scope for each release. The roadmap is primarily used as a guide; it is not a contract. Thus the roadmap is adjusted throughout as indicated by the backlog and project realities.

Next, the team reviews the items at the top of the backlog and commits to what can be delivered within the iteration. At the end of the iteration, the team, product owner and applicable stakeholders review the work, record necessary updates and requests, and complete a near shippable product. From there, the team holds a retrospective meeting in which they identify process improvement opportunities and the product owner updates the backlog with any new items. This process is repeated until there is critical mass of features that will allow the product owner to go live with final product.

The process allows for requirements to change based on review of the product finished to date and keeps the customer involved early and often. With the agile model, the development process is constantly delivering, thus minimizing waste by fulfilling requirements. Instead of waiting for the results of a final product, the product owner can release a sufficient product at all stages of development. To put this into context, think about Microsoft Word and all its features. Microsoft could release Word with the 50 percent least used features removed and most users would not notice.

The advantages of agile have persuaded federal IT managers and consultants to adopt the methodologies. In 2004, the Food and Drug Administration claimed a 35 percent to 50 percent cost savings could be achieved using agile versus a waterfall approach.

The Defense Department, one of the government's most complex and bureaucratic agencies, started using agile because it speeded the delivery of projects in a sophisticated IT market. In October 2009, President Obama signed the 2010 National Defense Authorization Act issuing a mandate implementing the agile model for IT acquisitions after the Defense Science Board concluded the department's fundamental problem was the deliberate processes it followed to acquire weapon systems and IT did not match the speed at which new technological capabilities were being introduced. In the same vein, NASA adopted agile for all its agencywide application development projects.

Barriers

Many federal organizations hold tight to traditional practices, and barriers prevent them from adopting agile methodologies. An entrenched project management culture has yet to give up rigid parameters, documentation and processes. Government officials who are accustomed to extensive reporting and creating paper trails to reduce risk and manage accountability typically embrace such methods. But at its core, agile ensures that risks are being managed without losing focus on the delivery of value.

Some federal managers still have misconceptions about agile methods. They believe the process requires them to dedicate too much time to assist developers on the fly. The quick pace of development can be an asset, but also a point of contention. Plus, roadblocks emerge when agencies require developers to report on the value of a final project. Agencies mostly use earned value management methods to measure value, something traditionally associated with waterfall models. Despite common opinions, EVM is not incompatible with the agile model; its innate flexibility accounts for issues such as deadlines, costs and rigidity.

Funding and procurement also are sticking points. Government programs can have substantial lead times, which create challenges to agile programs. With funding mapped out well in advance, the develop-as-you go method might seem counterintuitive. Furthermore, government contracts generally are not written to support agile methods, and IT managers are not always familiar with the processes. Many IT procurement and contract officials are familiar only with traditional management methods. They often believe if a project fails, it's not because the process is flawed, it's that the process was not implemented correctly. A common fix is to add more processes to the program. This can introduce additional impediments to its success, as well. Ultimately, training IT officials and contracting officers about the agile process will provide them the knowledge to address funding and contracts to support such programs.

Change Is Happening

Federal agencies have started to change the fundamental ways they do business. Transparency is the latest buzzword inside the Beltway and represents a significant cultural shift away from the closed-door politics that have long been the norm. Federal IT managers, as well as acquisition and procurement officials, should again look at how they approach the development of IT solutions. Applying agile methodologies promises to streamline technology projects. Until agency officials fully recognize that agile can deliver more value in the development process, without the hassles and hurdles associated with waterfall methods, contractors and consultants will have to push for its acceptance.

Familiarity with agile methods and processes should bring changes to how the government develops technology projects. In today's tech-forward federal landscape, it's clear that agile methods are more adaptable and better-suited to facilitate the future of government business.

Richard Cheng is a managing consultant at Excella Consulting and leads its Agile Center of Excellence. He mentors clients on the adoption and implementation of agile and is working on a solution for the Office of Personnel Management.