Measurement kit: First aid for CIOs

New chief information officers will want to know several aspects of the

information technology operation they inherit.

At a minimum, CIOs need to know the resources spent on each major system,

which organizational units are the primary customers and how satisfied they

are with the systems. CIOs should also want to establish a baseline against

which to compare their performance during their tenures.

The CIO Measurement Kit below provides basic, yet essential, information

to help CIOs better manage an agency's IT assets and services. The kit consists

of the following components: cost, user satisfaction, alignment, baselines

and targets.

Gathering and maintaining information on these components will provide

CIOs with insight to focus IT resources where they can have a measurable

impact on an agency's mission. The information collected becomes the backdrop

for effective capital planning. It also provides the measurement foundation

by which CIOs can determine to what degree IT contributes to their organization's

bottom line.

CIO Measurement Kit

Cost:

Know what is being spent on whom, where and why. What is the allocation

of IT spending? How much is spent (for systems and support) on each major

system? How well does the spending match the priorities of the agency and

its performance goals?

According to Gartner Group Inc., in general governments spend about

80 percent of their IT budgets on supporting, maintaining and enhancing

their legacy systems. That only leaves 20 percent for new starts. Where

should IT funds be directed to have the most impact? What needs to be done

to achieve 90 percent of the cost estimates?

User satisfaction:

User satisfaction is an indicator of the quality and effectiveness of

a particular system. If users are dissatisfied, then it's questionable whether

that system will make any contribution to mission results. The more satisfied

users are, the more likely the system will be used and the greater likelihood

users will be productive and mission objectives will be achieved.

Satisfaction should focus on the quality of the system, the information

and the service provided. Obtaining user feedback often uncovers ways to

improve the system and can foster better relationships with the user community.

How satisfied are the users of the agency's major systems? What would they

like to see improved?

Alignment:

The closer IT systems are aligned to mission goals and objectives, the

greater likelihood the IT systems will contribute to mission results. If

business or organizational objectives are poorly defined, then alignment

will be difficult to ascertain. What are the measures the agency uses to

judge performance? Are they being used?

Logic models help determine alignment. A logic model is a diagram that

shows the if/then flow of results. For example: "If the system provides

this information, then the user will perform a particular task better, faster

or cheaper (BFC). If that task is performed BFC, then customer service will

improve. If customer service is improved, then mission performance will

improve." Logic models are more accurate if developed with program personnel.

By measuring key results (e.g. BFC) along the "flow of results," CIOs will

be able to determine whether the IT system is on track and can make adjustments

as necessary so that an IT initiative does contribute to mission results.

CIOs need to have measures for both the IT domain and the business domain.

Baselines:

Baselines are used to compare and evaluate future performance. Choose

meaningful measures for both the IT and business domains. If baselines do

not exist, create them by choosing measures and collecting the data. Make

sure the data match the measures. CIOs can jump-start or improve their existing

measurement efforts by considering performance measures used by other organizations

and comparing performance levels with organizations with similar business

lines.

Targets:

Looking to the future, select targets that will contribute the most

to mission results. How are the targets for the IT initiatives aligned?

Do they match the performance goals and priorities of your organization

over the next three to five years? How will the achievement of those targets

contribute to the performance goals?

Implementation:

Measurement doesn't just happen. To provide useful and reliable information,

measurement must be systematic, iterative and important to management. The

information collected must be used for decision-making, e.g., during the

capital planning process; otherwise, people that collect the data will not

take it seriously.

Approach:

For each major IT system, identify or determine the following:

* System type: Legacy or new start?

* Business owner: Internal organization that is currently using or will

use the system?

* Process supported: Core or non-core?

* Cost: Amount allocated annually and projected in the future?

* User satisfaction: For each owner, establish a baseline for the level

of satisfaction with the quality of the system, information and service.

* Alignment: Develop a logic model with users to show IT's link to the

mission. Users judge the degree the system aids them to complete their objectives.

Determine which performance measures your organization uses and how each

major system contributes to them.

* Baselines: Choose measures, use existing data or collect new data.

* Targets: Using a logic model, develop targets of performance that

will contribute to your organization's performance goals and match its priorities.

Benefits:

CIOs will be in a better position to:

* Determine IT's contribution to mission results.

* Determine if the IT budget is being allocated to best achieve IT and

organizational objectives.

* Improve relationships with users and management by greater attention

to user satisfaction.

* Align IT systems to mission objectives.

* Identify opportunities to streamline processes.

* Improve relations with their business counterparts.

Plunkett is the General Services Administration's program manager

for performance measurement.