Recent forecasts by Gartner Group suggest that 2002 will be a difficult year for IT suppliers. Key elements of their forecast are:

A more conservative buying attitude at customers leading to higher barriers to entry for smaller and less stable companies.

More emphasis upon the ROI and other benefits from information management projects and less on the technology.

The consequences of these forecasts are that IT suppliers will increasingly have to focus upon a task that they have found difficult to do well - measuring the impact of information management projects at customer sites.

Those that do not focus upon measuring the benefit of their product or service, will unfortunately end up going out of business.

The first clear rule of evaluating information management (IM) projects is that the up-front cost of the project is only the beginning. An IM project is a bit like having a baby. Giving birth is the beginning of the process not the end of it. And babies have to be fed, watered, cleaned and educated.

Information Technology professionals call the total cost of the project the TCO or total cost of ownership. The TCO includes all the costs of selecting, installing, acquiring, integrating, upgrading and withdrawing an information systems plus all the staffing, support and training costs required to operate the system.

Information technology can be taken in many directions to support many goals - cost reduction, increased customer service, repeat purchase, flexibility of architecture, integration, ease of maintenance, etc. As a result, TCO ends up being the wrong yardstick for most information technology.

What turns out to be more important is the value opportunity created by the project. Increasingly it is hard to separate information management projects for business initiatives, so the evaluation framework needs to consider what the overall value created will be, might be or has been from the combined business and IM initiative.

The world is an uncertain place. As a result, IM initiatives are typically best done in an iterative fashion. As my co-authors and I comment in our book, Riding the Tiger, it generally takes three generations to get a piece of software right. As a result, evaluation processes need to look at multiple time periods and uncertainty. Probability adjusted results are important for evaluating the Total Value created over time.

In another view of risk, a supplier of IT products and services, can reduce the perceived risk of small suppliers by to partnering with larger firms, that are perceived as stable suppliers.

Measuring the impact of business initiatives can be done in numerous ways. We are very taken with the Balanced Scorecard approach made popular by Kaplan and Norton. The Balanced Scorecard approach essentially makes four arguments:

Financial outcomes need to be linked to other performance outcomes that look the impact on customers, processes and internal staff and their skills.

Relationships between financial and proposed non-financial outcomes need to be tested.

Defining Balanced Scorecard measures is most effectively done when roughly 80% of the measures can be compared to other industry measures. This argument made by the consulting firm , the Gartner Group makes the development of performance measures potentially easier and more useful.

Perhaps the most common mistake made by IT vendors is to attempt to measure the benefit of their product or service upon their client after the project has been deployed.

While in some circumstances, well-managed clients may have collected some performance data, generally performance measurement is not available, not collected or viewed as proprietary and not for public disclosure.

The key task, therefore, in creating white papers that address the value created by projects is actually that of designing an ongoing measurement process that starts at the beginning of the project.

Designing the measurement strategy for an information management project always has four types of measures:

Standard measures that the supplier would like to be able to compare across clients. These numbers fall into the 80% of measures that Gartner Group suggests should be non-custom.

Custom measures that are tied to particular areas of improvement that the client has particular interest in because of their strategy, performance level or competitive position.

Forward projections of performance measures over the life of the business initiative. These numbers have high uncertainty associated with them.

Dynamic performance numbers that reflect the impact of learning upon the organization due to (a) improvements in the IM project, (b) improvements in business management, (c) experimentation, or (d) changes in customer position.

The more you know about your IT product or service, the more difficult it typically is to put yourself in your customer's shoes. Developing a benefit oriented white paper to speed up the sales process requires:

Ongoing development of standardized data across clients to reduce the cost and increase the value of the measurement process.

A commitment in the sales process towards learning about the customer business - this should result in expressing business propositions in terms of benefits and payback, not features and competitive advantages of a technology product or service.

For more information on Eclicktick services in developing measurement strategies, surveying customers or writing white papers, please contact Alistair Davidson.