You are on page 1of 3

PMI Virtual Library © 2011 Jane Ann Pighin

Benchmark for Success!
By Jane Ann Pighin, PMP


ue to continual changes in technology and conditions in the marketplace, IT services should be benchmarked every 18 to 24 months. This will encourage competitiveness and identify improvement opportunities for both insourced and outsourced services. Additionally, benchmarking is a tool used to show executives and customers that IT services are cost effective and well managed. The term “benchmark” can mean different things. There are three types of benchmarks, which are generally used by IT (Table 1). The competitive assessment benchmark is the most commonly used and will be the focus of this article. Competitive Assessment Overview Competitive assessments assess the prices paid for externally provided IT services and/or the cost of internally provided services against marketplace peers that offer similar services and service levels. Competitive assessments identify strengths and actionable improvement opportunities for cost, quality, and productivity.  An independent company, “benchmarker,” specializing in benchmarking, analyzes collected data and

compares them with a “reference group,” which is typically made up of five or six companies, similar in size, complexity, and quality for the benchmarked service. Selecting a benchmarker who will treat all parties fairly and objectively and assure meaningful results is critical; otherwise, the results will not be taken seriously and resulting actions will not be taken. Market and internal business conditions, supplier contract re-competition dates, and contract expiration dates are factors when determining the timing of a benchmark. A re-competition can ensure market competitiveness at a certain point in time; however, other mechanisms (e.g., a competitive assessment) should be used to ensure competitiveness over the life of the contract. Project Structure A benchmark should be managed as a project. A WBS (work breakdown structure) and schedule should be created and a project manager assigned to provide task, schedule, and risk oversight. The benchmarking process has five basic steps: 1. Preplanning: Identify a dedicated and autonomous core team that can provide oversight and ensure schedule

Benchmark Type Best Practice Sharing Macro Level Analysis Competitive Assessment

Description Peer-to-peer exchanges, leadership boards, and community of practice teams A portfolio level analysis (e.g., comparing company IT spend as a percentage of revenues with those of other industries and companies) A granular review of internal services, plus a market price assessment of those services provided by external suppliers to ensure competitiveness.  Benchmarks identify strengths and actionable improvement opportunities for cost, quality, and productivity. 

Table 1: The three types of benchmarks.

. and complexity. A sub team of “data collectors” who can devote adequate time during the Data Collection step (approximately 1 to 1 1/2 months) must be selected. thus. | © 2011 Jane Ann Pighin 2 . The project manager must obtain agreement from team members and management on the benchmark scope. Benchmarking can be complicated and time consuming. and complexity. A good benchmarker. The Data Collection step is the most time-intensive and critical step for the project manager. It is important that questions from the data collectors are documented and quickly answered. The benchmarker uses mathematical models for comparison of the benchmarked service to the reference group. the benchmarker will validate the data through a series of iterative reviews with the internal and/or external service provider. The benchmarker’s terminology must also be fully understood. Anomalies in the data are sought. and difficulty to include or exclude certain data. An externally sourced service should have a benchmark clause in the contract at the unit price level. It is important to break down the work into logical. After data is gathered and submitted. as well as management stakeholders.PMI. however. The benchmarker must also fully understand the customer’s environment and take all relevant factors into account. Some suppliers embrace benchmarking as an opportunity to demonstrate value. Frequent statusing is necessary to ensure that data collection focals can meet the schedule. phasing benchmarks by technology or service group will encourage success. Without accurate and representative data. is an expert at ensuring a fair and valid comparison by using a robust process with normalized data. The service provider should be involved in the benchmark from the beginning. finance. manageable parts. This sub team should include subject matter experts from the service provider. PMI Virtual Library | www. A benchmark clause is necessary to ensure that marketplace assessments are performed against externally sourced services. 3. the benchmarker will not be able to adequately perform a marketplace assessment. scope. Benchmark dates should be negotiated with service providers. thereby mitigating these concerns. Data must be comparable in size. It is important that team members. this will lead to the selection of a well-matched reference group. A matrix can be used to document what service areas are included and excluded in the benchmark study. Often. Executive support is critical to show workload prioritization for team members. The project manager must ensure quick resolution of any anomalies. Other suppliers may be skeptical that a “like” comparison is possible and try to discredit the process and make the benchmark clause non-binding. clarification of collected data is made. This is the first step in making sure that the data is organized and accurately recorded in a fashion that will be comparable with peers when a reference group is selected in the next step. Determine the benchmark scope (e. this will improve process efficiency and reduce the potential for error due to misunderstanding of data collection questions. Create data collection templates. isn’t a fair comparison. they may argue that it is too expensive and time consuming. the price must be adjusted retroactively.g. Data Collection: Collect labor and non-labor costs. if classified areas are to be excluded. During this phase. and service level agreement (SLA) data on predefined templates. Data Validation: Validate that the collected data is reasonable and without obvious errors. In this step. time. which may indicate an error in either the understanding of the question or a mistake in keying in the data. Upfront consideration should be given to ability. If the benchmark results show that defined marketplace targets were not met. and may harm your relationship with them. the benchmark scope is sometimes adjusted if certain question areas are too difficult to answer. Outsourced services are analyzed separately from retained services because of the different cost elements involved. All team members and stakeholders must believe in the data’s accuracy and integrity.adherence. 4. For example. What is important is that the IT service performed by the selected companies is similar to the IT service being benchmarked in size. what is included and what is excluded) and reporting views (comparison breakdowns). Identify key milestones and create a project plan. understand the benchmarking process and methodology and commit to the schedule. and the sourcing organization. the reference group companies are in a different industry. Analysis: A “like for like” reference group is selected and used to compare with the collected data. The project manager will also serve as a liaison between the benchmarker and service provider. scope. can you reasonably break out its data from the total organization? 2.

So. Additionally. PMI Virtual Library | www. the benchmarker may identify contractual shortcomings. operational best practices. Benchmarking is successful whether it demonstrates a need for cost reductions or validates competitiveness of current pricing. she also leads competitive assessment benchmarks. number of ‘views. few questions should arise at this stage. contract re-competitions. she is a Standford Certified Project Manager and has a Master’s Certificate in Project Management from Stevens Institute of Technology. Prior to this role. the service provider gains facts and data about the competitiveness of its IT service. which have produced significant savings over the years. an action plan needs to be put in place and followed over the upcoming months and years so that savings are realized and recommendations are implemented. which will help run its business. | © 2011 Jane Ann Pighin 3 . why spend the money and invest the time in benchmarking? If the service provider understands and trusts in the benchmarking process. PMP. Even if the service is competitive. The benchmarker may ask questions that were not discovered during the Data Validation step. In addition to cost comparisons. The service provider will believe that the benchmark results are accurate and fair only if the benchmark process was objective and conducted with transparency. and productivity are captured in a report. Specific recommendations for improvements are included. along with estimated cost savings. benchmark success will be achieved! About the Author Jane Ann Pighin. The project manager must ensure that all findings and recommendations are well understood by the stakeholders. technological inefficiencies. 5. Benchmark findings and recommendations must be clear and in an “actionable” format. As long as benchmark findings are thoughtfully considered and the agreed-on next steps are followed. supporting major enterprise infrastructure deployments. is an IT sourcing specialist at The Boeing Company in Seattle. she helps develop and implement sourcing strategies for IT infrastructure services.PMI. Findings: Benchmark findings showing measurements based on the key performance indicators (KPIs) of cost. otherwise. Savings reaped from implementing the benchmarker’s recommendations are often ten times or more than the cost of the benchmark.’ and complexity. Summary The cost of a competitive assessment for one IT service varies depending on the scope. Washington. and operational/process inefficiencies based on best practices and lessons from selected peer companies. and technology enhancements. A feasibility study can help determine the next steps. however. Recommendations can include: price reductions incorporated into a contract/agreement. consistency. although it can cost several hundreds of thousand of dollars. a benchmarker should provide recommendations for improvement. In this role. It is important that careful review and analysis be given to each of them in order to determine which should be implemented. The benchmarker should provide the project manager with a weekly status report to ensure schedule adherence. The project manager and core team have little to do during this step. the potential benefits should significantly outweigh the costs. The project manager also needs to work with the benchmarker to make sure the findings are properly communicated. quality.It is imperative that the benchmarker has access to robust data for comparable companies. In addition to her PMP. she was a senior IT project manager at Boeing. and data integrity. Although the results cannot be altered. the way the message is delivered can be. Once the benchmark has been completed. the relationship with a supplier and/or trust of internal employees may be impacted.