Continuous Improvement & Benchmarking

Measure the service - Guide
Performance measures are used: • • • • at the start of an improvement project, to measure the performance gap. after implementing a new solution, to confirm the process has improved. as part of service monitoring, to help maintain the improved performance. to report to our stakeholders our contribution to social and environmental factors.

Performance measurement can have a strong motivational effect on staff. In a British study, performance measures reported to line staff within the City had a major impact on staff behavior. They became more “involved” in the service, they changed the way they worked, and they became more interested in their customers. Performance measures clearly communicate to our staff what we expect from them, and also how they are performing. Measures can become a great motivator of performance improvement, and their open publication recognizes and rewards achievement. Performance measures therefore play four important roles in Continuous Improvement and Benchmarking: 1. They objectively point us to service aspects which are failing and need improvement; 2. They provide a means of comparing our service with other cities; 3. They provide an indication of success or failure of the changes implemented; and 4. They enable objective evaluation on which to base recognition and reward of staff achievements The first step is to decide what to measure. We need a set of measures that give a clear picture of all aspects of the service, from the point of view of: • • • • the customers of the service; the desired outcomes of the service; our own internal performance in delivering the service; and our efficiency in delivering the service.

124054224.doc

Created: 29-Oct-04

Revised: 14-Mar-05

© 2004 ADB Institute

1 of 8

Continuous Improvement & Benchmarking When we look at the service delivery model, we can see that there are five elements that should be considered when working out what to measure: 1. Inputs 2. Process 3. Outputs 4. Immediate impacts 5. Outcomes
MODEL

Inputs
Including resources

PROCESS PROCESS

Outputs
Immediate outputs of the process

Immediate impacts
Results brought about by the process outputs

OUTCOMES
The outcomes which the city is trying to achieve through this service process.

The key questions in measurement are: are we buying resources economically, are we efficient in providing the service, and is the service effective. Measures should be developed which will clearly answer these three questions.

There are 3 parts of the service delivery model where performance measures are used:

Suppliers

Customers

What we use

What we deliver

What we achieve

Inputs
Process: What we do
Inputs are w hat the process uses. For example, inputs include people, permit applications, boxes of drugs Outputs are w hat the process delivers – for example, ministerial advice provided, w ork permits, patients treated

Outputs

Outcomes

Outcomes are the impacts or results the Outputs contribute to. For example, Outcomes might be “Improved living conditions” or “All children of age 12 to have completed level 6 education”

Input measures measure the inputs to a process, like number of people, number of permit applications etc. Input measures are usually easy to collect and relate to the process rather than the customer. Input measures are usually quality measures that ensure the inputs to the process are not in error or there is no deficiency. Some times the unit cost of an input is important where the item is costly, or where an item is critical in the process, the timeliness of the delivery may be important to monitor.

124054224.doc

Created: 29-Oct-04

Revised: 14-Mar-05

© 2004 ADB Institute

2 of 8

Continuous Improvement & Benchmarking Output measures measure how well the process outputs are delivered to the customer (internal or external). Output measures are usually: • • • • • • Cost - how much does the process or service cost to deliver, e.g. - cost to process a work permit application, and this will include efficiency and economy. Time - e.g. how quickly the outputs are delivered to the customer, and how long the customer waits to be served. Access – the availability of the service to those that want to use it. Quality - how good the output is, e.g. the quality of advice, the absence of typing errors, omissions and mistakes.. Affordability – the price paid for the service and whether or not the price is affordable by those that use the service. Customer service - how satisfied the customers are with the service they receive, e.g. - when they come to the front desk or telephone.

Outcome measures measure how well the objectives of the service are being achieved. The ‘objectives’ measures may be made at the point of immediate impacts or on the long-term service outcomes or both. In most cases the objectives of the service from the City’s perspective will be consistent with the customers expectations. However there may be other objectives, which the City requires but the customer, may not be aware of. For example, in solid waste collection, the City may want to minimize the amount of waste generated; for public health the City may wish to improve health safety levels, and for child immunization the City may wish to increase the rate of immunization protection. We should be in a position to say whether or not a service is meeting the City’s objectives. Defining the measure is often more complex than we first believe. We must be careful in this step because the way we make the measurement will influence interpretation and decisions. To achieve the correct decisions, the measure must be accurate, clear, reliable and consistent over time. The steps in defining performance measures are: 1. Identify the process, its Inputs, the Outputs and the Outcome to be achieved by the process. Then identify the key inputs, key outputs and the most important outcome, that need to be monitored and managed to ensure the service is a “success In the bottom section of the worksheet, write down the input or output plus is aspect (Cost, Timeliness, Accessibility, Quality, Affordability and Customer satisfaction) that needs to be monitored. For example, Raw material cost, Building application quality,
124054224.doc Created: 29-Oct-04 Revised: 14-Mar-05 © 2004 ADB Institute 3 of 8

Continuous Improvement & Benchmarking Rabbit population survey timeliness, Staff timeliness. You should always measure the Outcome. 2. Brainstorm a list of evidence which would tell you most about the delivery of the key Inputs, key Outputs listed in the worksheet you have used in Define the Process. For the Outcome identified, brainstorm a list of evidence that would tell you most about how successful the process has been in achieving its objective. For example, Building application quality may be evidenced by, applications that have been rejected, applications that are incomplete, applications that are re-submitted. The TBL strategy of avoiding road repair run off polluting the storm water catchments, may be evidenced by repair costs, such as pumping or scraping or solvent spraying performed to correct run off pollution. The objective of protecting the motorist from broken windscreens or stone damage may be assessed by monitoring customer complaints, or damage claims or by surveying drivers that live in the area for their satisfaction level. With this information we should now be able to define the Performance Measure Name and nominate the target value for these measures. If you cannot identify a target value at this point, then the Measure Name may need amendment. Where the base of reference for the measure is not constant, then a percentage measure should be used. For example, if we want to monitor the Quality of building applications, the measure may be the number of applications we receive in a month that are incomplete. If the total number of applications was the same every month, this is a valid measure. However if the “base” (being the total number of applications) is growing or is seasonal, then a percentage should be used; “the percentage of applications that is incomplete”. A proportional measure (%) should also be used if Benchmarking with other cities is to be performed, because the base of reference may be different in size. The number of building applications received for a large city will be different to the number received by a small city, therefore a proportional or percentage measure should be used. If a target “value” cannot be set, then it is likely that the measure name is not clear enough or is poorly expressed or cannot be reported with a numerical value. If you cannot “calculate” this measure then it needs to be redefined, because we need to show the measure trend, and this requires a numerical value. 3. Consider the characteristics of Cost, Timeliness, Access, Quality, and Affordability and determine what are most important to customers or to succeed in meeting service goals. For example, Home care services may have Accessibility to home help, as the most important measure, whereas for Child care, Customer satisfaction may be more important than say Affordability. List the best indicators, ideally only around five to six indicators should be required to monitor and manage a service and to satisfy the requirements of a CIB Review. If more are required then it is likely that the service is too wide in its definition. Use the Performance Measure Definition Worksheet to define the performance measures and plan how to collect the data. On the Performance Measure Definition Worksheet the columns refer to the following:
124054224.doc Created: 29-Oct-04 Revised: 14-Mar-05 © 2004 ADB Institute 4 of 8

Continuous Improvement & Benchmarking 1. Performance measure name – this is a very short title that describes the measure; e.g. “The percentage of applications received, that are complete and correct” – its better to work from the positive angle rather than the negative angle – aim high for quality and short periods for timeliness. For customer satisfaction an index can be used with a scale of 1-5. 2. Data required – this names the individual data items required to report the measure. Generally there will be two data items such as total cost and number of outputs delivered; e.g. “time and date of the permit being received, time and date of the application being issued, number of public holidays during the period” 3. Source of data – is the transaction, document or system or process where the data is being sourced from; e.g. date stamped onto the application and date written on the permit, HACC client register, Job sheets, or Asset register; 4. Calculations required – is the arithmetic needed to be applied to the data required to calculate the performance value; it may be a percentage, an average, a median or elapsed time; Excel can assist with these calculations; e.g. A/B*100 5. Responsibility for the data collection – this is the name of the unit or the person responsible for collecting the data, processing the data and providing the performance value for the period. It could be a business unit, a supplier, a person or even an external contracted person; Other information that can be included with this format at your option is: 6. The frequency of reporting the measure; daily, weekly, monthly, quarterly 7. The medium it will be reported on; poster, web page, management report 8. The audience it will be reported to; employees, customers, suppliers, senior management 9. The aggregation path such as all pools added together, or will the measure be reported for each pool; the path can be geographic, or demographic 10. The benchmark value or target that other councils or the private sector is actually achieving or you set for your own performance. Test your measures to ensure they are SMART: •Specific - they should be specific to the area you are measuring. For example, if you are measuring customer service, a good measure would be direct feedback from customers on how they feel about your service. A poorer measure would be the number of customer complaints, because this is indirect and only measures the worst cases of dissatisfaction. •Measurable - you should be able to collect data about them that is accurate and complete. •Actionable - they should be easy-to-understand, and it is clear what is good and what is bad performance, so you know when to take action.

124054224.doc

Created: 29-Oct-04

Revised: 14-Mar-05

© 2004 ADB Institute

5 of 8

Continuous Improvement & Benchmarking •Relevant - this means you shouldn’t measure unimportant things, just for the sake of measuring. A common mistake is to measure everything rather than the key relevant measures. •Timely - your should be able to collect the data when you need it. Measures that rely on data collected months before are useless for managing current performance.

124054224.doc

Created: 29-Oct-04

Revised: 14-Mar-05

© 2004 ADB Institute

6 of 8

Continuous Improvement & Benchmarking

Performance Measure Definition Worksheet
Data required - itemize Performance Measure Name
(what individual pieces of data are needed to build the performance measurement)

Source of Data
(exactly where you can find the individual pieces of data)

Calculations required
(show how the data are used to calculate the final performance measurement)

Person responsible / target date
(who on the team will get the performance data, & when)

Data collected

Individual data 1: 2:

Calculated data

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

1: 2:

Use another copy of this sheet if there are more indicators.

124054224.doc

Created: 29-Oct-04

Revised: 14-Mar-05

© 2004 ADB Institute

7 of 8

Continuous Improvement & Benchmarking
Disclaimer: The views expressed in this Toolkit are the views of the authors and do not necessarily reflect the views or policies of the Asian Development Bank (ADB) or Asian Development Bank Institute (ADBI), or their Board of Directors, or the governments they represent. ADB and ADBI do not guarantee the accuracy of the data included in this CD-ROM and accepts no responsibility for any consequences of their use. Terminology used may not necessarily be consistent with ADB official terms.

124054224.doc

Created: 29-Oct-04

Revised: 14-Mar-05

© 2004 ADB Institute

8 of 8

Sign up to vote on this title
UsefulNot useful