Geoffrey Wright POLS 30595-Essay #2 November 10, 2013 Practical Considerations for the Development of a Community Scorecard With

the increasing demand for improved accountability and governance in healthcare systems, especially those of developing countries, the need for reliable performance measures that allow governments and other stakeholders to identify deficiencies in the current system and to assess the impact of health system strengthening initiatives on service delivery and, ultimately, health outcomes is critical. One such method, the balanced scorecard (BSC) system, has evolved from its origins in the business sector and is emerging as an invaluable tool for governments and health organizations in the developing world as they look to improve quality and access of healthcare services. While the power of the BSC is well-established in the business world, its application to healthcare systems of low- and middle-income countries is limited, albeit promising. A pioneering BSC system carried out in Afghanistan in 2004 (Edward 2011) and CARE Internationals innovative community scorecard (CSC) approach in Malawi (Wild & Harris 2012) have both proven effective in monitoring and evaluating the progress of health service capacity and delivery. However, due to the diverse and complex nature of health systems and the fact that issues are addressed differently at the national, district, and community levels, no single scorecard method has emerged as a universal solution to address the need for better accountability and governance. Efforts to combine current models with novel ideas are needed to help better the strategic management of resource-limited health systems worldwide. Although terminology varies, researchers implementing the scorecard commonly emphasized the importance of developing relevant, site-specific performance indicators with measurable and achievable benchmarks while, at the same time, understanding the contextual perspectives in relation to scorecard implementation. This paper begins to analyze several of the methodologies used in the development and implementation of a

scorecard system, with the intent of applying a revised version to future health systems strengthening initiatives (HSSI) in Zambia. Developing Relevant Indicators In order for researchers and policy makers to maximize the benefits of a community scorecard, they must first ensure that the indicators are 1) measurable and given achievable benchmarks based on the realities of the country (Edward et al. 2011) and 2) are relevant to the population being served. Setting high, but achievable benchmarks not only gives hospitals and other health settings being evaluated a standard to strive for, but it also makes it easier to compare performance across hospitals/districts and identify areas that are lacking, say, in leadership, drug availability, or financial support. The power of the scorecard system lies in its ability to identify these deficiencies and observe the effects of resulting interventions. To see this in practice, the BSC study in Afghanistan utilized upper benchmarks (termed the green zone) set at a level that, in 2004, were being achieved by the six best performing provinces (out of 33 total). The red zone, or the bottom quintile, represented those provinces performing the worst in a particular sector, thus emphasizing areas needing special attention. These results provided much needed accountability and transparency for struggling health systems at the provincial level in Afghanistan. However, there is a recognized need to continue breaking down this scorecard system even further, into community and facility levels, if the most marginalized groups are to be reached. With this in mind, and with the possible future implementation of a community scorecard system specific to Zambia, how do researchers go about developing relevant and robust indicators? It is possible for indicators to be adopted from previously used scorecards. In the case of Zambia, for example, it may be useful to borrow items from the scorecard formulated by Mutale et al. (2013) for their research of the health facilities in three rural districts of Zambia. Their performance indicators fell into one of six domains: patient and community, human resources, service provision, and health systems preparedness for equipment, essential commodities and infrastructure (Mutale et al. 2013).

However, developers must be resolute in ensuring that the indicators are relevant to the particular population being served and a deep contextual knowledge of both the community and the service providers is necessary for the assessment to be most effective. Community involvement helps those developing the scorecard to better assess the realities of the country at the micro-level and ensures that the population they intend to serve has a voice. Furthermore, this involvement may be beneficial in fostering a more inclusive and participatory culture when implementing the scorecard, something Rabbani et al. (2011) highlights is a vital component to a successful and iterative scorecard system. The World Bank outlines a general methodology for developing a community generated performance scorecard (Singh & Shah): Divide Into Focus Groups: After gathering the community, participants will be classified into focus groups, ensuring that a heterogeneous mix of members is included in each group, based on age, gender, and occupation, among other things, in order to stimulate a discussion that allows for marginalized groups in the community to be represented and give any input that may have otherwise been ignored. Develop Performance Indicators: The focus groups brainstorm to develop criteria with which to evaluate the health facilities or services in question. The facilitator then leads the group in organizing the suggestions into measurable or observable indicators. Participation is crucial in this step to bring out a critical mass of objective criteria (Singh & Shah). Finalize Indicators: Indicators will be prioritized and reduced, perhaps through corroboration with the MOH or other stakeholders. In the Afghanistan BSC, 340 potential indicators were analyzed for face validity and importance, reliability, completeness, outlying values, and variation (Peters et al. 2007) and ultimately reduced to only 29 in the final BSC. The final number of indicators varies from scorecard to scorecard, but, as Peters et al. explained in their methodology, the final scorecard should have a limited

number of easily understood and robust indicators that represent the most important aspects of service delivery (2007). Ask Groups to Score Each Indicator: In order to get a relative idea of where the community stands on the issues decided upon, participants will be asked to score each of the performance criteria, typically on a scale from 1-100. Ask Groups to Explain High/Low Scores: This step is important in drawing out peoples perceptions by having them explain their reasoning for high or low scores. This can help researchers to explain outliers and can provide them with anecdotes regarding service delivery that can be useful when thinking about future interventions. The document adds that suggestions from the community as a whole regarding the new performance criteria they helped to establish would be insightful to conclude the process and move on to implementation. Once the scorecard has been finalized and implemented for a baseline study, the World Bank proposes a unique strategy that was initially carried out by CARE International in Malawi in 2002. Their innovative concept of conducting interface meetings in which the beneficiaries and service providers meet to discuss the results of the scorecard assessment has been shown to be effective in ensuring that community feedback is taken into account and that deficiencies in service delivery are acknowledged. The development of relevant indicators that are both measurable and achievable is a vital piece in establishing a successful community scorecard system. Setting upper and lower benchmarks are useful when comparing across facilities or communities and when analyzing the effectiveness of interventions over time. The central ideas surrounding the development of a scorecard, though, involve flexibility and valued input from the community. These factors are crucial in ensuring an inclusive and participatory culture, giving the scorecard a contextual basis that allows it to serve as an evidence-based decision-making tool for low- and middle-income communities.

Works Cited

Edward A, Kumar B, Kakar F, Salehi AS, Burnham G, et al. (2011). Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years' Evaluation in Afghanistan. PLoS Med 8(7): e1001066. doi:10.1371/journal.pmed.1001066 Mutale W, et al. "Measuring Health System Strengthening: Application of the Balanced Scorecard Approach to Rank the Baseline Performance of Three Rural Districts in Zambia." PLoS One E58650 8.3 (2013): 1-11. Print. Peters D. "A Balanced Scorecard for Health Services in Afghanistan." Bulletin of the World Health Organization 85.2 (2007): 146-51. Print. Rabbani, F. "Understanding the context of balanced scorecard implementation: A hospital-based case study in Pakistan." Implementation Science 6 (2011). Singh J, and Shah P. "Community Score Card Process: A Short Note on the General Methodology for Implementation." The World Bank. Web. 11 Nov. 2013. <http://siteresources.worldbank.org/INTPCENG/11433331116505690049/20509286/comscorecardsnot e.pdf>. Wild, L., and D. Harris. 2012. "More than Just 'demand': Malawi's Public-service Community Scorecard." Project Briefing: 69. Overseas Development Institute. Print.