You are on page 1of 19

This article was downloaded by: [Arif Setya

]
On: 27 January 2015, At: 06:30
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Environmental Sciences
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/nens19

General Policy Questions Need
Specific Policy Effectiveness
Indicators: An Example for
Emission Reduction Targets
a

Tinus Pulles & Toon van Harmelen

a

a

TNO Environment, Energy and Process Innovation ,
Apeldoorn, The Netherlands
Published online: 02 Feb 2010.

To cite this article: Tinus Pulles & Toon van Harmelen (2004) General Policy Questions
Need Specific Policy Effectiveness Indicators: An Example for Emission Reduction Targets,
Environmental Sciences, 1:4, 351-367, DOI: 10.1080/15693430412331314845
To link to this article: http://dx.doi.org/10.1080/15693430412331314845

PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information
(the “Content”) contained in the publications on our platform. However, Taylor
& Francis, our agents, and our licensors make no representations or warranties
whatsoever as to the accuracy, completeness, or suitability for any purpose
of the Content. Any opinions and views expressed in this publication are the
opinions and views of the authors, and are not the views of or endorsed by
Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and
Francis shall not be liable for any losses, actions, claims, proceedings, demands,
costs, expenses, damages, and other liabilities whatsoever or howsoever caused
arising directly or indirectly in connection with, in relation to or arising out of the
use of the Content.
This article may be used for research, teaching, and private study purposes.
Any substantial or systematic reproduction, redistribution, reselling, loan, sublicensing, systematic supply, or distribution in any form to anyone is expressly

tandfonline.Downloaded by [Arif Setya] at 06:30 27 January 2015 forbidden.com/page/terms-and-conditions . Terms & Conditions of access and use can be found at http:// www.

1080/15693430412331314845$16. 351–367 RESEARCH ARTICLE General Policy Questions Need Specific Policy Effectiveness Indicators: An Example for Emission Reduction Targets Tinus Pulles and Toon van Harmelen Downloaded by [Arif Setya] at 06:30 27 January 2015 TNO Environment.Environmental Sciences 2003/2004. The Netherlands. 1. Apeldoorn. resulting in conflicting and mutually inconsistent data interpretations. INTRODUCTION 1.P.00 # Taylor & Francis Ltd. Scientific and technical policy advisors should be aware of this when developing indicators to answer relatively simple policy questions.1. emission reduction 1.Pulles@mep. Information for Environmental Policy Interactions and processes in the environment are very complicated and.J. E-mail: M. International targets. We concentrate on indicators for progress towards emission reduction targets for greenhouse gases (Kyoto) and air pollutants (EU National Emission Ceilings Directive) within the European Union. pp. TNO Environment.nl 10. . Energy and Process Innovation. Vol. developed to inform the policy makers and the public on the progress of environmental policies towards the targets set.tno. Energy and Process Innovation. Moreover. 7300 AH Apeldoorn. The Netherlands ABSTRACT This paper describes the importance of different policy perspectives and contexts on the numerical values of policy effectiveness indicators. many of the interactions are not fully understood or still subject to scientific debate. Science in many cases responds to this by adding more and more data and information to the existing body of knowledge. Keywords: indicators. It is shown that both the policy perspective (monitoring. PO Box 342. planning and strategy) and the context (individual EU Member State or the European Union as a whole) have a strong influence on the relative score of the Member States. No. environmental information is in most cases rather complex. Address correspondence to: Tinus Pulles. hence. 4. The choices they make on perspective and context have a strong influence on the answers they produce.

1. This approach is often taken in the framework of international conventions like the United Nations Framework Convention on Climate Change (UNFCCC) where indicators are used to make data in different regions comparable. however. In environmental policy. an indicator is developed to simplify a complex state or development in the real world in such a way that a specific policy maker’s question is answered. The definitions as given in the table all clearly recognize that the indicator is not the real world itself. Such indicators necessarily simplify the complex environmental information.Downloaded by [Arif Setya] at 06:30 27 January 2015 352 TINUS PULLES & TOON VAN HARMELEN Environmental decision makers. Moreover. related to the exact policy questions to be answered. 2003b) or in reports in response to obligation of in- .2. As an example. we develop different indicators to answer the same policy question: ‘‘How far are individual European Union Member States and the European Union as a whole on their road towards meeting the emission reduction targets of Kyoto (greenhouse gases) and Gothenburg (long range trans-boundary air pollution)?’’. The Concept of ‘Indicators’ As indicated above. EEA. produced for a specific purpose. need quick and simple overviews that inform them on both the causes of environmental problems. The indicator developers will apply specific simplifications and aggregations. different indicator developers might use different simplifications and aggregations in answering the same policy question. scientific and technical policy advisors design socalled indicators to answer policy questions. In this study. To this end. It is an approximation. many indicators are purely meant to support the reporting of progress in the state of the environment. The consequences of the implicit choices in the indicator development are not frequently communicated to the indicator users and the limitations caused by the simplifications are not always seen and understood by the users. the state of the environment and likely consequences of the decisions they are preparing and taking. This is reflected in the various definitions listed in Table 1. either in State of the Environment Reports like those of the European Environment Agency (e.g. Choices on simplifications and aggregations might influence the developed indicators and hence the answers the indicators produce to specific questions. we investigate the consequences of different simplification and aggregation algorithms to convert data on progress towards an environmental target to indicators used in the policy process.

A parameter or a value derived from parameters that describe the state of the environment and its impact on human beings. A gauge or meter of a specified kind.drfurfero. Ex post.askoxford. European Environment Agencyz 4. the pressures on the environment. an indicator has predictive value. Source Downloaded by [Arif Setya] at 06:30 27 January 2015 1. Indicator – a number or ratio (a value on a scale of measurement) derived from a series of observed facts.) An information board or screen in a railway station. 2. Sources:  http://www. (Brit. An indicator is a variable or group of variables that provide policy makers with information. per capita waste production and many other indicators used . the driving forces and the responses steering that system. Other examples are emissions of acidifying pollutants.html ternational conventions and protocols. A flashing light on a vehicle to show that it is about to change lanes or turn. A thing that indicates a state or level.eu. 2003).int/EEAGlossary/E/environmental_indicator § http://www. 4. ecosystems and materials. before a policy measure is implemented. 5. Ex ante. This information has two different types of value. The Free Dictionaryy 3. total mass of certain waste streams. 3. after a policy measure has been implemented. annually reported by parties to the UNFCCC (UNFCCC.com/indicator z http://glossary. A. (Chemistry) A compound which changes colour at a specific pH value or in the presence of a particular substance. airport. An example of this is the national total emissions of greenhouse gases.eea. etc.com/books/231book/ch13a. an indicator has effectiveness value. can reveal relative changes as a function of time. An indicator has gone through a selection and/or aggregation process to enable it to steer action.POLICY EFFECTIVENESS INDICATORS 353 Table 1. expressed as CO2 equivalents. Joyce Furfero (2000)§ Definition 1. The Oxford Dictionary of English 2. and can be used to monitor a chemical change. Some Definitions of the Concept ‘Indicator’.thefreedictionary.com/concise_oed/indicator?view ¼ uk y http://www.

different alternatives could be chosen to develop specific indicators to answer the policy effectiveness question. In many cases. Obviously this might lead to different answers and both indicator developers and indicator users should be aware of this. we will derive in this paper different answers to the same policy question: ‘‘How far are individual European Union Member States and the European Union as a whole on their road towards meeting emission reduction targets?’’ in two policy fields:  Both the European Union (EU) and its Member States are parties to UNFCCC and have ratified the Convention’s first protocol on reducing the emission of greenhouse gases.3. 1. 2003b). Although they are sometimes still under discussion (for instance: the values of ‘global warming potential’ to be used in calculating CO2 equivalents (Ramaswamy. the EU has 25 Member States. (The UNFCCC and its Kyoto Protocol were negotiated when the EU had 15 members. the EU has developed legislation to ensure reduction of air pollutants according to the Gothenburg protocol by implementing national emission reduction targets in the National Emissions Ceiling Directive (EC. That is why in this study we use data for these 15 EU members only. both the EU and individual Member States have committed themselves to implement policies. calculation is rather straightforward.) So. As an example. Since June 2004. ensuring that they will meet these Kyoto targets for the first commitment period 2008–2012. The concept of effectiveness assumes some kind of reference or comparison to other situations or possibilities. Such indicators are the ones covered by definitions 1. 2 and 3 in Table 1. Policy Effectiveness Indicators Definition 4 in Table 1 adds a specific dimension to the indicator: the issue of effectiveness of policy decisions.354 TINUS PULLES & TOON VAN HARMELEN Downloaded by [Arif Setya] at 06:30 27 January 2015 in the EEA’s report Europe’s Environment: The Third Assessment (EEA. This type of indicator is relatively unambiguous. . 2001)). the so-called Kyoto Protocol. The ‘original’ 15 EU Member States took part in the EU burden sharing agreement. either ex ante while preparing decisions or ex post when monitoring the results of a decision. 2001) to be reached by 2010.  Both the EU and all of its present and future Member States are parties to the convention on Long Range Transboundary Air Pollutants (LRTAP) and as a consequence.

Context Member State (MS) level European Union (EU) level Perspective Indicator MS monitoring Done MS MS planning To do MS MS policy strategy Extrapolated gap MS EU monitoring Done EU EU planning To do EU EU policy strategy Extrapolated gap EU Meaning The effort already delivered by each Member State The remaining emission reduction requirement for each Member State The expectation (chance) of each Member State not meeting its target The contribution of each Member State to the total emission reduction achieved in the EU The dependency of the EU on the emission reduction still to be achieved by each Member State The sensitivity of the EU meeting its target to the chance that a Member State will not meet its target . aimed at adjusting existing policies when needed. In addition it could have a strategic perspective.355 POLICY EFFECTIVENESS INDICATORS An indicator developed to answer this policy question. a second dimension is important while developing an indicator to answer the question above in a complicated multinational situation as in the EU. could have an ex ante and an ex post perspective: it is looking back at the recent past and it looks forward towards the commitment period. These perspectives coincide with the different stages of the policy process: Downloaded by [Arif Setya] at 06:30 27 January 2015  the monitoring perspective: what has been done?  the planning perspective: what is still to do? and  the policy strategy perspective: are we doing well enough to meet the targets? Apart from these three perspectives. Different Context and Perspectives on Policy Effectiveness Indicators for Emission Reduction Targets. The indicator could be developed within Table 2.

The EU Member States however have since Kyoto made use of a provision of the protocol (article 3. 2002). called the ‘burden-sharing’ agreement. The EU agreed to a reduction of 8% for the averaged annual emissions during the first commitment period (2008–2012) relative to the 1990 emissions. Due to specific circumstances. to describe the progress of EU Member States towards the targets of the Kyoto Protocol. These data are summarized in Table 3. some countries could relatively easily accept a large reduction target (Luxembourg. Denmark. The base year for the fluorinated gases is 1995 in many of the 15 original EU Member States. the EU and its Member States agreed in 2002 on different emission limitation and/or reduction targets for each Member State according to economic circumstances. whereas others committed to limiting their emission increases to enable economic growth .1. In Council Decision 2002/358/EC (EC. (In our analysis we set the target year to 2010. part 1 of the Protocol (UNFCCC. Data Used The greenhouse gas data used in this study are obtained from a report. 1997)) that allows parties to meet the targets jointly. 2.) Each of the Member States in Kyoto agreed on the same emission reduction target. MATERIALS AND METHODS 2. 2003a). UK).356 TINUS PULLES & TOON VAN HARMELEN different scopes or contexts: Downloaded by [Arif Setya] at 06:30 27 January 2015 (1) The Member State context: (a) What are the efforts undertaken by the Member States? (b) What are the requirements the Member States still need to accommodate? (c) What are the expectations or chances that the Member States will not meet the target? (2) The EU context: (a) What is the contribution of Member States to meeting the EU target? (b) How dependent is compliance at the EU level on the efforts the Member States still have to do? (c) What is the sensitivity of EU compliance or what risk does the EU run with respect to Member States not meeting the national targets? Table 2 summarizes these different contexts and perspectives. produced by the European Environment Agency (EEA. This paper designs six different indicators (listed in the third column of Table 2) for each of the three perspectives in both contexts. Germany.

5 657. .0% 7. The most recent complete data in this source are the emissions for the year 2000. volatile organic compounds (VOC) and NH3 by 2010 to the levels given in the Annex I of the Directive. Greece.2 4108. Ireland).0% 6.0% 4.4 509.4 6.0% 0.0% 21. Spain.9 747.4 80.0% 0. EU and Member States Greenhouse Gas Emission Trends (1990 and 2001) and Targets (Burden-Sharing) for 2008–2012. The emission reduction targets as given in Table 3 are the ones after the burden sharing.2 558.0% 27.0% 25.7 83.1 61.0 545.9 560.4 289.4 1216. The data on SO2 and NOx of the NEC Directive (Table 4) used in this study have been downloaded from the electronic Data Service on the web site of the European Environment Agency (EEA.9 72.9 78. the location where emissions of the NEC Directive pollutants take place is quite relevant for the environmental impact. (Portugal.2 69. This is reflected by the fact that no joint emission reduction target for the EU is defined and a burden-sharing mechanism is not negotiated.0% 6. Different from the climate case. In this study we use the SO2 and NOx data only.3 10. NOx. Also the emissions for the base year 1990 are taken from this data source.0% 13.2 107.3 141.0% 12.0% 15. EU total Austria Belgium Denmark Finland France Germany Greece Ireland Italy Luxembourg Netherlands Portugal Spain Sweden UK Base year (1990) (million ton of CO2 equivalents) 2001 (million ton of CO2 equivalents) EU Burden sharing target (% of 1990 emission) 4203.357 POLICY EFFECTIVENESS INDICATORS Downloaded by [Arif Setya] at 06:30 27 January 2015 Table 3.9 150.2 70.5 132.0% 13.8 70.5% 28.8 993.1 219.0 53.5% 21.9 211.8 382.5 77. The NEC Directive requests the EU Member States to reduce the national emission totals of SO2. Table 4 also presents the 2010 national targets according to the NEC Directive.4 85.2 8.2 69.5% From: Table C4 in Greenhouse Gas Emission Trends and Projections in Europe 2003 (EEA. 2003a). 2004a).

NECD targets from Handley and Taylor (2003). the extrapolation of ‘Done’ for the period current year to target year equals ‘To do’ resulting in an ‘Extrapolated gap’ of 0. Only if ‘Done’ is on the straight line from base year emission to target emission. 2004a). EU and Member States Emission Trends (1990 and 2000) and Targets (National Emissions Ceiling Directive NECD) for 2010 for NOx and SO2. . It is in fact a linear combination of the indicator ‘To do’ and ‘Done’.358 TINUS PULLES & TOON VAN HARMELEN Table 4. Algorithms Figure 1 explains the calculation for a certain year (‘current year’ in the graph) of the progress towards meeting the target according to the three following perspectives:  Monitoring perspective: the effort that has been delivered from base year to the current year is indicated by the arrow marked ‘Done’ that gives simply the decrease in emissions from base year to the current year.  Strategy perspective: the arrow marked ‘Extrapolated gap’ indicates the linearly extrapolated over.2.or undercompliance in the target year. Downloaded by [Arif Setya] at 06:30 27 January 2015 NOx (thousand ton/year) Austria Belgium Denmark Finland France Germany Greece Ireland Italy Luxembourg The Netherlands Portugal Spain Sweden UK SO2 (thousand ton/year) Base year (1990) 2000 NECD 2010 Base year (1990) 2000 NECD 2010 204 334 277 300 1900 2730 290 118 1940 23 570 286 1270 334 2760 196 329 209 236 1440 1580 321 125 1370 17 413 405 1430 252 1740 103 176 127 170 810 1051 344 65 990 11 260 250 847 148 1167 79 362 180 260 1320 5320 493 186 1650 15 202 288 2180 106 3720 38 165 28 74 654 638 483 131 758 3 92 288 1520 57 1190 39 99 55 110 375 520 523 42 475 4 50 160 746 67 585 From: EEA Data Service (EEA.  Planning perspective: the arrow marked ‘To do’ indicates the emission reduction required from the current year to reach the target emission in the target year. 2. as is graphically depicted in Figure 1.

55 ((2001–1990)/(2010–1990)) smaller. Please note that indicators calculated in the EU context for each individual Member State will add up to the same indicator for the EU as a total. Table 5 lists the equations used to calculate the indicators. In this case the EU indicator will be the weighted average of the Member State indicators. 2003a) therefore are a factor of 0. but these will always need more data and information. we calculate the emission values relative to the national base year emission for each Member States separately (MS context) or the aggregated EU base year emission (EU context). The numerical values in the EEA’s report (EEA. This obviously does not need to be the case. The indicators in the Member State context will not. To express the two different contexts. 1. the difference is only that we relate the gap to the 2010 target. the indicator ‘Distance to Target’ used in the EEA’s report Greenhouse Gas Emission Trends and Projections in Europe 2003 (EEA. Reduction possibilities might have been partly exhausted in the reporting year or the country might have policies in place or preparation that take effect later in time. whereas the EEA’s report is relating the gap to the interpolated 2001 target. More sophisticated approaches might be developed. The interpolation in both the distance to target indicator and our extrapolated gap indicator implicitly assumes that the target is reached along a linear time path. Calculating indicators for the different perspectives.Downloaded by [Arif Setya] at 06:30 27 January 2015 POLICY EFFECTIVENESS INDICATORS 359 Fig. 2003a) is equivalent to our ‘Extrapolated gap MS’ indicator. .

MS Done EU Idone.year EMS. E stands for the emission of MS or EU in.MS ¼ EMS.EU Explanation Calculates the reduction achieved relative to the Member State base year emission Calculates the reduction still to be achieved. relative to the EU base year emission Calculates the linearly extrapolated over. The Algorithms Used to Calculate the Six Different Policy Effectiveness Indicators. I. stands for the indicator.base To do EU Ito do.or undercompliance in the target year.base To do MS Ito do.year EEU.EU ¼ EMS. the base year (tbase. respectively. the current year (t) and the target year (ttarget.360 Table 5. 1990 in the Kyoto Protocol).MS ¼ EMS.target EMS.EU ¼  Ito do.MS ¼  Ito do.target EMS.MS  tbasetarget t  Idone.year EMS. relative to the EU base year emission Note.or undercompliance in the target year. relative to the Member State base year emission Calculates the linearly extrapolated over.EU  tbasetarget t  Idone.base EMS.year EMS. Algorithm Done MS Idone.EU ¼ EMS.base EEU. we use 2010 as the average of 2008–2012). TINUS PULLES & TOON VAN HARMELEN Downloaded by [Arif Setya] at 06:30 27 January 2015 Indicator . with appropriate suffixes.base Extrapolated gap EU   tt Igap.base Extrapolated gap MS   tt Igap. relative to the Member State base year emission Calculates the reduction achieved relative to the EU base year emission Calculates the reduction still to be achieved.

in the similar indicator in the European context (the dependency of the EU on each Member State meeting the targets). having the sixth largest emission out of 15 countries. whereas Portugal is ‘the worst’. The differences between the indicators can sometimes easily be understood.  The ‘To do’ indicator expressing the reduction requirements still remaining for each individual Member State points to Luxembourg again as ‘the best’.  The ‘Extrapolated gap’ indicator in the Member States context shows again Luxembourg as ‘the best’ and this time Ireland as ‘the worst’. results in Germany as ‘the best’ and Spain as ‘the worst’. RESULTS Downloaded by [Arif Setya] at 06:30 27 January 2015 Figure 2 graphically presents for climate change the values for the six indicators. . the corresponding indicator in the EU context now shows Germany as ‘the best’ and Spain as ‘the worst’. resulting in relatively high (positive or negative) impacts for large (emitting) countries compared with small (in terms of emission level) Member States. These results show that not only the pattern of numerical values is sensitive for the different indicators. however. In fact. the indicator for the contribution given to the total EU achievement. calculated by the algorithms of Table 5. The indicators in the left part of the graph are reflecting the Member State context while the indicators on the right part of the graph represent the EU context. the Netherlands. the middle scores on the indicators according MS context can be explained by the fact that the Netherlands scores better than relatively small countries such as Portugal. Each indicator is sorted from ‘the worst’ to ‘the best’. The difference between the Member State and the EU context is rooted in the relative size of the base year emission. whereas Austria now is ‘the worst’. Ireland. Sweden is ‘the best’ and Italy is ‘the worst’.POLICY EFFECTIVENESS INDICATORS 361 3. ‘To do’ and ‘Extrapolated gap’ are presented from the top of the graph downwards. but also the ranking of the individual Member States is quite different for each indicator:  For the indicator ‘Done’ representing the ‘effort’ delivered by each Member State. scores relatively low in the EU context compared with the MS context. The indicators ‘Done’. Austria and Greece. But this is not very relevant in reaching the EU target. For example. resulting in a worst score at the ‘Extrapolated gap’ indicator with a third worst place after Spain and Italy. Luxembourg appears as ‘the best’.

. The six different indicators for distance to target of greenhouse gas emissions in 2001 for the 15 EU Member States separately (dark bars) and the EU as a total (light bars). By this example it becomes clear that using specific indicators for different contexts can acknowledge and clarify differences in opinion and sense of urgency on Member States’ progress in policy. 2.TINUS PULLES & TOON VAN HARMELEN Downloaded by [Arif Setya] at 06:30 27 January 2015 362 Fig.

the differences between scores on indicators according different perspectives are less easy to understand. In Figure 3 the highest. Greece. However. France. For six countries (Finland. The ‘Extrapolated gap’ indicator is adding more complexity since it is a linear combination of the other two indicators. Although one might expect that scoring high on ‘Done’ would lead to scoring low on ‘To do’. lowest and average ranking for each Member State are given. Luxembourg. this is not always the case since emission reduction targets differ substantially from country to country. Denmark and Portugal) the highest and lowest ranking differ eight or more places.Downloaded by [Arif Setya] at 06:30 27 January 2015 POLICY EFFECTIVENESS INDICATORS 363 Fig. six or seven places. a similar analysis is presented in Figure 4 for the NEC Directive reduction targets for NOx and SO2 in the 15 EU Member States. To establish whether or not this behaviour is typical for the greenhouse gas emissions and the Kyoto targets. Belgium. For other countries (Germany. Austria and Ireland) the highest and lowest rank differ between five. The rankings for individual Member States on the six different indicators for climate change policies. For the other countries (Italy. the EU Context is not relevant. since there is no joint target . countries are sorted according to increasing averaged ranking. Spain. The highest ranking differs three places or less from their lowest ranking. In this case. 3. the Netherlands. Sweden and the UK) the ranking is not very sensitive to the choice of the indicator.

TINUS PULLES & TOON VAN HARMELEN Downloaded by [Arif Setya] at 06:30 27 January 2015 364 Fig. The figure shows that also for these pollutants the ranking of each Member State differs . 4. The figure therefore only presents the results for the three perspectives in the Member State context. Rankings of EU Members States on the three MS context distance to target indicators for NOx and SO2 against the NEC Directive targets for 2010. or burden sharing mechanism in place.

planning and strategy. six (greenhouse gases) or three (NEC Directive) simple indicators generate a variety of scores for a majority of countries. we put forward three definitions of effectiveness according to three so-called perspectives. Do we address effectiveness in terms of what has been done. We illustrated this with the cases of greenhouse gas policies and the air pollutant policies within the EU. what is still to be done or what is expected to be the result in the target year? Each perspective stands for a different stage of the policy cycle. The policy question ‘‘How far are the EU and its Member States on its road to meeting the emission reduction targets?’’ for both policy areas has been translated into different indicators. For each of these perspectives. The indicators as discussed in this paper assess the total effect of all policy measures together. So. Moreover. The differences seem to be smaller than in the case of the greenhouse gases in Figure 3. we have shown that a certain type of policy indicators addressing the effectiveness of policies is subject to a more ambiguous interpretation. we developed three accompanying indicators. since many general (e. we argued that the used definition of policy effectiveness can be dependent on the government (level). Downloaded by [Arif Setya] at 06:30 27 January 2015 4. even in these straightforward examples. This is due to the variety of possible (and thus subjective) definitions regarding the issue at stake here: policy effectiveness. since they are based upon the national total emissions only. Hence. energy prices) . Nevertheless. respectively monitoring.g. Because of this complexity. depending on the perspective and context. but this is mainly caused by the fact that the EU context is not used here.POLICY EFFECTIVENESS INDICATORS 365 quite a lot for the three indicators used here. the indicators need to be specific to contexts as well. What is meant exactly by policy effectiveness? To this end. In addition to this. DISCUSSION AND CONCLUSIONS Indicators to support policy decisions are usually quite straightforward and easy to understand. This is in fact the purpose of developing such indicators: to simplify a complex state or development in the real world in such a way that a specific policy maker’s question is answered. favourable national indicator values do not guarantee the existence of a conscious and effective policy. indicators have to be defined and explained explicitly in order to be able to address specific aspects of policy effectiveness.

Official Journal L130. This adds complexity to the data and the analysis that goes beyond the scope of this paper. 27 November 2001. Hence. 22–30. 36. (2001). 2004b). completely different rankings for EU Member States occur.int/ environmental_issue_report_2003_36/en EEA. preferably in the form of explicit indicators. scapegoats mostly come in herds. Therefore. general policy questions. Greenhouse gas emission trends and projections in Europe 2003. Official Journal L 309. This is surprising since the use of a single policy effectiveness indicator is internationally widespread and accepted to make a simple and convincing point. 10.Downloaded by [Arif Setya] at 06:30 27 January 2015 366 TINUS PULLES & TOON VAN HARMELEN and specific (e. p. policy makers will also need indicators that show the efficiency of individual policy measures. unification) factors influence national emission developments.eea. EC. European Environment Agency. It is even more surprising that hardly any literature exists on the proper use of policy effectiveness indicators.int/assessment_ report_2003_10/en . usually covering several parties. European Environment Agency. (2002). 1. The present paper showed that even with simple examples (15 actual reported national totals to be related with 15 different targets and. in the case of greenhouse gases. Council Decision of 25 April 2002 concerning the approval. Environmental assessment Report No. just as ministering angels do. 15 May 2002. Only when several interpretations related to different perspectives and contexts are addressed explicitly. robust conclusions can be drawn on the effectiveness of policies. based on the results of another recent EEA report on air pollution (EEA. (2003b). Directive 2001/81/EC of the European Parliament and of the Council of 23 October 2001 on national emission ceilings for certain atmospheric pollutants. EEA.eu. capped by a EU total). We conclude that policy effectiveness is such an ambiguous term that the use of only one indicator should always cause suspicion. In a further study we will investigate this issue further. URL: http://reports.eea. Environmental issue Report No.g.eu. can only be addressed effectively by more than one specific policy effectiveness indicator. (2003a). on behalf of the European Community. Europe’s environment: The third assessment. of the Kyoto Protocol to the United Nations Framework Convention on Climate Change and the joint fulfilment of commitments thereunder (2002/358/CE). pp. It would create an understanding between parties if different contexts and perspectives are acknowledged. REFERENCES EC. Because in the real world. URL: http://reports.

T. Cambridge: Cambridge University Press. An initial assessment of Member States’ national programmes and projections under the National Emissions Ceiling Directive (2001/81/EC).com/books/231book/book.int/dataservice/metadetails. In J. (2004a). and theories (8th ed. (2001). V. URL: http://dataservice. ETC/ ACC Technical Paper 2003/8. Topic Report No.int/program/mis/ghg/submis2003. data downloaded in April 2004.eu. Addendum.). Report of the conference of the parties on its third session.1.).eu. ETC-ACC 2003. UNFCCC. eionet. (2003). EEA.J.Downloaded by [Arif Setya] at 06:30 27 January 2015 POLICY EFFECTIVENESS INDICATORS 367 EEA. asp?id ¼ 620. Houghton (Ed. Part Two: Action Taken by the Conference of the Parties at its Third Session. Radiative forcing of climate change.int/reports/ETCACC_TechPaper2003_8_NECD_InitAss Ramaswamy.eu. Held at Kyoto from 1 to 11 December 1997. 18 March 1998. Macroeconomic stabilization policies: Goals. & Taylor. A. Air pollution in Europe 1990–2000. Trends in emissions of ozone precursors (CLRTAP/EMEP). UNFCCC. P. C. Available: http://air-climate. All national submissions of Greenhouse Gas Emission inventories and National inventory Reports in 2003 can be found at UNFCCC’s web site at URL: http:// unfccc.html . institutions.int/topic_report_2003_4/en Furfero. URL: http://www. Climate Change 2001: The Scientific Basis. FCCC/CP/1997/7/Add.eea. (2003).eea. European Environment Agency.drfurfero. (1997). (2000). URL: http://reports.html Handley. (2004b). 4/2003. European Environment Agency’s data service.