This action might not be possible to undo. Are you sure you want to continue?
A TOOLKIT FOR APPLYING PARTICIPATORY APPROACHES
This project was part funded by REEEP
MONITORING AND EVALUATION OF THE IMPACT OF RENEWABLE ENERGY PROGRAMMES A TOOLKIT FOR APPLYING PARTICIPATORY APPROACHES
Written by Kavita Rai, IT Power Acknowledgements to: John Green, Teresa Marston, Jayantha Gunasekara, Gisela Vogt, Wendy Annecke, and Mila Jude. Designed and edited by Naomi Davidge Sketches by Emanuele Scanziani
................................................................................................................... Determining indicators for PM&E ......................................................................................... Strengths........................................................................................................................................................................ Evaluation .................................. Why is monitoring and evaluation important? .................................... 39 Renewable energy projects and the Millennium Development Goals .................................................................................. Renewable Energy Projects considered in this Toolkit .............................. Planning and designing the PM&E ................................................................................. Understanding participation in development and renewable energy intervention ......................................................................................... Personal interviews ............................................... Why should M&E be conducted regularly? ............................................................................................................................ Participatory rural appraisals ................................... 44 Analysing information from participatory approaches ... Output vs......................................... Acronym ......................... Who are the potential users of the Toolkit? ... Outcome/Impact ..................................................................................... Designing the M&E process ....................................................................................................................................................................................................... Using an equity oriented participatory approach ................................... Opportunities and Threats (SWOT) analysis ...................................................................................... 1 SECTION 2: MONITORING AND EVALUATION ........................................................................................ Weaknesses... 46 References ................... How do you approach a community? ......................................................... 50 ...................... 46 Building ownership of participatory monitoring and evaluation ................................................ Survey ....................................... Stakeholder meeting and analysis ................................................................................................................................................................... Why do we need a Toolkit? ............................ 4 SECTION 3: PARTICIPATORY MONITORING & EVALUATION 8 SECTION 4: PARTICIPATORY METHODS ........... Monitoring ........................................................................ SECTION 1: INTRODUCTION .............................................. Basic elements of participatory monitoring and evaluation .................................................................................................................................................... 19 19 25 27 27 37 37 SECTION 5: ASSESSMENT INDICATORS ........................................ Approach used in the Toolkit ........................................................................................................ 49 SECTION 6: ASSESSMENT OF RESULTS .............................TABLE OF CONTENTS i ii 1 1 1 2 3 4 4 5 6 6 8 8 9 11 12 15 16 Glossary ..... A participatory approach to monitoring and evaluation ................................................................................. The Content of the Toolkit ..........................................
peoples' activities and interests. environmental features. often conducted to check differing or similar views or changes on the same indicator. increase in knowledge and skills of stakeholders resulting from activities generated by the project or programme intervention. institutions and environment caused by the energy programme or project intervention. negative. and behaviours that enable people to express and analyse the realities of Rural their lives and to plan. Triangulation Triangulation is a way to confirm datasets through different methods or Wealth Ranking i . monitor and evaluate their actions accordingly. intended or unintended. 'social equity' means keeping in mind 'who' is using and benefiting from the renewable energy technology programme/project. sources. Mapping is a participatory rural appraisal exercise that provides information on the physical or socio economic characteristics of a community through maps drawn based upon their own perception. methods. A transect walk is a PRA exercise where the facilitator/evaluator takes an observational walk with a small group of local people through a transect (line that is defined by the group or according to interest of the observer) showing appropriate resources. Output Participatory Participatory Rural Appraisal (PRA) is a family of approaches. All beneficiaries who contribute towards the implementation of the project should be able to access and gain benefits from the programme or project without any discrimination during or after the process of intervention. An impact can be positive. Timeline is a PRA exercise that provides information on an individual or a group activity pattern during the course of a day. An indicator is an easily measurable criterion that provides information on trends or changes after the renewable energy programme or project intervention. Outcomes are often cumulative results of an output. Wealth ranking is a PRA exercise used to categorise communities or a group of people according to their economic status. Appraisal Social Equity/ approach Stakeholder Timeline Transect Walk In this toolkit. The outputs are the direct tangible measurable results or products delivered to the intended beneficiaries of an energy intervention.GLOSSARY Evaluation Impact Indicator Mapping Monitoring Outcome An evaluation is the systematic collection of information about activities. Outcomes are specific changes in attitude and behaviour. season or years. Monitoring is the systematic process of observation and collection of information at the individual household or community level to ascertain progress or impact of a project or programme. Impact is the long-term effect on the stakeholders. characteristics and outcomes of the energy programme or project that may lead to inform and improve the effectiveness of the programme. A stakeholder is the person or organisation that has an influence or interest in the project or programme.
ACRONYMS FGD M&E MDG NGO O&M PM&E PRA RET SWOT Focus Group Discussion Monitoring and Evaluation Millennium Development Goals Non Governmental Organisations Operation and Maintenance Participatory Monitoring and Evaluation Participatory Rural Appraisal Renewable Energy Technology Strengths Weaknesses Opportunities Threats ii .
The toolkit addresses. It can help field technicians and social mobilisers in the field to competently 1 . and the usage of participatory tools and methods. many renewable energy programmes are implemented with an emphasis on the technical and financial delivery mechanisms. However. Renewable energy technologies provide reliable and affordable energy supplies to millions of people in developing countries. Also. The toolkit provides a clarification of participation in monitoring and evaluation processes. This information can feed back to governments and donors to improve the effectiveness of their future programmes.INTRODUCTION Why do we need a toolkit? SECTION 1 use of the renewable energy systems. The approach mentioned in the toolkit intends to help practitioners and donors alike to forge strong 'partnerships' with users rather than let them remain only as 'recipients'. This toolkit provides tools to bridge the gap between programme developers and users or stakeholders. In the process. The content of the toolkit This toolkit describes how to listen to users. as many renewable energy technologies are maintained and managed by the users or communities. The common M&E approach used by practitioners of renewable energy programmes is to conduct conventional rapid assessments or an evaluation at the end of the programme. This neglect results in minimising the positive impact of a programme or project. in particular. particularly after the withdrawal of institutional support. The content of the toolkit is not technology specific. Timely monitoring and evaluation (M&E) is one way to enable organisations and communities who install and implement energy technology systems and programmes to check the efficiency and effectiveness of their work. take account of their opinions and make them active players in the installation and delivery of renewable energy systems. The participatory approach is qualitative. there is a neglect of the users’ perspective and actual needs. so this toolkit does not cover quantitative methodologies that may also be useful as part of the M&E process. there is a growing recognition that the effective involvement of people or users is required to gain positive impact through a sustained maintenance and Who are the potential users of this toolkit? The toolkit is designed to be used by both government and nongovernmental agencies and communities involved in the implementation of energy projects. how social equity can be included in the M&E process. design of an M&E system.
rather than face it all in the evaluation phase. Box 1: Monitoring technical components of the project/ programme It is necessary to monitor and evaluate both the technical and socio-economic aspects of a project intervention simultaneously. a list of actions and plan can be made for the local operators or household owners to generate data on the technical components of project installation/s. Renewable energy projects considered in this toolkit The focus of this toolkit is on the following types of renewable energy technologies: 2 . Isolated off grid systems (micro hydro. some of the tools can be combined with quantitative methods. Other renewable energy technology options/systems based on biomass (biogas. To use a participatory approach in the technical M&E process. wind and solar for water pumping. Alternatively.monitor the project or programme activities and build up a wide range of data. improved cook stoves). Group discussions and stakeholder analysis can be conducted with the owners. Individuals or organisations can pick ideas from this toolkit to plan an M&E process tailored to their own individual countries and regions. such as the survey. Different users have different interests that require different information from the M&E process. technical operators or management committees. solar). A thorough use of the participatory methodology in an M&E system will enable the practitioners and users alike to review and overcome any hurdles during the implementation. wind. to enable the triangulation of data or information. Technical M&E needs specific technology related indicators and technical experts.
and that the benefits are not reaching only one section of community and not the other. The following three sections deal with methods. It is an approach to find out whether justice is done to all groups of people. ethnicity. Social equity in essence looks at the progress or impacts of the renewable energy programme intervention based on race. gender or economic group differences. Using this approach will confirm that the project impact focuses on different groups. Section 1 and 2 are introductory and section 3 focuses on understanding the participatory and social equity approach to monitoring and evaluation.Approach used in the toolkit The toolkit uses a participatory approach to M&E. especially the poor and vulnerable members of the community who may otherwise be left out if the programme or projects are technical or financial in nature. The combination of using participatory methods and social equity will provide an impetus to gain in-depth understanding of the social relations in the communities and help programme implementers to focus on where the benefits should go or are going. indicators and assessment of results. 3 . It also focuses on the issue of social equity and its integration in the monitoring and evaluation process.
It is extremely important that professionals trained in renewable energy issues conduct indepth baseline studies. To carefully select who will participate and benefit. and the areas of improvement. Just as a good chef carefully puts together a variety of ingredients for different courses of a meal. The implementing or donor agency should 4 . Each phase must also link in to the next and cumulatively produce a range of outcomes/impacts specific to the RET project or programme. Monitoring is different from evaluation. a good project planner has to carefully plan activities for the different phases of the project or programme. in fact. Each phase of a programme will present an opportunity to fulfil different needs and expectations. Thorough baseline information will provide the best indicators to measure progress and change. monitored and evaluated. Before we go on to indicators. what the needs and fulfilment of various stakeholders are. It is also a process for critical reflection. Monitoring and evaluation is often considered a 'side dish' and is not placed in the main menu in the planning of renewable energy programmes that are technology driven.SECTION 2 MONITORING & EVALUATION Why is monitoring and evaluation important? Before defining monitoring and evaluation. Monitoring Project planners and implementing agencies should differentiate monitoring and evaluation into two distinct components in the planning process. let us look briefly at the meaning and purpose of monitoring and evaluation. or when the different activities will take place. how the project will be carried out. are all essential 'ingredients' needed for a project to be carefully planned. can be referred as an 'on going' evaluation. one does not often realise that these same ingredients should in fact come from the core of the M&E process. Monitoring should be frequently and regularly conducted and. However. let us briefly look at the importance of the monitoring and evaluation process. the success or failure of some aspects of intervention. These ingredients are also the 'indicators' that constitute the monitoring and evaluation of each renewable energy project. The information generated from monitoring activities will provide valuable clues on the occurrence of problems. Monitoring is a systematic process of observation and collection of information by implementing agencies at the individual household or community level to ascertain progress or impact of a project or programme.
conducted ideally a year or two after the project construction. The most important and rigorous process of evaluation would be the second 'impact assessment' that should be conducted a few years after the project implementation. After an evaluation. It is better to conduct two evaluations during a programme or project implementation. Both the monitoring and evaluation need to include the main stakeholders. Monitoring is also particularly important if there is no specific baseline information or assessment.know through the monitoring process if the aims and objectives of the project are being met and changes made accordingly. I mp leme nta tio n/ A consistent set of M on itor in g monitoring data can lead to an effective summarisation of the progress made. Evaluation is often not a continuing process. it is often difficult to conduct a P roject complete evaluation. The first one could be a mid-term evaluation. characteristics and outcomes of the energy programme/project that may lead to inform and improve the effectiveness of the programme. An evaluation is the systematic collection of information about activities. Make sure that both the processes are adequately budgeted. the project implementers may decide to take the project further. replicate efforts elsewhere or stop the project. particularly the users of the technology and the implementing agencies. An evaluation builds upon the monitoring P la n ni ng / Se t data and the two can i nd ica tors be seen as mutually dependent. Without a proper monitoring system. La te Ev a l ua tio n/ Impact Assessment M id -t e rm t Ev a lua t ion Evaluation An evaluation focuses on the effectiveness and relevance of the renewable energy project or programme to the communities or 5 . A regular monitoring process will enable an evaluation to be conducted with accuracy and in a relatively short time. FIGURE 1: The Project Cycle BASELINE C omp let ion /Ex it R ev iew /P la n fo r n ext phase household. to assess the possible impact. Budgets for M&E processes are not integrated properly in project planning.
before and within project/ construction period Inputs and outputs of resources: financial. management. lesson learning. These trends help in understanding and changing the strategies that may lead to an increase in positive impacts. Inputs Resources What you put in: Outputs Delivery Why should M&E conducted regularly? be A well-organised regularly implemented monitoring process will provide a trend analysis on different impacts resulting from the implementation of the renewable energy project or programme. It should include planned actions such as field visits. It can be both positive and negative. communities. technical. The basic M&E process should include resources and time as shown in Table 2. feedback process Evaluation Usually towards the mid or end phase of project/programme Impact and influence. external evaluator(s) or donors/financiers Effective lesson learning. environmental and institutional Implementers Stakeholders (project staff and communities) Result Increase in efficiency and improved work plans. biannual or annual period. both short term and long term. A timescale needs to be drawn up for each intervention and the expected output for each quarter. social. data derived acts as a basis for evaluation Designing the M&E process All programme or project managers must design a mechanism for the monitoring and evaluation process to be successful.TABLE 1: The main components of monitoring and evaluation Monitoring Process Content Frequent. having benefits and shortcomings. project impacts: financial. Remember that monitoring and evaluation can show both negative and positive outcomes TABLE 2: Inputs and outputs from the M&E process Outcomes Changes Impacts (+/-) Effects Difference between project outcomes and the outcomes without intervention: Direct or indirect. stakeholder meetings and a systematic reporting plan. exit strategies Project staff. Results of what Collection of you do: results: Goods & Services Practices/ behaviours Time Financial Technical Human 6 .
implementation and exit strategies. Data needs to be derived using the same units of analysis. inconsistent data. You need to follow this rule even for qualitative exercises. Box 2: Basic ground rules for M&E Completeness: Create goals and objectives. Command: Establish basic command and knowledge of the geographical area. It provides a platform for users and project implementers to deal with the negative or unanticipated events and strengthen the positive aspects. from beginning to end of the programme or project. stakeholders and the aspects you need to monitor and evaluate. 7 . Consistency: Consistency and more consistency: Be consistent in your approach at all stages of M&E.and effects. particularly beneficiaries. Commitment: Commit to the process by setting personnel and funds aside for M&E in your planning. set time plans. It is probably the most important ground rule. set indicators. “Less”. and preferably with the same stakeholders. is better than “more”. and interact regularly with stakeholders. consistent data.
It gives importance to the perspective of the 'user' and not just the 'provider'. Participation does not entail the sole involvement of users in constructing a hydro project or installing a home solar system. Alternatively. but it is used in Sri Lanka where communities who want to install a micro hydro scheme in their villages identify first the water resource. Communities identify resources and conduct initial project planning and design. then design the distribution based on their knowledge about the village geography and locations of houses. It is not a 'one time activity' but a continuous one. It means bringing the knowledge and real needs of people in to planning the intervention of specific projects or programmes. 2. particularly the beneficiaries during and after the process of intervention. Project management committees formed by local community members' who organise themselves formally and regulate the functioning of the system. take a pro-active role. the product of the participation process should lead to the empowerment of all stakeholders.SECTION 3 PARTICIPATORY MONITORING & EVALUATION Understanding participation in development & renewable energy intervention Participation is not just a fancy word in development but if used well can prove to be a powerful tool. Share of cost and maintenance by the households or community members involved in the installation of the renewable energy system/s. all the local community members. 3. A participatory approach to monitoring and evaluation A Participatory Monitoring and Evaluation (PM&E) approach involves 8 . in more advanced projects. 4. This is a rare practice. both men and women. Full or partial labour contribution by the users or beneficiaries during the installation of the renewable energy systems. The following are the common processes of participation found in the implementation of renewable energy projects: 1. measure water flow and head. Most importantly.
the stakeholders expressing their views and knowledge to support programme implementation. The purpose of the PM&E process can be two fold. First, to involve people in project planning and implementation, rather than just be 'recipients' of the technology. Second, assess the level of outcomes and impacts of the project or programme in association with users and stakeholders. Using a participatory approach, the communities and project implementers should be able to continuously improve their management styles, upgrade facilities and provide feedbacks to each other or to other stakeholders such as donors or manufacturers. Therefore, a participatory M&E becomes successful if the people who are involved in the energy project or programme own the process. Participatory processes can include various types and degrees of involvement, control over, and decision making during the energy intervention process. It can lead not only to an effective implementation of an energy project but to a degree of empowerment and social transformation.
Five general functions of PM&E Project management and planning Impact assessment Organisational strengthening or institutional learning Understanding and negotiating stakeholder perspectives Public accountability Source: Estrella and Gaventa (1998) process. In the second, institutions that are implementing renewable energy systems in association with communities or entrepreneurs use participatory methods extensively to involve communities in the various stages of PM&E. The first approach is an intensive process that may take much effort, dedication and learning from the users and stakeholders. The second approach is widely used and can be subjective. For example, many practitioners consider a participatory approach to be 'conducting a PRA assessment'. However, a PRA is only one of the various methods to gain information. This toolkit focuses on the second approach. It will include how monitoring and evaluation can be participatory and consider the users' perspective. Although participatory methods are purely qualitative, the M&E process does need indicators to measure progress. The indicators put
Basic elements of Participatory Monitoring and Evaluation
A participatory approach is not a 'single' approach. There are two ways to take it further. In the first, the owners and users need to take ownership of the process by developing their own actions and create a favourable decision making
forward in the next sections are provided as a guide and are not a comprehensive final list. You will need to improvise according to the technological specifications, communities and geographical area where you will implement projects. The PM&E should following phases: include the
Remember that a participatory monitoring and evaluation: Is only a tool and not a problem solver. It depends on how you use the tool to improve your project or programme implementation. Is a lengthy process of interactions, understanding and positive action from all the stakeholders involved. It is not a formula for success but can be if managed effectively.
Decide on the participants and find out their motivation. Establish clear aims and objectives of the PM&E with stakeholders. Develop clear indicators mostly with the communities/ stakeholders. Gather information based on indicators using participatory methods. Analyse results with the communities. Take joint action. Make clear assessment of who will take these actions further. The methodology should also: Analyse results in terms of impacts and achievements of the renewable energy implementation. Assess the strategy of the renewable energy project or programme, its effectiveness and implications for future action.
TABLE 3: Differentiation between traditional & M&E approaches
- Information derived from project beneficiaries using quick survey methods. Activities imposed by project developers. - Project monitoring and evaluation cycle based on project or donor driven indicators. - Progress and impact measured to reach project goals and objectives. - Participation may not involve all groups and can be led by powerful leaders.
- First empowers communities by providing information to assess their own needs and capabilities. Activities planned after joint decision or discussions. - Programme people act as facilitators. Indicators derived from discussions. - Progress and impact measured to reach users' expectations and improvement of self-management. - Participation of diverse groups including women a priority.
A participatory evaluation needs to involve stakeholders, especially the beneficiaries in the evaluation process. It is also important that the project or programme evaluators play the role of a facilitator.
Using an equity oriented participatory approach
This toolkit differs from other manuals and toolkits in that it uses a social equity oriented approach. If your goal is to achieve high positive impact on poverty
reduction, the inclusion of social equity as a central concept within the participatory process is necessary. Renewable energy projects need to focus on 'people' because if the users do not understand the implications of its usage, it might create the loss of financial investments and failure of the technology intervention. Often, technicians who perform feasibility assessments can get into the trap that 'communities' are all homogeneous. However, there are many variations that are specific to country, regional or local areas.
Box 3: Principles of Participatory Monitoring and Evaluation
Participation All stakeholders participate in the M&E processes. Under an equity-oriented approach, participation should include all groups, men and women without any discrimination. All stakeholders work to develop indicators, clarify expectations and priorities, and build ownership of outcomes. All stakeholders need to inform each other, particularly if there are problems or issues. It will encourage an increase in transparency and accountability. All stakeholders, particularly the users, negotiate on the process, collection and analysis of data and the actions that will follow the evaluation. Stakeholders, including users, learn from the PM&E process to own the process and have an influence on improving actions. Flexibility is essential as the process entails constant lesson learning, negotiation and ongoing change.
Stakeholder involvement Information sharing Negotiation
Source: Modified from Blackburn et al, 2000
There is a common misinterpretation that using participatory processes will automatically generate 'equitable' benefits. The planning stage is crucial in any PM&E process. A participatory approach often mobilises the stronger members of the community. The inclusion of women in decision making. if you want the PM&E process to be successful. In a participatory approach. for the process to be owned by the users. The interests of the richer powerful sections of the beneficiaries may dominate meetings. all personnel involved in the project or programme needs to play an active role. All beneficiaries who contribute towards the implementation of the project should be able to access and gain benefits from the project without any discrimination during and after the process of intervention. Therefore. is an important indicator of using a social equity approach. The planning process should include discussion between stakeholders and programme planners on setting possible impact indicators as well as a 12 . In an equity- oriented approach.Unlike other traditional processes. you have to take into consideration time and careful planning. participation of all members of the community or a wider range of stakeholders needs encouragement. preferably with the communities. for example. Planning and designing the PM&E A participatory process can lead to better negotiation processes between users and suppliers. Social equity needs to be considered in renewable energy projects because most experiences show that projects often favour the richer sections of the community and remain within the domain of the men. a focus on 'social equity' means keeping in mind 'who' is using and benefiting from the renewable energy programme or project and how the intervention can influence results that provide benefits to those who need it.
Partners and local people need information about potential activities. It is also advisable that an organisation has a dedicated person for M&E activities. Deciding on content of PM&E: Project activities For a PM&E to be successful. It is extremely important to keep in mind that there could be a 'participation overload' for the community members. The results will allow the project or programme implementers to act upon any inequalities that arise between men and women or between rich and poor.timeline for different activities. In a normal M&E process. it is important that project implementers discuss M&E activities and plan with users at the beginning of the project. To include social equity in a monitoring system. All project or programme personnel and users/stakeholders need to be aware of their roles and responsibilities in the PM&E process. in participatory processes. Designing an M&E needs a careful analysis beforehand of what type of information is required from the process. Therefore. It will allow for a clear strategy and follow up of the problems and issues within the project. it is often difficult to determine the 'end' by itself. it is essential to generate disaggregated data from the project initiation phase. However. The whole idea of a PM&E system is to get local people completely involved in the process so that the intervening actors and agencies begin to phase out their involvement by the end of the project. the 'end' is made clear to 'derive the means'. plan and take consent from users/ communities or beneficiaries of the programme. There is a need to create social equity indicators to effectively monitor progress and benefits from the renewable energy intervention for all groups. 13 .
make small slips of paper with the attributes as shown above and give them to small groups of 6-7 persons. Then. technology assessments Land and water ownership related to implementation of the renewable energy schemes Willingness to provide labour MAINTENANCE Technical know how Willingness to pay for basic services Repair parts Constraints and ability to overcome maintenance problems Distance to repair centre/markets Users participation in meetings Continuing role/benefits from energy services to women Possible conflicts and resolution mechanisms Time constraints Ensuring that the community is participating and providing result is an important aspect of the PM&E process. To do this in a participatory context. Similar exercises can be carried out in different focus groups using different topics. 14 . Women and the poor should be included in the participatory processes by first forming different groups depending on the cultural elements of the communities you are working in. Each group should sort out the top three or four attributes that they feel are most important to them. problems and lessons. MANAGEMENT Leadership Sense of responsibility Planning skills Ability to participate in constructive group discussions Capacity to decide on maintenance of renewable energy systems Ability to work well on Management Committees Women's role in decision making regarding meetings. they should be able to explain in the larger group the reasons for doing so. An example can be the access and use of energy resources by male and female members. there can be another session to take this forward and develop measures to improve on them or take action for the next step. obstacles. Remember that each exercise should be based on providing feedback to the M&E process by discussing indicators.Participatory Exercise 1 Planning attributes for M&E Points for discussion on management and maintenance issues during initial M&E planning with community members. Once the different groups have explained their choices.
Make sure that the team prepares beforehand all items for the participatory exercises. The team should also be prepared to conduct the various components of analysing energy issues and be able to provide answers to various queries that the community might have. Project managers should ideally be responsible for the activities if there is no specific person responsible for PM&E activities. Therefore. Therefore. Observation and documentation of the process is essential for information and analysis. PM&E activities for different stages of the intervention need to be budgeted at the initial design phase of the project or programme. skill and resources If the programme developers want to learn lessons and target for achievable positive impacts. Make sure that participants are aware that you will use a certain 'number of hours' for the exercise and set a convenient meeting point. participatory approaches may require the expertise of a trained person. A proper budget allows timely M&E activities to take place and assign clear responsibilities for individual project or programme members. Indicators should also be able to clearly link to the overall programme or project goals and users aspirations. personnel. Team members need an allocation of duties on 'who should conduct what'. Determining indicators for PM&E The most important step in an M&E process is determining the indicators for the process as well as the output/outcome. It is important that the resource person/s conducting the PM&E exercises be equally well versed in renewable energy interventions or the data that is required for the assessment. However. the key role of an indicator in the PM&E process is to provide specific pointers in gathering the type of data that will meet the objectives of the programme/donors and that of the beneficiaries and stakeholders. An indicator is an easily measurable criterion that provides information on trends or changes after the renewable energy programme or project intervention. For larger energy projects. Often. All team members need training in participatory approaches to conduct exercises with the community. Participatory exercises can consume a lot of time of the community members. the budget for M&E is nonexistent.Deciding on time. under funded or only meant for a final evaluation. 15 . it is preferable to train social mobilisers from the local area in using participatory tools so that they can interact and help the communities/users to monitor their progress in terms of implementation and in the final evaluation. Teams need to rehearse the exercises to give communities a sense that 'you know what you are doing'. one should avoid highly productive seasons (such as harvest or planting time if it is an agrarian community) or busy period during the day.
financial or institutional situation that you want to change after the renewable energy intervention. precise and consistent. number of participatory meetings and number of male and females participating in meetings. indicators are set or discussed with the users in the planning process. Outcome/Impact The outputs are the direct tangible measurable results or products delivered to the intended beneficiaries of an energy intervention. Following are some basic steps to develop indicators with different stakeholders: Identification of the social. Process and efficiency indicators will vary from one project or programme to another and needs discussion amongst stakeholders. This toolkit will focus on the first two indicators. and so on. These will be the impact indicators (see Section 4). Output vs. Such indicators will help you develop process indicators. While some outputs may be direct. Examples include training conducted by month/year using 40 hours of staff time. such as numbers 16 . An indicator is useful if it is measurable. indicators for efficiency targets can be helpful. If it is a whole programme you want to monitor and evaluate. Examples can be number of local people trained in the renewable energy technology intervention. Discuss how you can achieve the proposed impact. Both the users and the project implementers should be able to understand it in the same way.If you use the PM&E approach. They could be the electrical units or households benefiting from the project intervention. number of people employed. Discuss the difference in the situation that you want to change or achieve over the number of project or programme years. particularly the implementing organisation. This will help you develop baseline indicators (see Section 4). reduction in health expenditures and increase in participation of women. The lack of indicators will limit the monitoring and evaluation process. Examples include increase in income. Baseline indicators are the measures taken before the programme intervention and is used as a starting point to track changes and trends. electricity to be provided to users by month/time and so on.
An outcome may be immediate. do not monitor fuelwood savings when you are installing solar home or pico hydro systems that do not allow energy intensive cooking practices. or final. You will also need to monitor the processes and activities consistently. emission reductions and so on. The before/after situation should be Remember to include indicators specific to the technology intervention. direct or indirect.of households using the technology for lighting and cooking. In the evaluation phase. intermediate. It is not the definite end of a process. Examples may be employment generation and its further benefits. increase in knowledge. The monitoring process will determine the outcomes or changes of the technology implementation. The projects longterm effects are analysed in an impact analysis and need to be integrated into the evaluation assessment. or improve on the project activities if you have not fulfilled your project objectives. 17 . Outcome can sometimes be synonymously associated with impact. other outputs such as provision of information or increased participation of women and the poor may be difficult to measure. you will need disaggregated data (based on differences according to gender or socio-economic groups). but it should help in the provision of an exit strategy for project implementers. it is best to discuss these outputs further and to dig deeper in assessing whether these outputs themselves are of quality and what further impacts it has created in the community or outside. Outcomes are specific changes in attitude and behaviour. For example. positive or negative. especially if the communities are highly differentiated. An outcome/impact analysis ideally requires a long-term range of data collection. if successful. very clear in an impact analysis. intended or unintended. indicators to such outputs need careful examination with the help of stakeholders. In a monitoring process. Other external factors may also influence the process whether intended or not. and skills of stakeholders resulting from activities generated by the project or programme intervention. If you use the equity approach.
Changes and of RE services differences in the (electricity.Frequency of tariff analysis collection .Focus Group Discussions .Management . services heating) . access to energy cooking.Number of male/female members in committee .Household electricity/ energy interviews .Technicians Group (local) Discussions .Changes in management with incorporation of poor and women .Increase in number of training trained of quality services personnel provided by the technician .Access to of fuel savings (differences in health/schools share of where RE provide services household expense resulting from fuel switching) .Users of the energy technology . you have to set indicators with the users/stakeholders.Number of of the meetings held management .SWOT .Users provision .TABLE 4: Indicators in relation to output and impact/outcome Content Output Indicators (examples) Outcome indicators (examples) Methods1 Information sources .Economic benefits .Changes to health after fuel switching . If you were to use a participatory approach fully.Access/ usage .Focus . 1 See details in Section 4 18 .Interviews Committee or owner/s Effectiveness .Number of technical problems solved Note: These are only examples.Management committee .Resource mapping .Stakeholder analysis .Local technicians Energy Services .Stakeholders .Consistency rates of .Number of problems solved by committee .Focus .Timeliness of .Number of .Household interviews Effectiveness .Management management Group Committee meetings Discussions members/ owner/s .Frequency of rules and regulations usage .
Be aware that participatory methods: Can lead to repetitive predefined answers by respondents if participatory exercises are conducted more than once. This is especially important in countries that have high security risks. for both the respondent and the evaluator. there can be a continuous exchange of information without distraction. Will generate highly qualitative data that are site specific and cannot be generalised to a larger context or area. Interpretations may need to be triangulated with other conventional methods such as a survey or pre-existing data. make sure that you find out about this and adapt accordingly. especially with regard to interactions with communities. It is important that there are trained personnel. It is important to familiarise yourself with the activities of other organisations working in the same project or programme area. Before you visit. In this way. Be prepared to tackle antagonism and conflict. Discuss overall issues and get a feeling for what you would need to expect in your dialogue with the communities. How do you approach a community? Step 1: Planning Approaching any community needs planning. Are subjective. So. Contact the local leaders and government officials if you need permission for the visit. Figure 2 on page 20 shows a step by step process to using the participatory approach. It is important that there are at least two people carrying out the participatory or interview exercises. Be cautious when you first contact any community or user groups. Participatory exercises or research may have been carried out by others in the same area before you. You should remember that these methods help to organise and extract information from the people. learn about the community and make sure that you will get an audience by choosing the right time to conduct participatory exercises.SECTION 4 PARTICIPATORY METHODS A series of participatory techniques or methods can be used in a participatory assessment. They could provide important information. for participatory methods to be successful. Can only be a 'tool' and will not bring you instant results. 19 .
.Triangulate results . discussion of indicators 5.Conduct stakeholder analysis .Formulate an exit strategy with users/ stakeholders 20 . contact people .Review literature. If no baseline exists. Focus Group Discussions Problem ranking.Prepare materials and plans for participatory exercises 1.Formulate action plan incorporating changes with users/stakeholders .Record all the data . especially the indicators. Mapping.Review with users/stakeholders .Review and analyse results of impact and strategies . During M&E. Wealth ranking with different groups 3. familiarisation walk 2. Conduct Personal interviews (can draw personal time lines) 6.FIGURE 2: Step by step process to use the participatory approach in the M&E process Preparation Step 1 . Seasonal Mappings and Time lines 4. start similar process and spend more time drawing up and reviewing the indicators. review the above.Familiarisation visits .Prepare initial reports Field work using participatory methods Step 2 Reporting Step 3 Action plan or exit strategy Step 4 Monitoring .Write reports/ Implement changes Evaluation . Transect Walk Initial rough mapping exercise. Call group meetings: Discuss plans and actions ________________________________________ 7. technology interest / issues ranking.
Get their initial ideas and set a time for meeting that is comfortable for all members. you will find a possible 21 . In the participatory exercise tables 2 and 3. An excellent example is to take demonstration toolkits of the technology (for example solar or low wattage light bulbs). A quick walk through the area will allow you to gain an overall understanding of the local infrastructure. large posters. Step 3: Hold meetings to provide information Depending on the different stages of the project intervention. introduce the technology. to participate. explain the monitoring and evaluation processes to the users/stakeholders and find out how and when they can participate. and link it to their own needs. The communities/users can discuss practical needs and their own roles in the installation of the technology and the M&E process after deriving information from the demonstrative materials. one of the icebreakers is to take some demonstrative information materials with you. and then relate it to the energy assessment. men as well as women. If you are approaching a community for the first time. Provide appropriate information on efficient use of energy to avoid this situation! When you hold participatory meetings with the community. the communities will gain knowledge about the technology. resources and the community members. You can start the meetings by mentioning some of your initial impressions.Step 2: Assess local resources Look to leaders or others who will potentially help you in your assessments or activities. videos demonstrating the technology (can show with a solar powered system) and leaflets. With direct and visible information.
some of the following can be the discussion points and subsequently the data for monitoring the input of different forms of renewable energy use. You can also assess the capabilities and the ways of working in a community by discussing the management of other resources such as water for irrigation or drinking purposes.list of discussion points on energy access. 22 . This will be your baseline figure. they can conduct similar exercises within their own meetings. Step 4: Record all information for further planning Once you have information of energy access and usage. you will still have to go through the same process. Use the same data again the next time you hold a meeting during the monitoring process. Crosscheck any inconsistencies while you are still in the field. Once the community or users are comfortable with the information. If the community management find the process helpful. you can make an action plan with them to determine the various stages of the M&E process. The project team members should discuss any conflict issues or potential ones amongst members of the community at the initial project stage. usage and needs. Once you have set up the project. make it a point to record all the information. If you are approaching a community midway or at the final stages of the project. Remember to keep notes of all the meetings and participatory exercises to use in the next phase or meetings.
rice. others) Usage in bundles/month (preferably in weight categories) Distance for collection/responsibility Constraints over the past 2 years Produced by households or supplied Average number of livestock per household Usage of the charcoal/dung (preferably in weight categories where appropriate) Costs of the livestock/charcoal if bought or sold Constraints. barley. s up p ly a nd d emand i n t he g p area/household b efore p roj ect i nterv ention j v TYPE OF FUELS Kerosene or Paraffin Diesel LPG Coal Candles Batteries DISCUSSION POINTS Fossil fuels Nearest available supply centre Services of supply: regular or irregular Cost of the commodity Approximate usage (in units/month) Constraints (if any) Type of appliances using batteries Usage (in numbers and hours) and costs per month Nearest available centre Fuel Wood Renewable Energy Types of wood Usage in bundles/month (preferably in weight categories) Distance for collection Rate of fuel wood if bought per bundle/weight Constraints Responsibility of collection Seasons for collection Difficulties over the past 2 years Types (maize. millet. if any Crop residue Animal dung/charcoal 23 .Particip atory E xercise 2 p Possible p oint t o d iscuss r eg arding g g energ y a ccess.
communal battery charging. seasonal flow variations) Potential usage and benefits/constraints Maintenance and repair Management Distance from nearest grid Household needs for electricity Cost and willingness to pay Constraints and possible benefits Grid services Usage for households Usage for productive enterprises Continuing affordability Constraints and benefits 24 . ownership. if any Days with sufficient wind (seasonal calendars) Potential usage and benefits/constraints Number of livestock Availability of animal waste and water Cost of the commodity/ willingness to pay Nearest available supply centre Constraints and possible benefits Availability of wood Usage Affordability Constraints and foreseen benefits DISCUSSION POINTS IMPORTANT FOR M&E Approximate usage (in units/month) Running costs Maintenance and repair Constraints or problems Benefits Wind Approximate usage (in units/month) Maintenance and repair Constraints of benefits Usage of the biogas (users/time) Cost of the installation Availability of animal waste and water Availability of maintenance and repairs Constraints and benefits Usage (type/weight) Fuel wood savings Constraints and benefits (such as smoke reduction) Maintenance and repair facilities Usage (numbers and hours) and costs per month Management of the plant Maintenance and repair Constraints and benefits Biogas Improved cook stoves Hydro Water resources (distance.Particip atory E xercise 3 p Possible p oints t o d iscuss w ith r eg ard t o g sp ecific t echnolog y f or b aseline a nd M &E p g TECHNOLOGY Solar DISCUSSION POINTS FOR BASELINE Sunlight hours (seasonal charts) Nearest available supply centre for energy/technology Cost of the energy services: Willingness to pay Potential usage and benefits: households. health centres Constraints. ownerships.
regional or local). A stakeholder is also the one affected positively or negatively by project activities. installers and manufacturers. energy utilities. A stakeholder meeting provides a good opportunity for different 'voices' to be heard. during and after project implementation. political parties or any such person or institution that is involved in the project. When do you conduct stakeholder analysis? a Who is a stakeholder? A stakeholder is the person or organisation with an influence or interest in the project or programme. Box 4: Types of stakeholders Primary Secondary External Individuals or groups affected directly by the project intervention Actors involved as intermediaries in the project such as members of NGO’s. utilities. Stakeholders can be members of organisations involved in the project such as donors. it will help determine the interest and influence of the main stakeholders in the programme. Donors or governments (if they are not the implementers) may fall into these categories 25 . Remember that each stakeholder has a different role and function in the renewable energy intervention. If different stakeholders are involved in a programme. influence refers to the way in which stakeholders can take control or have influence in decision-making processes. Representatives from the communities. government (national. and to discuss the issues that arise before. NGOs. project are all different types of stakeholders. While interest refers to the stakeholders' particular concerns and stake in the project itself. One does not have to bring all stakeholders together for a stakeholder analysis.Stakeholder meeting and analysis Participation of stakeholders is the key to the success of any project or programme. installers etc Actors not formally involved in a project but whose actions and decisions may create an impact on the decisions and activities of the project or programme. distributors. male and female. poor and rich or leaders involved in the A stakeholder analysis can be conducted at all stages of the project or programme (see figure 3). In the planning/baseline phase. it will be useful to monitor their actions intermittently to sum up an effective lesson learning evaluation of the project or programme.
control over resources and relationships Setting and achieving indicators set for programme/project Expectations and needs from the energy project Involvement/feedback in different stages of the M&E process Action Plans (inputs. Issues relating to equity and gender needs should be discussed specifically in a stakeholder meeting. This will ensure institutional participation and support for the project. Stakeholder workshops should ideally be a one-day session and need to be prepared well for it to be successful.FIGURE 3: Stakeholders involvement at different stages of project or programme Stakeholders’ Interest/Influence B ASELINE M ONITORING E VALUATION Action Follow-up u Result Stakeholder consultation process The stakeholder analysis needs an expert facilitator and other helpers to help him or her with the process of taking notes and reporting. It is important that there is a presence of female members of the community as well as representatives from different clans or ethnic groups and poorer members. community members can provide feedback on the institutional support they received. The session can also help in sensitising them towards these issues. This is the main reason why community members need to be participants in the Box 5: Points to discuss in a stakeholder meeting/analysis Strengths. Cards and boards for participatory exercises are important in stakeholder discussion for members to come up with different ideas and explanations. Weaknesses. resources) 26 . In a stakeholder meeting. Opportunities and Threats (SWOT) for each stakeholder Find out the varying interest and potential impact of the project on their interests Find out the varying influences such as power. participation.
Alternatively. translators can be sought if there are a few members who do not understand each other. communities including men and women. In the appraisal process. stakeholder meeting. A SWOT analysis can help in two ways. you have to analyse all the information and negotiate. manufacturers and so on) and what kind of opportunities and threats exist for each group of people. Some of the common tools used within a PRA are games. the knowledge and views of local people are provided high priority. to plan themselves what action to take. It is also often used as a quick. PRAs are semi structured and are best carried out by an interdisciplinary team. Strengths. maps.If different stakeholders hold different views and interests. One of the main objectives of PRA is that users may learn from the process and carry out similar exercises on their own. cheap and flexible way of gathering information. and behaviours that enable people to express and analyse the realities of their lives and conditions. calendars and small group interviews. and threats (SWOT) analysis A SWOT analysis is an effective and simple planning. and to monitor and evaluate the results (Chambers and Blackburn 1996). a PRA is only a family of approaches. opportunities. weaknesses. First. methods. diagrams. this analysis helps in identifying what the strengths and weaknesses of the project are for different stakeholders (may include project partners. It is very important that participatory stakeholder assessments be conducted in the local language or dialect. entrepreneurs. Participatory Rural Appraisals Practitioners often think that conducting participatory rural appraisals (PRA) are the means and ends of a participatory approach. 27 . However. monitoring and evaluation technique that addresses aspects of the strategic planning and implementation processes. Group discussions within such meetings need to be planned to facilitate the presence of appropriate members in each meeting. Community needs and interests should always be given first preference as they are the primary stakeholders.
W orksheet f or p SWOT a nalysis f or p lanning a g renewable e nerg y p rog ramme o r p roj ect g g j Strengths Finance available Willingness to provide labour by communities Opportunities Actions* Budget all activities well List responsibilities Make an action plan Actions* Weaknesses No finance No technical or management skills Threats Possibility of grid electrification through political pressures Conflicts over distribution of energy and use of resource Actions* Look for loans Create funds Assess possible training facilities for staff/users Actions* Find out details from Electrification Dept Active participation and consensus from all users/community Good local manufacturers Link them to communities Funding through government subsidies or other agencies Approach funding agencies with plan * Mention who will be responsible for the action Please note that you should discuss specific points with different stakeholders to find out their perceptions.large sized (A stand to put up the paper) and Marker pens of different colours Stones. 28 . seeds. etc. Similar ideas need regrouping into one section.Materials required for participatory exercises Paper. You can carry out an exercise through group discussions to think about the various issues. A facilitator is required to complete this exercise. Discuss the various points in detail and ask the participants to list out the most important ones (could be three to five). Individual cards can then be collated on a large sheet of paper or board. or different coloured paper squares. leaves. Particip atory E xercise 4 . pebbles. coloured beads. He/she should be aware that in a SWOT analysis. a threat could also be an opportunity. One approach is to use four different coloured sheets of paper cut into small squares with each participant given a few cards to write their own viewpoints.
However. different communities may have their own methods of categorising whether a household is poor or rich. charts. the first step before a wealth ranking should be the differentiation of social groups based on their ethnic or linguistic origin. In this process. allow the community members to list criteria such as income. The social group differentiation process should be carried out with the community. and land distribution for the person considered poor or rich and so on. For step 3. you can link it to discussions on indicators such as access and control over resources. and decision-making. Facilitators should have the materials (papers. 29 . The wealth ranking exercise should ideally be carried out as one of the initial participatory exercises. If you use an equity approach. To find this out is the main purpose of wealth ranking.TABLE 5: Seasonal calendar and time availability of different members of the community PRA Methods* Ranking: wealth ranking. livestock. pens) ready before the process. the most common being high. middle and low-income groups. you have to be cautious in taking this step because. individual charts Interviews: Focus Group discussions & in some cases. in some communities. individual * Discussed in detail in the text Baseline Monitoring Evaluation xx xx xx xx x xx x xx x xx x xx Wealth Ranking Wealth ranking is used to categorise communities according to their economic status. It is useful to determine the relations amongst social groups and importantly the assets and strategies needed to exit from poverty. proceed to make sure that there is sufficient if not an equal representation in the participatory meetings. Once you have identified the different social groups. One has to specify the reason for undertaking this task. Proceed to conduct the wealth ranking exercise. preference of problem ranking Mapping: groups or personal Trend analysis: seasonal calendars. it may provide reasons for hostility to break out if different social groups do not get on well with each other. After the completion of the wealth ranking exercise. women headed households or single parent households should also be included. However. Wealth ranking exercises may be difficult to conduct accurately in large villages or settlements. involvement in various community activities.
FGDs are extremely important in a participatory process because they provide channels to exchange information and ideas between the communities and the service providers or project implementers. You have to cross check data relating to particular indicators using different sources and techniques to formulate a reliable information system. as it is often a sensitive issue. 2. 7. different focus groups can be formed outside the community. Reconfirm and crosscheck the process. can effectively participate in the process. Write the name of the household head on small cards. Advantages and Disadvantages: The advantage is that illiterate people. seasonal variations. Document the results and process. and focus group discussions. 5. 4. grouping similar like-minded organisations such as renewable energy suppliers. Remember that participants can provide either exaggerated or underrated information. especially the poor. Participants can divide the set of initial cards into each wealth category (and according to different social groups). gender (women and men). and other such activities need detailed discussions according to the availability of the members of different groups. During the implementation of community projects. Participants of each focus group may be selected after discussion with community members or key informants. it is important to differentiate groups according to social categories (rich or poor). Create a household list of all users/ beneficiaries in conjunction with participants. 6. Count the piles and record.Process 1. A focus 30 . 3. The communities may also not be ready to categorise themselves according to economic categories. and so on. Use the ranking information as you perform and analyse individual time lines. Ask participants to list the types and characteristics of different wealth groups. Dialogue: Focus Group Discussions and Interviews Focus group discussions (FGDs) help in the identification of interests. access and usage of various resources from specific groups. technical installation. NGOs and donors. government representation of energy planning. Depending on the objective or different information sought. and so on. The moderator should facilitate and guide the discussions and not put in his or her opinion. Implication of results: Time planning with regard to participation in meetings. The disadvantage is that some members may dominate the process.
otherwise it will be difficult for each member to contribute. and the possible outputs you envisage. 3. Ask their opinions and if they have issues to discuss. In this way. 7. FGDs are useful to gain information on indicators that are related to energy access. 31 . 4. When you start the meeting.group may be formed to discuss particular issues or to find solutions to existing problems within these group categories. An FGD should not exceed 10 to 15 members. the reactions of different groups to the same issues can be determined and compared. it may be important to know which impacts can be/are derived from a particular technology by different social groups. Advantage: It is possible to attain indepth targeted information about attitudes and behaviours that may not become clear using other techniques.Identify the topic/issues you want to discuss in a focus group. they Process 1. If the participants are committed. Specific points should be prepared for discussion and if there is time. Create a checklist. Identify the target group who will be able to provide feedback on this topic. 6. arrange for some snacks and drinks for the participants to show appreciation for the time provided by them. Identify a common place to meet in discussion with community members. Record all the points made in the meeting for further discussions during M&E. duration for the meeting. Use charts and pictorial representations if required. 5. explain the process. Do not drag the meeting on for more than two hours unless people in the group want to discuss it further. Identify group members through discussions with leaders or contact persons in the communities. 8. Remember to consider men and women/rich and poor or have separate interest groups. Involve the people in the discussion. service and use of different groups and particularly the changes that occur before/during or after an energy project intervention. 2. members can be provided with some discussion points before the meeting. Detailed discussions can take place on the pros and cons of participation of various groups. For example. If possible.
focus group discussions help to bring the voices of all people to contribute and not just a few powerful individuals.often suggest solutions to problems. which would be acceptable to service providers and users. ethnicity. Focus groups can also be formed to bring two divided communities to solve one particular problem. Results: If group discussions show conflict in the community. 32 . Focus groups help to clarify differences especially when there are divisions in the societies in terms of gender. Importantly. these issues need to be solved before the project is implemented. economic groups and so on. Women and poor may be left behind in discussion if it is mixed group. Project implementers may stop partnerships or even mediate between groups and individuals if there are severe conflicts. Disadvantage: Prominent and powerful members may dominate the discussion.
Lessons: The focus group provided considerable depth to the teams understanding of how customers think about electricity consumption and appliances. because of a lack of information from the utility. each household has to deduce the electricity consumption of each appliance. In one exercise. which would use the most electricity'? Group discussions also led to the understanding that both males and females experimented differently about the energy consumption of each electrical appliance. a marked improvement was shown. the women had acquired valid knowledge through practical experience. individual ranking of appliances was not very accurate. but participants were not sure of the correct order of these. each member of the group was given a set of pictures of selected appliances (see below) and the group was asked 'If all these appliances are in use for half an hour. the others tried to save electricity. such as the iron. 2004 33 . After the increase in knowledge about the electricity consumption. In all. it was hard for participants to distinguish between the electricity use over a short and longer period. For example. electric heaters and water heaters. The group recognised a few of the high-energy consumption appliances. only one of the participants in the group used electricity inefficiently. Results: The group ranked the selected appliances from 1-10 according to their perceived energy consumption. It was also found that on their own. and make their own decisions on what to use and for how long on an ongoing basis. It was expected that the discussion would indicate the potential of users to reduce electricity consumption and engage in energy efficient practices. et al. In most cases. Source: Annecke. One of the group discussions was based on the users' awareness of energy efficiency. While the men were more knowledgeable on technical issues.CASE STUDY: Focus group discussions with poor in Argentina determining knowledge of users regarding energy efficiency Process: In a study to assess the usefulness and acceptability of an electricity prepayment programme in Argentina. but when the group discussed energy efficiency and re-ranked the appliances. and how. the electric shower and the electric heater. one woman turned off all the lights. the main lesson learnt was that the trial and error learning initiative could be assisted by appropriate input from the utility. Therefore. It was clear that the group had no common reference point. and started testing each appliance by checking on the prepayment meter screen for an individual consumption. the group members expressed that they were cautious about using appliances such as iron. W. the research team conducted different focus group discussions.
Monitoring and evaluation processes can investigate energy specific access and control as well as compare the old and new situation. 34 . After mapping. health clinics. 7. If it is a monitoring or evaluation exercise. schools. Select an accessible venue to carry out the mapping exercise. 6. Depending on the information you want or the information they want to tell you. Process When developing baseline information. water. roads. Have an idea of the features you want to map before the exercise. Clarify the idea further and discuss which particular features are important for the local community in order to install a renewable energy system. particularly in relation to gathering information on access and control over resources. 4. 3. water source. Such information can provide important indicators for the renewable energy intervention. Discussions can be based on the maintenance and management of these or other resources by the communities. and on men and women. Use different groups if the area covered is large. use the map or draw the different features of the technology (power house. Ask members to draw or use different materials (such as stones. solar installations. Select community members to assist in the mapping exercise and discuss the details before starting the exercise. 5. and so on. Use large sheets of paper and place them on the ground. mapping of location and usage of water taps and wells can lead to an idea about how locals use their resources. land. Avoid drawing large areas. resource or mobility mapping. If you have a map made before. Ask one person at a time to draw the project area. coloured beads) to show different features of the area such as houses. use the same map for further discussions and trace the changes. farm maps and other services.Mapping: social. such as forest and fuel wood. pebbles. transect walks can be conducted whereby team members walk with the local community members and 1. access to resources) and discuss accordingly. ask them how these features have been used in the project and what the impacts have been to different households (in the mapped out area). transect walks Mapping through transect walks and drawings by the communities can be effective. 2.
Maps can be drawn with either individuals or groups. 2003 35 .improve on the map already drawn. The transect walks were also used in all study areas to enhance or reconfirm information collected through other techniques. Personal maps will show individual perspectives and may draw attention to differentiating between gender and social groups. Source: IT Power. fodder and fuel wood collection areas. Socio-Economic Impact Assessment of Improved Watermills. Discussions can be carried out simultaneously. The study team conducted transect walks along the river valleys where the mills were located. participatory mapping provided an understanding of mill location and approximate distances to fields. Example 1: Participatory mapping of mill locations in India In a watermills study conducted in India. diesel mills. market towns and other significant landmarks.
Advantage/Disadvantage: While transect walks and mapping are good for gathering data on access and control in smaller areas. Results: Timelines are perfect for planning activities. Monitoring and evaluation exercises can draw data on where the frequent problems lie and the solutions that communities come up with. Results: Mapping can lead to decisions on the location of electric poles. community Individual timelines can Which season are they free? Season Activities help you understand the Men Women Group 1 Group 2 daily activity patterns or Crop financial flow of Jan-Mar x x x x planting households. Timelines of different household April-June Harvesting x x x x members of active age can be an important Charcoal July-Sept x x making determinant for planning users' inputs in projects or Collect Oct-Dec programmes. You can chart time lines with groups to determine a variety of information such as seasonal variations of their activities and different work processes. mills and biogas digesters. If you draw timelines according to different social or economic groups or men and women. Timelines can be drawn with individual persons (for TABLE 6: Example of a seasonal calendar and time one day during different availability of different members of one particular seasons) or in groups. timelines can be drawn to show changes in rainfall. Advantage: Timelines can be drawn individually or with members from a few households. In terms of gathering technical data. large geographical areas may be difficult to capture in the mapping exercise. solar panels. particularly in terms of access to settlements where the poor or disadvantaged beneficiaries may live. river flows. x x wood. They can help in the construction of projects if the beneficiaries are involved. It does not require the presence of all the community members. It can provide details on the spatial area to install the renewable energy technologies. rest 36 . wind and other such parameters that may be needed for technical assessments. powerhouses. Diagrams: Time seasonal calendars lines and Time lines and seasonal calendars are extremely important tools that can help draw a time plan for project implementation and monitoring. sunlight. it will provide you with the perfect basis for gaining results for an equitable outcome.
FIGURE 4: Timeline for seasonal activities First quarter Second quarter PREPARATION Process 1. Frustration is high of those who have been 'asked' the same questions many times. This toolkit will not detail the survey method because it is not a participatory method. Respondents often feel 'neglected' and 'want something back' from a surveyor. 5. Try to discuss the points in an environment where you are not interrupted by others. Third quarter CHARCOAL PLANTING HARVESTING COLLECTION FUELWOOD Personal interviews Personal interviews may need to be conducted to review the data provided during group exercises. The survey can then target for more information that is specific. it is 37 . Survey A survey can be used to gather quantitative data and is probably the best method to assess the impact of a renewable energy intervention at the micro level. Therefore. 3. Make a checklist of possible points or questions you want to discuss. 2. however. Select from different social/gender groups. 6. 4. be conducted using a participatory approach by informing the respondents about the processes involved and explaining the intentions of the data collection process. A survey can. A survey is best designed after you have conducted participatory exercises with the communities and received all the important information. Identify possible respondents for the interview. A survey might be needed to establish a good baseline data and particularly to look at the impact of the project or programme intervention. Personal interviews are often needed as a back up to large-scale participatory exercises. Inform the respondents and check if they are available for the time set aside. seasonal charts and gender issues can be further discussed in personal interviews. It is an extremely useful tool to gain an indepth understanding of a particular intervention in the lives of beneficiaries or stakeholders. Think about whether these points will provide answers to specific M&E indicators. Set a time and day for the interview to take place. Timelines.
It was also found that in larger groups the more vocal women tended to lead the discussion. when women said they were too busy the interview team moved on. Box 6: Feedback on using participatory approaches from the field: extract from a watermills study in India Despite the useful information provided by the village map and seasonal matrix during the first mission. This approach also seemed to be welcomed by the women. Such experiences are common. initial observations showed that this approach alone would not provide the necessary details about women's time spent on grinding grain and husking rice because these activities happen only a few times a month. The other benefit of an individual (door-to-door) approach was to avoid the tendency of group answers.advisable that you do not carry out the same assessments repeatedly. especially if you want to conduct in-depth monitoring or impact research. a door-to-door interview with 5-8 households in each village was felt to be a more effective way of collecting the desired information. Therefore. Socio-Economic Impact Assessment of Improved Watermills. Women were approached when in their homes and less likely to be disturbed in their work. it is important to use a combination of methods and approaches. As a result. 2003 38 . it was agreed that a preferred approach to gain an understanding of women's time use and livelihood issues would be to spend time with them in the villages over a few days. Source: IT Power. However. Although it was planned to shadow women at their work. making individual variation and opinion difficult to draw-out. Be sensitive to the needs of the respondents and ensure these needs are met. feedback from the women implied they do not wish to attend group meetings in which they could see no clear benefit and which disrupts their work. many saying they were glad for the opportunity to talk about their experiences. with the others agreeing passively or not participating at all. Designing a survey questionnaire needs to be targeted specifically to the goals and targets set by the community or the project developers. To this end. In order to elicit a range of views and to minimise disruption it was then decided that a door-to-door survey should be used for collecting information about household milling habits and their perceived impact of upgraded watermills.
surveys and background information would provide the ideal information for a strong M&E output. A triangulation of data derived from participatory methods. Basic indicators are required nonetheless to indicate progress and impact. as one of several methods for M&E. For example. Participatory methods can be used therefore.SECTION 5 ASSESSMENT INDICATORS This section will focus on defining specific M&E indicators to determine the impact of renewable energy projects or programmes. It is difficult to set accurate measurable impact indicators using participatory approaches because the indicators need to be derived from discussions and the results are qualitative. The general indicators in table 7 are important to provide not only an overall impression of the area and households' socio economic characteristics but also to determine the results in the analysis and reporting of the M&E process. Tables 7 and 8 show a set of indicators that you can use for discussion to generate information for the M&E component of your programme. Below are a set of indicators that can be used irrespective of the type of renewable energy intervention. it is important in the first step to set a basic overall structure of the project or programme area and its inhabitants. Accurate data on some of the indicators below may not be directly available using participatory methods but these indicators are designed to guide discussions with users and stakeholders. Data to fulfil the description of most measurable indicators are best derived from a survey. You have to design your approach in a way that allows you to discuss the indicators with the users or beneficiaries during group discussions or whilst undertaking a mapping exercise. quantity of renewable energy technology schemes installed Monitoring and evaluation will need to consider specifically the impacts that result from the project or programme intervention. stakeholders and programme implementers The specific project or programme area The types. Remember that indicators for an M&E process will depend on: The needs and priorities of users. household characteristics will enable practitioners to distinguish rich and poor households. size. 39 . In any monitoring and evaluation process.
Literacy rates in area Indicators .Leadership .Income access. and decision-making 40 .Types/ numbers of local organisations .Expenditure: according to commodities Social networks .Communication Number/percent of households served by/ access to: .Electricity .Number of household members according to sex.Distance from nearest electricity provision centre .Number/ types of houses owned /no.Percentage of beneficiaries' literate/illiterate General area and house-hold characteristics Basic infrastructure .Markets .Transport . scale .Different sources of water.Number of male/female/children in area . type.Types/ numbers of rooms .Local organisations .Expenditure rooms/ type of roofing.Type/quantity of assets (land.Nearest health centres or hospitals .Water .Distance from nearest markets .Nearest transport connections: roads (type).TABLE 7: Overall area and household characteristic indicators Variables Demographics .Distance from nearest communication facilities .Type of leadership .Number of male/females involved in different organisational capacities.Proportion of male/ female/children . education.Income and assets .Number/ percentage of households served/ not served by project/programme . flooring of house . of Household characteristics .Household member characteristics . age. Distance to source .Health . occupation .Proportion of households in region/area . vehicles . livestock) .Local management systems .
Type of appliances/usage from the renewable energy services in the institutions .Number of productive enterprises started .Type/quality of management of the enterprise . school.TABLE 8: Output and impact indicators for the project/programme General Energy access and usage for institutions Indicators . news.Awareness levels of users of tariffs/customer satisfaction Energy access and usage for productive enterprises Access to .Years of operation of each productive enterprise .Electricity required/used by the enterprise .Quality and reliability of energy services .Number/type of communication technologies after information/ installation of the renewable energy scheme communication .Number of households/percentage of population (male/female) benefiting from institutions served by the project or programme .Number and type of institutions (health.Number and types of productive enterprise/s started after the renewable energy project or programme . health and safety and others 41 .Number of households/families benefiting from the productive enterprise .Increase in access to information of: income generation activities.Investment costs and returns . small industries) served by the renewable energy project or programme .
Number of males .Number of . clan) and economic groups in the project area (see table 9). you should start to define definite indicators that would lead to monitor the access and usage of energy.Interviews Gender Family . Table 10 provides some indicators that you could start to discuss with the users and stakeholders. The overall aim of your project or programme should lead you to decide how to differentiate the groups. TABLE 9: Disaggregating groups for the social equity approach Types of Groups Economic . it is important that you carry out different participatory exercises with men and women to find out their roles.Income groups according to income & status expenditure/ or as indicated by users Ethnicity .If you use the social equity approach.Members of groups . if you want to determine the impact on gender.FGDs . this exercise will form the core for differentiating the various impacts according to gender.FGDs group) .FGDs .Official statistics . the first exercise you have to conduct in all your participatory exercises is to determine differentiated data with regard to different social (ethnic.Single group (depending on specific location) . and gaining an overall understanding of the project or programme area.Ranking .Number of memberships in other groups Social networks 42 . Remember that indicators should be chosen specific to the project or programme site/s.Surveys household (in each .Mixed group . responsibilities. social groups.Household surveys/ interviews . gender.Type/ number of others members in households .Survey and households in each interviews/ FGDs group . After disaggregating the specific data relating to different social and economic groups.Classify the community/ies Methods . If you want to follow this approach.Male/female Classification . For example.Wealth ranking / FGDs .Official statistics . nuclear. joint.Mappings . interests and influences amongst others specific to the renewable energy technology intervention.Extended. .Official statistics and females in the .Mapping . rich and poor and so on.Survey .
Number of households/percentage of population with access to the energy services (electricity/ biogas) .Decision making positions .Users perception of management .Savings of CO2 emission through fuel switching .CO2 reduction rate Other benefits .Number of trainings .Rate of deforestation .Levels of transparency .Repayment schedules for the renewable energy system .Increase in the quality of services Financial .Division of costs and benefits Ownership .Leisure .Levels of management skills generated .Levels of local maintenance skill creation. male/female) access/control to use.Control over the ownership.Biomass utilisation .Number of households/percentage of population with access/ usage of productive enterprises/ equipments and appliances .Number of households (groups.Changes in economic levels of users (income/expenditure) resulting from fuel switching or new employment opportunities.Equitable access and usage of electricity and other benefits Indicators .Increase in listening to radio/watching TV . number of trainings . connection and tariffs .Number of male/female in Management Committees . and management maintenance and management of the renewable energy technology and its benefits .Frequency of meetings .Access to maintenance services .Social networks 43 .Increase in prestige Environment .TABLE 10: Renewable energy output and impact indicators for households General Energy access and usage Variables .Increase in number of hours for relaxation .Number of new employment opportunities .Affordability of the renewable energy services .repair and maintenance costs .Cost/percentage of household income and expenditure spent on: . maintain and manage the energy .Increase in social activities .the installation.Distance to local service centres in area .costs of appliances .Participation in meetings. . increase in self esteem. bargaining power Maintenance .Number of households/percentage of population switching fuel from traditional to renewable energy services .Number of households/percentage of population using the energy services (in units/ months) .Forest area affected .Users access to maintenance operators/mechanics .Costs of maintenance .
Make sure that you discuss energy access and usage using the same units during baseline. Therefore. monitoring and evaluation. TABLE 11: Renewable energy output and impact indicators for households MDGs Eradicate poverty and hunger Common indicators to measure outputs and impacts .Number of households benefiting from projects according to income/gender .Number of poor households with increase in productive activities after the installation of the renewable energy project/programme . Different beneficiaries will have different perspectives and usages. Gathering information on energy expenditure is often very tricky. it is important that you differentiate between the various social groups. Be consistent.Be careful while gathering data from a participatory exercise. Below is a list of indicators that you can discuss with local users and beneficiaries.Differences in income and expenditure resulting from use of renewable energy . Renewable energy projects and the Millennium Development Goals Renewable energy schemes may contribute directly or indirectly to fulfilment of the MDGs. such information needs to be taken at an overall level but. it can be discussed further with the participants. If you want to use the equity approach. for definite outputs and impacts. This needs to be done immediately and. individual beneficiaries should be interviewed.Number of people employed by the project . if discrepancies arise. One way to do this is to get accurate information on quantity of the energy source and multiply it by the cost per unit of the energy usage. Take an example of energy source and usage to show the participants.Availability of finance for poor households to access energy from the projects 44 . The M&E process will be effective if you use the same indicators to carry out participatory exercises at different stages of the project cycle.
Increase in income for women from the renewable energy projects/programmes .Quality of participation in household decision-making by women .Number of new income generation activities resulting from renewable energy implementation .Reduction / replacement of CO2 emissions .TABLE 11: continued.Number of adult/ informal literacy classes in evening .Presence of doctors and nurses/ health attendants .Number and quality of lighting and energy access for schools .Improved access to energy resources for men & women .Availability of teachers .Number of women attending meetings/ training .Number of technology transfers from one country /region to another .Improvement of sanitation after electricity or renewable energy technologies/programmes introduced .Development of private sector such as manufacturing workshops .Ability to undertake new activities by women (and men) .Differences in time spent by women on different household activities .Savings of fuel wood/ smoke .Differences in the total daily workload of women/men .Access to medicines .Decrease in workload for women . MDGs Achieve universal primary education Promote gender equality and empowerment Common indicators to measure outputs and impacts .Access to medicines .Increase in the preservation of forest land .Lighting in hospitals and rural clinics .Use of refrigeration in rural clinics and hospitals .Usage of local renewable resources with less negative environmental impact .Increase of access to clean drinking/pumped water .Access to medicines ..Reduction of smoke / use of clean fuels or elimination of smoke Reduce child mortality Improve .Number of beneficiaries from the new opportunities Develop global partnership for development Note: These indicators are not exhaustive...direct or indirect Combat HIV/ malaria and other diseases Ensure environmental sustainability ..Changes in number of visitors to the health clinic if served by the renewable energy project .Electricity/ number of installations in slum areas ..Number of extra hours children spend on education at home .Presence of doctors and nurses/ health attendants because of better services such as electricity .Lighting in hospitals and rural clinics maternal health .Presence of doctors and nurses/ health attendants .Reduction of smoke / use of clean fuels or elimination of smoke .. The next table provides indicators for social input 45 .Number of women beneficiaries .
Female . some simple steps are provided to follow the assessment of the data generated from the various exercises. After you conduct a participatory exercise. the management committee can record ongoing performance. However. information derived from the participatory exercises form the most important source of M&E data. the differentiation should look like the one in the following figure: Step 1: Record all information Users and communities can generate and record some information for you. economic & gender groups Using the social equity approach means that you have to differentiate the groups according to social. household users can record specific household information and social mobilisers can monitor certain activities depending on the scale of the programme. Step 2: Differentiate social. You have to conduct these differentiation exercises early on in the process.Male . While these constitute one set of data. make sure to duplicate the results on to another sheet of paper as the ones worked upon during participatory exercises may use different materials such as stones and beads to show different resources or features. economic or gender characteristics before you start an analysis.Male . When you are assessing your result.Female 46 .Female . the operators or FIGURE 5: Differentiating socio-economic groups Social group (ethnic/clan) High income group Medium income group Low income group . For example. you need to analyse the information from the beginning of the project to devise an understanding of the trends and patterns that may evolve as you interpret the data. In this section.SECTION 6 ASSESSMENT OF RESULTS Analysing information from participatory approaches Participatory exercises can generate a wide range of data.Male .
planners can play around with the costs and input the labour or work earlier or later. Installers: March is the best season to install/ construct projects because it does not rain. Take action: You can further use the timeline feature or seasonal calendars to come up with a solution or plan for project or programme intervention. there may be other information generated from the discussions and exercises. The above data may come from stakeholder analysis and focus group discussions. All households could perhaps provide a few days of labour despite their work or an 'additional financial contribution' that the management can decide with them. project developers do not seriously consider users' perspectives. you are providing a structure for the analysis. energy usage and management capabilities. the communities need to provide 25 percent of construction costs in terms of labour. Example 2: Triangulating results from participatory exercises for provision of labour inputs by users Users: We are extremely busy planting crops in March and April. Financiers/NGOs: Financial dealings close by March. Expert facilitators will carry out this differentiation process while conducting the participatory exercise itself. If you do not. However. If you have specific themes. In addition. You can alternatively triangulate information to confirm the results from different sources such as observation and surveys. monitor and evaluate responses. set at the initial stages of project 47 . Step 4: Triangulate the results After you have completed recording and reviewing the data. Initial results need to be reviewed with users/stakeholders. If you have a wide variety of results generated from different exercises. Additionally. you will see that the answers to the same questions from different stakeholders are triangulated to form actions and plans. In the following example. Qualitative data may also be subjective. you will end up with a wide range of results. How do you solve such a problem when 'your' best time is the 'worst' time for the users? Often. The usefulness of participatory approach will show only when you know how to use the data effectively and carefully implement. It will enable you to cross-check on information and avoid major differences in results. start by listing key themes such as overall demographic information. social characteristics. one of the most useful exercises to undertake is to triangulate the different data sets. In this way. All community members meet to discuss the issue. it is best that you review it immediately when you are within easy reach of the location and people. You need to differentiate each theme and review them. review them with the stakeholders. All major activities need to be carried out before March.Step 3: Differentiate and review the main themes The main themes of the PM&E process may be determined before the participatory exercises. Therefore. intervention. Each participatory activity needs a separate analysis and completion at the end of each exercise. energy access.
but also providing another step towards empowering them. Reporting would also mean providing structure to the analytical results. omission of the indicators needs to be reviewed at the monitoring stage. it should be clear that the following have been achieved: Problems are identified by all stakeholders. The most common structure relates to the access and usage of energy (see table 12). you are not only informing them of their choices and progress.Step 5: Review initial results with stakeholders If the renewable energy intervention programme is a large one covering a wide geographical area and a large number of individual projects. In this way. Evaluate the outputs and impact that has been achieved. Step 6: Reporting While you report the findings. Similarly. especially the users. 48 . Determine a possible exit strategy for programme and project personnel. Any omission or review of the indicators needs to be reported consistently. Undertake a detailed examination of the success and lesson learning factors for similar interventions. After each monitoring process. Assess whether actions and responsibilities have been successfully coordinated and implemented by various stakeholders. A report is effective when all stakeholders assume responsibility and learn from the process. All monitoring reports should clearly recommend the plan and actions to improve on the themes that lack progress. Such a process will lead to transparent negotiations and lesson learning and is found to be an extremely effective foundation on which to base an exit strategy. Actions for the problems have been identified and discussed. Structure your report layering the progress of each monitoring exercise so that it will be useful for the mid-term or end evaluation. it is important that some of the initial results be reviewed again with stakeholders. the analysis needs to come up with a conclusion on the progress of the main themes. the evaluation process should: Identify whether the objectives originally planned are completed or not. and most importantly make effective decisions and action plans to move forward. Readjust any changes to the original objectives. Responsibility has been taken for the action. If there are indicators that do not show any importance to the assessment of a particular renewable energy technology (such as wood fuel savings for solar home systems or pico hydro).
but also the servicing and collecting of tariffs. transparency and the need for co-ordination of all stakeholders. there is a higher chance for the energy project or programme to be sustained. hrs/ day or month) Quantity Litres/nos /day or month Costs Currency/ day or month Expense Currency/ day or month ENERGY SOURCE Electricity-grid Electricity-RE LPG Kerosene Candles Paraffin Batteries Charcoal Fuelwood Twigs Dung Others 49 .Building ownership of participatory monitoring and evaluation Building ownership of the tools and process of a participatory approach to M&E is probably the toughest challenge because of the demand for accountability. In this case. Example . The participatory approach also builds ownership with different stakeholders as they develop their own skills and discuss amongst themselves various options to better implement and maintain a project. There needs to be ownership by the various stakeholders of the programme. However. they need to manage not only the maintenance of the technical system. If ownership is maintained by the stakeholders. It is in such cases that a participatory approach would prove to be a tool for reducing costs by allowing for immediate mitigation activities if a problem or conflict occurs. It is also an approach that needs to be built into programmes and the system of implementation. the approach is best suited if the same users were to own and maintain the TABLE 12 energy technology and its various systems. One must keep in mind that different stakeholders have different perceptions and expectations.Energy Access and usage data needed for baseline and M&E Access Usage Lighting/ In cooking/ distance radio Usage Time (no.
Monitoring and Evaluation in Rural Electrification Projects: A Demand. Chambers R. No. Tools for Community Participation. 10. (2000). UK Srinivasan L. Estrella M. Report prepared for DFID. and Gaventa R. and Gaventa J. The Power of Participation. Report On The Acceptability And Socio-Economic Impact Of Prepayment Meters in Merlo. (1996). OED Working Paper Series. Who counts reality? Participatory Monitoring and Evaluation: A literature review.. Endelli M. and Carpio C. (1998).Oriented Approach. France. Mainstreaming Participation in Development. IT Power (2003). IDS Policy Briefing Issue 7.REFERENCES Annecke W. and Blackburn J. UNDP/ The World Bank.. Blackburn J. Socio-Economic Impact of Improved Water Mills in Chamoli district. Buenos Aires. Northern India. World Bank. Chambers R. (2004). Washington D. For EdF. ESMAP (2003).C. Brighton: IDS. (1993). 50 . PROWESS/UNDP -World Bank Water and Sanitation Programme. IDS working paper 70.
IT Power has undertaken over 900 projects in more than 100 countries. for institutional and private clients. The United States and the European Commission. Since 1981. climate change policies and business consulting. Australia and USA. solar water heating and landfill gas utilisation. including wind. China. South Africa.co. Spain. Kenya. www. By providing opportunities for concerted collaboration among its partners. The Netherlands.org . Canada.reeep. The United Kingdom. Ireland. biomass. Italy. REEEP aims to accelerate and expand the global marketplace for renewable energy and energy efficiency technologies.itpower. www. Mongolia. The group specialises in renewable energy. mini and micro hydro.The IT Power Group works internationally.uk The Renewable Energy and Energy Efficiency Partnership (REEEP) is a global privatepublic partnership launched by the United Kingdom government along with other partners at the Johannesburg World Summit on Sustainable Development in August 2002. with a network of offices in the UK. photovoltaics. IT Power's projects involve a wide range of sustainable energy technologies that will help countries meet their Millennium Development Goals. The partnership is funded by a number of governments including: Austria. India.
Chineham.uk . The toolkit focuses on the participatory approach and describes how to take account of users’ opinions and make them active players in the installation and delivery of renewable energy systems. Grove House.co.Renewable energy technologies provide reliable and affordable energy supplies to millions of people in developing countries. Hampshire RG24 8AG. UK Tel: +44 1256 392700 Fax: +44 1256 392701 itpower@itpower. This publication is a toolkit designed to enable organisations and communities that install and implement energy technology systems and programmes to check the efficiency and effectiveness of their work. Lutyens Close. IT Power.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.