This action might not be possible to undo. Are you sure you want to continue?
Two groups have to be thanked for their help and guidance during the development of this book: the Impact Assessment Taskforce, and the Impact Advisory committee. Our dedicated taskforce is made up of representatives of Partner organisations from four continents and representatives of international strategic organisations. The Impact Assessment Committee is composed of academics and researchers with a range of expertise. All individuals volunteered their time and effort to ensure that this becomes a comprehensive and practical set of tools. We would not have been able to have done this work without this crucial help. Impact Assessment Committee Ray Boshara, New America Foundation Judith Bruce, Population Council Lisa Dacanay, Asian Center for Entrepreneurship Gregory Dees, Duke University John Elkington, SustainAbility Dean Karlan, Innovations for Poverty Action Lewis Mandell, University of Buffalo Lata Narayan, Meljol Sara Olsen SVT Consulting Michael Sherraden, Washington University in St. Louis Peter Scholten, Scholten & Franssen Fred Ssewamala, Columbia University Impact Assessment Task Force Rammani Acharya, JA Nepal Patricia Formadi, WADEP, Ghana Johanna Cloete, JA,Namibia Gilberto Mendez, CCF, USA Ingrid Jones, Partnere Perfemijet, Albania Zeina Khoury. Jordan River Foundation, Jordan
Copyright 2008. Aflatoun, Child Savings International. This work may be reproduced and redistributed, in whole or in part, with or without alteration and without prior written permission, solely by educational institutions for non-profit administrative or educational purposes providing all copies contain the following statement: Copyright 2008, Aflatoun Child Savings International. This work is reproduced and distributed with permission of Aflatoun, Child Savings International. No other use is permitted without express prior written permission of Aflatoun, Child Savings International. For permission contact: firstname.lastname@example.org
Introduction Chapter 1 -Aflatoun’s Quality Assurance and Impact Assessment Strategy (AQIS) AQIS tools in line with the Theory of Change Principles of AQIS Chapter 2. Quality Assurance (QA) Partnership Process Multiple Partnerships in a Country Participatory Evaluation of Secretariat Chapter 3. Impact Assessment: Surveys and Partner Database Survey Overview and Methodology Analysis Steps to Collecting Surveys When to Conduct the Surveys The Partner Database Chapter 4. Impact Assessment: Other Tools Collecting Stories and Audiovisual Material Appendix 1- Learning and facilitation visit Appendix 2 - Database manual Appendix 3 - Suggestions for Training of Surveyors Appendix 4 - Survey planning guide Complementary Research Projects Data Collection Summary Quality Assurance Impact Assessment Appendices
Welcome to the Evaluation Manual, your reference manual on Aflatoun’s Quality Assurance and Impact Assessment Strategy (AQIS). As the Aflatoun network is constantly expanding, partners have expressed a need for safeguarding the quality of the programme and understanding its impact. This is crucial to ensure efficient and effective delivery of the programme. Thus, the tools described in this manual aim to build credibility for the concept of Child Social and Financial Education by showing that the Aflatoun Network can deliver a high quality programme in a cost effective manner. We also want to be able to show the level of impact the programme has on the children who are part of the programme across the world. The manual outlines two core key processes that are practiced within the Aflatoun Network: - The Aflatoun Partnership Process - The Aflatoun Partner and Field Surveys Aflatoun’s Quality assurance and Impact assessment Strategy (from now on ‘AQIS’) combines a range of innovative and strategic approaches. It has been developed by the Aflatoun Secretariat working actively together with an advisory committee of experts and two partner taskforces – the Quality Assurance Taskforce and the Impact Assessment taskforce. Let us now take a deeper look as to how what Aflatoun’s AQIS system is all about.
t chap er 1.
Aflatoun’s Quality Assurance and Impact Assessment Strategy (AQIS)
Before we can begin discussing how Aflatoun’s AQIS system measures programme effectiveness, we first have to understand what the desired outputs and outcomes are. On the next page you find a visualization of the change the Aflatoun programme aims to bring about – our Theory of Change. Aflatoun hopes to establish change in two ways: 1. Providing a high-quality Child Social & Financial Education (CSFE) curriculum, through the programme’s five core elements. Through the curriculum, children develop competencies and start to change related behaviour, which results in their broader social and financial empowerment. 2. Engaging in global, regional, national and local advocacy. Aflatoun advocates the inclusion of CSFE in national curricula, encourages child-friendly banking and stimulates the development of child participation systems. This is done through collaborations with different institutions such as central, commercial and microfinance banks, educational departments and multilateral institutions. The diagram below illustrates Aflatoun’s Theory of Change by breaking it down into four steps: Curriculum Quality and Output, Competency Outcomes, Behavioural Outcomes and Behaviour Impact. It also shows how advocacy is an ongoing process that supports the theory of change.
AQIS tools in line with the Theory of Change
The elements of AQIS monitor and evaluate the different expected effects of the Aflatoun educational programme and advocacy. Thus, AQIS has also been split up into four sections, complementing the theory of change: Quality Assurance (QA): This monitors the quality and output of the curriculum and Aflatoun’s partner organisations. The main steps include the selection of strategic partners who are capable of programme implementation. We facilitate QA through several tools and processes that are aimed to guarantee certain standards for partner selection and Secretariat performance. Output tracking: It is necessary to understand the scope, outreach and activities of the Aflatoun Programme, and this will involve measuring output – in other words, numbers; numbers of children, amount of savings, numbers of schools etc. The Secretariat facilitates all partner organisations with tools for tracking their programme outputs. Outcome Assessment: Competency and behavioural outcomes are more difficult to quantify. Competency outcomes refer to skills, attitudes and knowledge that the children develop after having gone through the Aflatoun programme. Behavioural outcomes refer to concrete changes in children’s behaviour. To be able to measure
these, we have defined concrete indicators for these and developed a number of different research tools. Our partner database and field surveys are examples of such tools. Impact Assessment: Aflatoun has asked leading research institutions, who use cutting-edge impact assessment methods such as randomised evaluations, to look at the impact of our programme. All these projects are complementary and optional as they require substantial time - and sometimes resource - commitment. The diagram shows which tools are available at the different levels of AQIS.
The AQIS tools highlighted above are divided into two categories: - General tools for all partners. - Complementing projects that can be agreed to in cooperation between individual partners, the Secretariat and researchers. The duration of these complementing projects can range between one or more weeks (e.g. process review) to multiple years (e.g. longitudinal studies). We will look at the various tools in more detail in the coming chapters.
things simple. The tools in this guide and the partner database are meant to be easy to use. 3. Cost effective: All partners are given a free Aflatoun partner database and template of field surveys. 4. Open: There are several ways in which the learning in the different elements of AQIS are shared and used to progress our overall knowledge: - Network sharing: The design, method, and results of the evaluation tools are shared over a global network of partner organisations and research institutions. This is done through regional and international meetings, learning & facilitation and the member’s area of our website. - Partner database: We have developed a comprehensive database to be used by partners and in which we can collect data that will be analysed and published. The software for the database is open-sourced, which means that it can be used by other organisations. We aim to spread knowledge rather than limit it to our network! - Publications: The key programme results of the whole network are published in our annual Children & Change publication. An important part of the quality assurance results will be also included in Aflatoun’s annual report.
Principles of AQIS
In order to make AQIS powerful and effective, we follow five principles: 1. Smart 2. Simple 3. Cost Effective 4. 5. Open Collaborative
1. Smart: In the network-wide field surveys we have tried to focus on the most important competencies and indicators of the programme, as per Aflatoun’s theory of change. 2. Simple: We aim to make your and our work easier and keep
- Curriculum development: Results from our AQIS tools will provide key inputs for the development of the Aflatoun Curriculum and materials. 5. Collaborative: We are part of a network, and can draw on each other’s knowledge, skills and experience to constantly raise the quality of our programme in all aspects. Experts, Partner organisations, teachers and children are all involved in our processes. Specifically for AQIS, Aflatoun consults: - The AQIS (Impact) Advisory Committee: An expert committee with members from a variety of research fields. The committee meets bi-annually and advises on AQIS and organisational strategy, as well as the methodology and implementation of specific research projects and evaluation tools. - The Quality Assurance Taskforce and Impact Assessment Taskforce: During Aflatoun’s global Campaign Launch in March 2008, several groups of partner organisations were formed to deal with particular topics. The Quality Assurance taskforce and the Impact Assessment taskforce consist of representatives of the Aflatoun network and delegates of our strategic international partners. They advise and decide on AQIS strategy and
implementation as well as provide an on-the-ground perspective. - Children and teachers: To be able to test the tools, we asked a group of teachers and students to give their feedback, which has proven to be extremely valuable. Aflatoun’s AQIS is moving in an exciting direction. The more we understand about quality and impact of our programme, the more we can make the change that we want to see in the world!
t chap er 2.
Quality Assurance (QA)
certain quality standards for organisations joining the Aflatoun network in all the regions. The selection process happens in parallel to the six steps of programme implementation as described in the partner manual. There are three phases in the partner selection process: Phase 1: Initial contact Phase 2: Associate partner Phase 3: Implementing partner The partner selection process is summarized in the graph below. This graphic shows the three phases in the partner selection process, and how they correspond to the six steps of programme implementation.
The main role of Aflatoun’s Quality Assurance (QA) is to maintain high programme quality and delivery. It is composed of three elements: 1) Quality partnership selection process – setting standards for partner organisations wishing to be part of the Aflatoun Network 2) Training – ensuring that the concept of the programme is clearly communicated to partners and teachers 3) Secretariat participatory evaluation- To ensure that the Secretariat is carrying out its function effectively and efficiently as per the needs of the partners and the programme. In this manual, the QA tools we will focus our attention on are the partnership selection process and the Secretariat participatory evaluation. More information on Aflatoun’s training model and strategy can be found in the Training Manual.
As the Child Social and Financial Education movement grows and the Aflatoun Network expands, it is crucial to protect the integrity of the Aflatoun Concept and Programme. This begins with setting
Duration of selection process The duration of the selection process differs between organisations and can vary between 12 and 24 months. This will typically depend on the amount of time which an organization needs to establish organisational and financial support for the program. The Aflatoun Network also reserves between 1 and 12 months for its visit to programs already running live in schools. The general rules for partner selection are explained in the following section. Please refer to the “QA Questions and Answers” section of the member’s area of our website for exceptions and conditions for the signing of the partnership agreements. Phase 1 – Initial Contact This is an exploratory phase for both Aflatoun and the new organisation that has just been in contact with the Secretariat. It corresponds to the first two steps of the implementation process: Step 1 – Initial Preparation Step 2 – Planning The key elements of this phase are outlined below. 1.1 Research and Initiate Contact The Aflatoun Secretariat has set criteria for partner selection in line with the vision of Aflatoun to reach every child with CSFE. The Secretariat reserves the right to work with countries that
are strategically important, and organisations that request the programme.
Secretariat 1.1 Research and Initiate tContact Our priorities for country selection a. Countries with a child population of above 500 thousand b. Low income or middle income countries c. Countries in conflict or high need Initial Contact The selected organisation must meet some of the following criteria Focus fields: a. Child rights based and/or a child centric organisation b. Experienced in rights based/ social development education c. Experience in financial education d. Focus on poverty/child poverty e. Experience in training f. Experience in working with the education system or the non formal sector Organisational: g. Member of a larger organisation /network h. Leadership and commitment i. Ethical fibre j. Scale mindset k. In existence for 3 years minimum l. Sound financial statements of the last 2 years/ externally audited accounts m. Documented annual report last 2 years
Once the Secretariat team determines that the potential organisation meets most of the criteria for selection, the next phase is reached. Contact with the potential partner organisation is usually initiated through: - A recommendation from a member of the Aflatoun network. - Desk research by the Secretariat - Referral from another network /a multinational/ a bi-lateral organization - A member/ part of a large organization eg. INGOs - Self referrals - Potential partners met by Aflatoun staff in meetings or conferences attended
Secretariat 1.2 Scoping Provide Documents a. Partner Manual b. Questionnaire for Potential Partners
Initial Contact Complete & Submit - Questionnaire for Potential Partners - National legal NGO registration form - Organisational annual report for last 2 years by independent, accredited accountant - Externally audited accounts of the organisation for the last 2 years - Two references by international Donor, from multilateral/bilateral/ INGOs
1.2 Scoping The Secretariat provides the potential organisation with core documents to ensure understanding of the Aflatoun concept. At this stage, the organisation is requested to complete the potential partner questionnaire. If, upon completion of this phase, the Secretariat feels the contact does not meet the necessary criteria, Aflatoun can choose not to proceed with the partnership, and close dialogue. In some cases, a site visit may be conducted.
* Annual Report for 2 years, must include: - Financials according to internationally defined standards - Externally Audited Financials/ Accounts - Show a variable donor base for financial sustainability
1.3 Dialogue While still in active dialogue with the Secretariat, the organisation starts its first two steps of implementation: Initial Preparation and Planning. The organisation is requested to provide some documentation: The action plan in which the organisation outlines its strategy, planning and budget for programme implementation. The plan will be discussed with the Secretariat, and will serve as the basis for further
planning and communication. Written statement from education department (local/ national) granting permission to implement the Aflatoun programme.
Secretariat 1.3 Dialogue Provide Documents a. Action Plan Template b. Compilation of books and sample books if required Initial Contact Complete & Submit - Action plan and realistic budget - Provide written statement by the local/national education department granting permission to implement the Aflatoun programme (if applicable) Implement - Step 1: Initial Preparation: & Advisory Committee Meeting - Step 2: Planning Comments Feedback on the position of education departments
will prepare a summary of all relevant information in order to begin the approval process. Once approval is granted by the General Board of Aflatoun, the organisation is eligible to move to Phase 2. The partners in the Initial Contact Phase are welcome to participate in the regional meetings, twinning and sub-regional meetings.
Secretariat 1.4 Approval Review & Approval Process a. Approval by Programme Manager b. Approval by Management Team c. Approval by Board – Regional Representative d. Ratification by the Board. Extend Invitations to a. Regional & subregional meetings b. Twinnings Initial Contact If the organisation is interested in attending meetings they do so at they own expense, exceptions are made on need basis. Comments In case Aflatoun will not proceed with the partnership, an e-mail will be sent to the organisation to communicate this.
If the organisation is working in a country where an Aflatoun partner is already present, the two organisations will be linked so they can collaborate. 1.4 Approval Once all these stages have been completed, the Aflatoun Secretariat
Phase 2 – Associate Partner 2.1 Signing Memorandum of Associate Partnership (MAP) In this phase, the organisation becomes an Associate partner. This is formalised with the signing of the Memorandum of Associate Partnership (MAP)1 by the Associate partner and the General Aflatoun Board.
This can be found in the Additional Materials booklet as part of the Aflakit 15
2.1 Signing Memorandum of Associate Partnership (MAP)
Review & Sign MAP
Legal sign off of Memorandum of Associate partnership (MAP)
Secretariat 2.2. Implementation Provide Documents: a. The eight workbooks b. Trainer manual c. Games manual d. Support via phone / e-mail when required. Training of Trainers a. Provide all training materials b. Training during sub-regional meetings, subjects demand-driven Subject to availability of funds
Associate Partner Implementation - Step 3: Materials Development (contextualisation and translation) - Step 4: Training - Step 5: Live in Schools Notification - Notify Programme Manager when programme goes live in schools
Comments An organisation is understood to be “live in schools” when the first children in the programme have their first Aflatoun classes (using the contextualised materials and the lessons being given by trained teachers).
2.2 Start of Implementation of Steps 3, 4, 5 of partnership process The Associate partner organisation is requested to start the implementation of steps three to five of programme implementation: Step 3 - Material Development Step 4 - Training Step 5 - Going “Live in Schools” or within in-formal settings The Aflatoun Secretariat will be in close touch with the associate partners to support them throughout the process.
2.3 Sharing & (Partner) Training Associate partners will be invited to attend and participate in the Aflatoun events at least once a year. Every year Aflatoun alternates between Regional Meetings (partners from a specific region) and International Meetings (partners from all regions). Partner organisations are invited and encouraged to attend.
Secretariat 2.3 Sharing & Training Invite partner organisation to: a. Sub-regional trainings b. Regional meetings c. International meetings d. Twinnings e. Member area f. Email-based consultative process of the network
Associate Partner Attend a. Regional meeting (once every 2 years) b. Sub-regional trainings c. International meetings (once every 2 years) Subject to availability of funds 2.4 Learning & Facilitation Visit Report
Secretariat Provide Documents: a. Guide for learning & facilitation visits b. Actual learning & facilitation visit within one year after associate partner goes “live in schools” c. visit to the programme site by the Aflatoun Secretariat d. Briefing with partner staff on lessons learned from visit
Associate Partner Preparation a. Help with the preparation of the learning visit b. Recommendation report done jointly by partner and Aflatoun Secretariat
2.4 Learning & Facilitation visit Once the organisation launches the programme and goes “live in schools” or in the non-formal settings, the Aflatoun Network commits to visiting the programme within one year. The “Guidelines for Learning & Facilitation Visits” can be found in Appendix 1 We encourage you to share these guidelines with staff and schools in your programme.
2.5 Approval The final step towards completion of Phase 2 is, once again, the approval process. The Aflatoun Secretariat will prepare a summary of the associate partner’s programme which will be presented to Aflatoun’s General Board for approval. Once this approval is gained, the organisation moves to Phase 3.
Secretariat 2.5 Approval Review & Approval Process a. Approval by Programme Manager. b. Approval by Management team c. Approval by Board – Regional representative d. Ratification by the board.
3.1 Signing Implementing Partnership Agreement (IPA)
Secretariat Provide IPA
Implementing Partner Review & Sign IPA
Comments Legal sign off of Implementing Partnership Agreement (IPA) Upon signing this agreement, the implementing partner is entitled to: a. Participate in the decision making process of the movement b. To be actively involved (if nominated by other members) on the Aflatoun Board and task forces c. To participate in twinning, (sub-) regional trainings/meetings, and international meetings d. To advocate on behalf of the movement at the global and regional level
Phase 3 – Implementing partner 3.1 Signing the Implementing Partnership Agreement (IPA) At this stage, both Aflatoun’s General Board and the partner organisation sign the Implementing Partnership Agreement (IPA)2. The partner organisation then becomes an Implementing Partner. This membership is ongoing for as long as the partner continues to implement the Aflatoun programme as specified by the IPA.
Requested documentation once an Implementing partner There remain a set of recurring documents that we request from implementing partners: 1. Each year partners are requested to provide their annual report
This can be found in the Additional Materials booklet as part of the Aflakit
and their audited accounts 2. Every 2 years partners are requested to provide an updated action plan and budget 3. When there is reason for concern about a running program, the Board may request extra documentation.
Multiple Partnerships in a Country
-The Secretariat will only really be in direct, individual contact with the coordinating/lead organization in the country. The other organisations will be included in general communication and updates and will be invited for regional meetings. We stress that all organisations, whether they are the coordinator/lead or not, are an integral and recognized part of the Aflatoun movement. - If it is complicated or impossible to have one coordinating/lead organization for any reason, multiple partnership agreements may be signed in the country, with a maximum of 5 organisations. However, these organizations will also be requested to work on developing future alliances in order to bring down the number of partners per country.
In the case where Aflatoun is in contact with more than one organisation in a country, the partnership selection process becomes somewhat more complex. On one hand, Aflatoun would prefer to work directly, and sign agreements with, the owners of the programme on the ground ( i.e. the partner organisations who are in charge of the programme at the local level). On the other hand, to make the Network more organised and manageable, we would like to have partners choose coordinating/ leading national organizations or alliances that can sign a partnership agreement with Aflatoun (instead of each individual partner). We understand that the possibility of these partnerships depends on a variety of factors (e.g. the size of an organisation/country/target population, geographic position, political/NGO landscape). The rules of thumb on partnership with multiple initial contacts in a country are: - Initial contacts in a country choose one coordinating/lead organisation from amongst themselves. This is the organisation that will sign the partnership (MAP/IPA) with the Secretariat.
Participatory Evaluation of Secretariat
Our role as Aflatoun’s Secretariat’s role is to facilitate the Aflatoun network and represent it with key stakeholders. It is therefore crucial that we respond to demands from partners. We need to engage in a process of self-reflection, asking questions like: - Do we listen enough to our partners? - Do we do what we say we will do? - Do we do things in a participatory and inclusive manner? - Is our work effective, timely and efficient?
To help us asses our performance, we ask a third party to do an annual participatory evaluation of our work within the network and with our external stakeholders. We ask for feedback from both the partners and stakeholders, and use this to assess the quality and facilitation of our communication and how effectively we use our resources.
t chap er 3.
Impact assessment: surveys and partner database
Some of the surveys are mandatory for partners to carry out, while others are optional. Required surveys are necessary in order to ensure that we have a base level of quality assurance and comparable information from all Aflatoun partners.
Required Surveys Partner Survey: Partners complete this once a year. It looks at basic organisational information and activities, and details about the programme delivery and costs. School Survey: This survey is an interview with the main Aflatoun teacher or the head teacher. It captures relevant information about Aflatoun programme delivery at the school level. It asks important questions about student savings as well. In order to make sure that Aflatoun is running well, all partners are encouraged to visit their schools at least once a year. (If you have a small programme or are in your first year, you should try to have your visits correspond with the end of the school year.) This survey can be done during those visits.
The goal of Impact Assessment is to measure the outputs and outcomes of the Aflatoun programme through the collection of relevant information from partners, schools, classes and children. The two main tools, which are the focus of this chapter, are the Surveys we have developed and the partner database, designed in collaboration with Tata Consultancy Services in India.
Survey Overview and Methodology
Five Surveys In the past, no uniform system existed to compile information for publication or internal controls. This resulted in unreliable data that was sometimes based on guesswork and estimations. Another major point of frustration for partners was that, although they could see clear results from the Aflatoun programme, there was no way of quantifying them. Our challenge was to respond to this issue and coordinate how best to measure these outputs and outcomes. One solution was to create five different surveys that would provide a comprehensive evaluation of our programme. These were tested and evaluated by the Impact Assessment Task Force and other partners in the Network. We have designed them so that they are simple and easy to use. They are also available in French, Spanish and Arabic. Surveys may need to be translated and contextualized for your programme.
Unlike the Partner and School surveys, the Class and Child Surveys are not required. That said, we believe that they provide valuable information and we would like to see as many partners as possible using them. They are designed to be quick and resulting data entry easy.
Optional Surveys Class survey: This survey will provide you a better understanding of how the programme is implemented at the class level. It is both a basic class ledger book analysis and an evaluation of the teacher’s perceptions about the programme and its outcomes. It also has a section that attempts to determine children’s understanding of key Aflatoun concepts, like the motto, character and song. This does not have to be done with all classes but with a few from each school. Young Child Survey: This optional survey looks at young children and seeks to determine if they understand the key themes and messages of the Aflatoun programme. The survey is aimed at younger children in the Aflatoun programme and questions correspond with themes from Aflatoun workbooks 1-3 (for children age 6-9). Older Child Survey: This optional survey looks at older children in the programme and attempts to see whether they know and can apply key Aflatoun concepts. The survey is aimed at older children in the Aflatoun programme and questions correspond with themes from Aflatoun workbooks 4-8 (for children age10-14).
database. As you see from the chart below, the information is used to become the basis for all major Aflatoun publications. These publications will be widely disseminated to raise the profile of Child Social and Financial education and the Aflatoun Network.
All surveys are to be handled by the Aflatoun partner. The Partner Survey should be filled out by the Aflatoun coordinator in collaboration with the partner. School, class, and student surveys are to be administered by surveyors sent by the partner.
The results and publications are also meant to be a tool for your programme. Once we have processed the information, we will provide you with reports specific to your programme and give you access to the data to so that you can use it for your purposes. We hope that it will be both an advocacy tool for your lobbying efforts and also an administrative report that details what you are doing. You may wish to use it for programme planning and fundraising, or for communication to stakeholders like the Ministry of Education.
Once all information is collected at the partner level, it is the partner’s role to send it to the Aflatoun Secretariat via the partner
Steps to Collecting Surveys
The Aflatoun team, in consultation with the Impact Assessment Taskforce, have created some guidelines to aid you in the process of filling out surveys. The steps are outlined as follows: 1. Budgeting for time and costs 2. Contextualisation of surveys 3. Creating survey packages 4. Training your Surveyors The first step is to decide which surveys you want to implement. Remember, the Partner Survey and School Survey are mandatory, whereas the Class Surveys and two Child Surveys are optional.
Tip: The information in the mandatory surveys is essential for us to be able to conduct effective Impact Assessment. We decided to make the rest of the surveys optional because we understand that they require the use of partners’ resources. Deciding which surveys to do is based on both your logistical capabilities and organisational needs. If you really want impact data, consider doing the optional surveys.
them to reach a school, how long for their lunch, how many schools can they visit per day? From this you should be able to determine how long you expect your surveyors to be at each school.
Survey Planning 1.Total Number of Schools 2. How many schools per day will you visit? 3. Time per School Visit (reference for Q.8) One School � 6 hours � Two Schools � 3 Hours � Three Schools � 2 Hours �
Based on this, you will be able to determine how much time you have in each school to do surveys. Based on pre test results, the average amount of time for each survey is provided below. Number of Surveys Per School
Survey Time Per Survey/Visit 60 minutes 30 minutes 20 minutes 20 minutes Number of Surveys Total Time (survey time X # of surveys)
4. School (required) 5. Class (Optional) 6. Young Child Surveys (Optional) 7. Older Child Surveys (Optional)
1. Budgeting for Time and Costs Time: Time constraints can affect which surveys you do and how many of them you can fill out. This will mean that you will have to plan ahead. The best way to do this is by imagining a day worth of school visits. Imagine your surveyors going to your schools. How long will it take
8. Total Time (Must be equal/less than the Time per School Visit, Question 3)
Evaluation Budget Costs1. Human Resources Costs- Cost of paying surveyors and partner staff time 2. Training Meetings- Cost of delivering training 3. Material Costs- Cost for creating surveys and printing and disseminating evaluation results 4. Travel Costs- Cost for transport between schools 5. Data Entry Costs- Costs for paying for data entry TOTAL BUDGET
Training Costs - This includes costs associated with providing training to surveyors, including space, facilitator time, and food. Material Costs - You must also take into consideration the cost of copying materials for training and surveys for the evaluation. Also included is the cost of printing results and disseminating them to your different stakeholders or potential funders. Travel Costs - Determining transport costs will depend on your organisation’s situation. If all schools can be accessed by motor bike or public transportation, then costs will be low. If your schools are spread out and require overnight stays or the use of your organisation’s vehicles to reach them, then these costs will be higher. Data Entry Costs - If you plan on paying someone to enter your data, insert the salary cost here. Total costs of the evaluation should be calculated ahead of time. This might affect some of your previous planning decisions. It might also require that you include evaluation as a future cost in outside funding proposals or when you are budgeting for the Aflatoun programme.
After deciding which surveys you are going to implement and how many schools will be visited, you can begin to budget what the costs will be for your programme. Human Resource Costs (Surveyors) - If your own staff will be conducting the surveys, then this will not cost additional money. However, the time required for your staff should be taken into consideration. Otherwise, you can pay a surveyor to visit the schools, or find volunteers. Finding good surveyors can be difficult. Surveyors should be comfortable speaking and interacting with children and should be reliable and honest. While they are working, you can ensure that they have visited all the schools by asking the head teacher to sign the surveyor’s sheet and you can occasionally call the head teachers to follow up on the surveyor’s performance.
2. Contextualisation It is important that the teachers and children understand the
This can be found in the Additional Materials booklet as part of the Aflakit
questions of the surveys very well. This means that surveys will need to be in the local language, and that they must fit the local context. Some question wording may not make sense in translation or might not be culturally appropriate. You are able to change wording of questions so long as the general theme of the survey does not change. The surveys are available in English, French, Spanish and Arabic. In the Partner Database, you will find all the translated versions as well as a blank template that can be used to translate surveys into different languages. If you are planning on translating the surveys, please use the templates provided. The important things to remember are: All survey questions must be numbered the same All survey answers must be numbered the same You will notice that in the templates provided for the child surveys, we have indicated where you should insert the relevant currency. The three questions where this is applicable are: - If you had [large local currency amount], how much would you spend right now?
- If you had [small local currency amount] every week, how much would you save each week? - Can you tell me the name of the currency in [country]? (younger child survey only) For the first question, the number should be an amount that is a sizable sum for a child. The second should be an amount a child might receive as an allowance. The responses should be in quarters and we suggest that you choose easy round numbers:
Total Currency amount 20 Dollars 100 Rupees 200 Taka 4 Euro 8 Pesos Answer 1 0 0 0 0 0 0 Answer 2 ¼ 5 25 50 1 2 Answer 3 ½ 10 50 100 2 4 Answer 4 ¾ 15 75 150 3 6 Answer 5 All 20 100 200 4 8
Choosing or Adding Questions: You can add questions that are specific to your programme, and remove you do not feel are appropriate.
3. Create Survey Packages Once you have chosen which surveys to use, translated and contextualised them, you will need to prepare a survey package to give to your surveyors. All surveys can be found in the database. They are stored in the Print Survey Forms subsection, which is within the Survey section. This is presented in greater detail in Database Manual which is found in Appendix 2.They are available as Microsoft Word documents and you should download the appropriate ones and save them on your computer. You can find a standard survey package in the available languages that contains a set number of surveys. If you want your own survey package, you can print the number of surveys you require per school and make photo copies for each school. Save paper by printing on the front and back of pages and remember to staple your packages together for safekeeping! While all this is available in the database, we suggest that you create a folder in your computer to handle all your surveys. You cannot save any changes into the documents in the database, so it is important that you have them saved for yourself. Be sure to include the Surveyor Cover Sheet3. This sheet will provide brief instructions for your surveyors. Fill in the “Expected # of Surveys” column on the survey cover page. Bundle all the surveys
with the correct number of copies of each and add the survey cover page at the top. Once you have you standard survey package, you can photocopy for the appropriate number of schools. If possible, use a photocopier that prints on two sides to save paper!
Partner Tip: If you decide to conduct the children surveys, we suggest surveying 5% of your Aflatoun children with 50 surveys as a minimum, and 1000 as a maximum. Questions should be read out to the children by a surveyor who should also fill in the forms. Remember, we are most interested in the quality and not the quantity of these surveys.
4. Train your Surveyors: Once you have set your survey process, you can begin to train your surveyors. The Secretariat has developed a simple guideline in Appendix 3 that you can use as a basis for your training. Most important is making sure that there is a common understanding of the content of the surveyors and how the process for your programme works. The surveyors can either choose to hand out the surveys to the teachers or they may choose to fill in the teacher’s answers themselves. Share the stories! Encourage the surveyors to note down anything interesting that they have noticed or heard during their visit to the school. At the back of the survey pack is a little form where they write notes about a special story, or enterprise or fun anecdote.
When to Conduct the Surveys
Now that you understand how to plan for the requirements of data collection, you can determine how to go about planning when to begin your evaluation. The Impact Assessment Taskforce and the Secretariat request that all partners do their evaluation towards the end of their country’s school year. This would allow for Aflatoun students to be in the programme for the longest period of time before they are surveyed and thus have the most accurate programme data. You have therefore to decide when the best time for beginning your process is. This requires you to determine the following three things: 1. Preparation time: How long do you expect that it will take for you to find your surveyors and train them? Also, will you need to request permission to access schools? What is the minimum time required for this process? These two tasks can be done at the same time as long as they are both done before your first school visit. 2. Duration of school visits: How many weeks will it take you to complete your visits to all the schools? This can be done by a simple calculation: 3. Final date: What is the last week or month that you can enter schools? Some countries have an extended period of time for exams or class trips in the final weeks of the school year. It is
important that you think about your evaluation in terms of the last week/month that you can enter a class and access all students, as opposed to the last few days of the school year. Based on this information, you can plan ahead to insure that you have enough time to conduct your visits and that the information you get will be the most relevant.
The Partner Database
The Partner database has been designed by the Aflatoun Secretariat in collaboration with Tata Consultancy Services. This was a corporate social responsibility project for Tata Consultancy Services and they worked hard to make the database comprehensive but also easy to use. Additional information about the database and its functions are available in the Database Manual in Appendix 2. The Secretariat is also here to provide technical assistance and we are happy to help you if you run into specific problems.
t chap er 4.
Impact Assessment: Other Tools
Collecting Stories and Audiovisual Material
People can identify with stories and audiovisuals of change. Stories provide the most tangible and personal evidence of impact. Stories and audiovisuals serve multiple purposes: - They enable us all to share broadly what happens in individual Aflatoun programmes. - They can be crucial advocacy tools and can be published in different media, or in advocacy materials. - They serve research purposes. Stories can provide input for qualitative research, but can identify key indicators of success if a large number of stories is collected. Together with the Impact Assessment Taskforce, the Secretariat has drafted guidelines for collecting stories and audiovisual materials. More information on these can be found in Aflatoun’s Communication Manual. Remember, at all times, children’s rights must be protected and respected. 1. “A Child’s Story” It is Aflatoun’s mission “to inspire children to socially and financially empower themselves and become agents of change in their own
As much as the surveys tell us about the programme, we think it is essential to gather more qualitative information as well. This Chapter outlines other tools for outcome and impact assessment that are part of AQIS. Some of the tools described are for all partners - network wide tools - while others are complementary projects that may only work with individual partners. Some rely on outside expertise or volunteers and depend on additional funding. Network wide tools include: - “A Child’s Story” and “Stories of Change” - Photos, drawings and audio-visual projects Complimentary projects supplement and enrich the core Aflatoun AQIS strategy. These projects provide more in-depth impact research and quantitative research that will improve the quality of the Aflatoun Programme. They include: - Qualitative Research - Process Review - Social Return on Investment - Randomized Evaluations - Longitudinal Studies
lives and for a more equitable world.” This tool tells the story of change in the life of a child, and shows the mission at work. Writing “A Child’s Story” though, requires some preparation.
Five step plan to get to meaningful, powerful children’s stories: - Step 1: Speak to teachers and stakeholders about good examples within your Aflatoun programme. Try to get stories from both urban and rural schools, high and low income children, and both boys and girls. - Step 2: Read the “General tips for interviews. Part A”4 - Step 3: If needed and appropriate, arrange a tape recorder for the interview - Step 4: Interview the children, following the tips and the “A Child’s Story” Guidelines5 - Step 5: Write the story! Keep in mind to select meaningful/powerful quotes, and to follow the child protection guidelines
- Step 6: Write the story! Keep in mind to select meaningful/powerful quotes, and follow the child protection guidelines
3. Drawings Often, children communicate their emotions and ideas better via a drawing or painting than through words. Drawings are fun, simple but effective guides to understanding children’s experiences within, and ideas about, the Aflatoun programme. They carry great value both for advocacy and research purposes. When visiting a school – either for programme purposes or for your data collection – ask a few children whether they would like to make a drawing by asking them a general, non- leading question like: “Can you draw what Aflatoun means to you?” When selecting pictures to be shared with the Secretariat and network, it is important to keep in mind that the value of drawings lies not primarily in whether the drawing is done artistically or is particularly beautiful, but in its underlying message or story. 4. Photos, Audio & Videos Using photographs, videos and audio recordings is a great and fun way to show the world the Aflatoun programme in action! Just like the stories, audio and video provide anecdotal and qualitative evidence of the programme’s impact. When we have many of these, we can even use them as quantitative indicators for impact.
2. Stories of Change Change does not only happen at the individual child’s level, but can also occur in groups of children, within other groups of stakeholders in the programme, throughout the partner organisation, or in the local community. Meaningful “Stories of Change” in the programme and throughout your organisation are important tokens of impact.
Six step plan to get to meaningful, powerful “stories of change”: - Step 1: Talk to children, teachers, parents, volunteers, your staff, the local community, other stakeholders in and around the programme - Step 2: Identify specific events, activities, or changed competencies/ behaviour - Step 3: Read the “General tips for interviews. Part B” 6 - Step 4: Arrange a tape recorder to tape the interview - Step 5: Interview relevant participants or stakeholders of the programme
This can be found in the Additional Materials booklet as part of the Aflakit 29
A few basic tips for taking photos or recording video or audio: - Focus on children’s empowerment and stories of (positive) change - Make sure to have a mix of photos of boys and girls, young & older children - Make sure to follow the child protection guidelines for reporting on children - Test the equipment before you take it to the field! - Make sure the lighting is good when taking videos or photographs - Choose a quiet background or a background that is lively and represents the location where the recording is done - Make sure only one person at a time speaks More information can be found in the Communication Manual
- Ask oral permission from teachers and also students when taking pictures in a school setting - If you are planning on publishing photos or images for external audiences, seek the written permission of the child for the use of their image We hope that you will share these guidelines with all your staff involved in this area. 5. Case Studies Aflatoun will be doing case studies of particular Aflatoun partner programmes. This will include details about the programme implementation and how it currently operates. It will be used as a learning tool for partners both new and old and is meant to be used as a tool to facilitate discussions and sharing within the network. Case studies will be done by the Aflatoun secretariat or researchers and will include field visits, interviews with programme staff and stakeholders, as well as partner input. It will be published within the network at appropriate sharing moments and may be published more broadly.
Guidelines for Reporting on Children Aflatoun thinks it is imperative that the rights of the children we work with are protected. One of the key elements of this protection is the ethical use of the children’s images and materials. We believe that a few key principles should be followed. It is important to remember that, should a country’s laws or norms differ, that the rules of the country be respected. Be sure to check the rules in your country to see what standards you may be expected to follow. If there are no national rules, we ask partners to follow a few key principles: - Gain written permission from children, and their teacher, when doing Children’s Stories or highlighting a child in an audiovisual production
Complementary Research Projects
The remaining research tools and projects will complement our overall data collection and provide different types of information. 1. Qualitative research Qualitative research is conducted to study the quality and development of the programme, particularly Aflatoun’s core elements, workbooks and programme implementation tools (such as the piggy bank, bank account, Aflatoun clubs etc). This can be done through interviews with key informants, child interviews, participatory action research, focus groups, and focused evaluations. The resulting information will be used to refine the programme itself, as well as the impact assessment indicators. Research projects of this type are carried out by independent researchers with coordination from the Aflatoun programme managers and the partners. 2. Process Review Process Review looks at how the programme works at the partner level. Using a mix of participatory methods and business knowledge, it attempts to look at how the different components of the programme are being implemented and delivered to determine if they are effective and efficient. This is done by comparing the programme outcomes with Aflatoun’s strategic objectives, with a special focus on communication, participation, policy and decision-making processes
within Aflatoun programmes. It also attempts to look at the bigger picture to make sure that Partner programmes are working in a way that furthers the goals of the Aflatoun Network. This kind of study is usually done with research executives and graduate students. As such, it is a collaborative process, meaning that the partner and the person conducting the research work together for a common goal or objective. The results of this study can help us create guidelines of best practice for partners. 3. Social Return on Investment (SROI) Conventional financial evaluations compare output to costs, but do not capture the social and economic benefits of projects and programmes. However, Social Return on Investment (SROI) evaluations aim to capture these benefits by giving a monetary or social value to key indicators in our programme and by comparing them to the costs of implementing the programme. The initial pilot is conducted by the Secretariat and Social Evaluator team. After that, the tools will be available to all partners. 4. Randomized Evaluation Randomized evaluation is the most rigorous way of measuring the impact of social and developmental programmes. It requires that some children get access to the Aflatoun programme and others do not. You are able to determine the effect of the programme by monitoring and comparing the resulting changes within these groups.
5. Longitudinal Studies Longitudinal studies attempt to determine the long-term impact of the Aflatoun programme. Some organisations follow individual student development over an extended period of time, complimented by indepth interviews, to determine if any changes in individual behaviour can be attributed to a programme or intervention. We are considering the installation of an alumni network to keep connections with the programme & to ensure the possibility of tracking alumni.
Data Collection Summary
Phase & element 1.2 Scoping Phase 1: Initial contact - Questionnaire for Potential Partners - National legal NGO registration form - Annual report for last 2 years* - Externally audited accounts for last 2 years - Two references by international Donor, from multilateral/bilateral/INGOs - Action plan and budget ** - Provide written statement by the local/national education department granting permission to implement the Aflatoun programme (if applicable) - MAP: Memorandum of Associate Partnership 7 - Joint recommendations by partner and Aflatoun Secretariat
Below is a reminder of which documents are required from partners at the different stages of the partnership process:
Documentation / information required of partners
Phase 2: Associate partner 2.1 Signing of MAP 2.4 Learning & Facilitation Visit
Phase 3: Implementing partner 3.1 Signing of IPA - IPA: Implementing Partnership Agreement 8
*Each year partners are requested to provide their annual report and their audited accounts **Every two years partners are requested to update their action plan and budget
This can be found in the Additional Materials booklet as part of the Aflakit 33
Which documents to expect from the Secretariat All these documents can be found in the Additional Materials booklet in the Aflakit. Questionnaire for Potential Partners During the initial contact phase, potential partner organisations are asked to fill in this questionnaire. They are asked for organisational information (e.g. mission, thematic focus, budget, accountability, Human Resources, existing programmes, references). Action Plan The next element in the initial contact phase is the Action Plan. This crucial document outlines the potential partner organisation’s strategy and approach towards successful implementation of their Aflatoun programme. Intended planning, budget and outreach are included. All partners are asked to update their action plan and budget every two years. Memorandum of Associate Partnership (MAP) The MAP demonstrates the shared commitment of the Associate Partner and the Secretariat to beginning work together. It formalizes the spirit of productive cooperation and recognizes the good will of both parties to work together towards fully implementing the Aflatoun Programme. It aims to help the Associate Partner develop the capacity that will see its transition to fully Implementing Partner status. Implementing Partnership Agreement (IPA) These are signed after an Associate Partner has been “live-in-
schools” and a learning & facilitation visit by a Secretariat staff member has been done within one academic year of the programme starting live in schools. The IPA recognizes that the Associate Partner has a proven ability to deliver the programme. It sets out the respective roles and responsibilities that govern the partnership. Learning & Facilitation Visit Report with Guidelines Within one year of an associate partner going “live in schools,” the Aflatoun Secretariat will have a learning & facilitation visit to the programme site. Guidelines for these visits have been made to ensure shared expectations at the partners’ and Secretariat’s end.
Partners are requested to collect and submit data once a year. This can be done at any time throughout the school year, provided that all data has reached the secretariat before December 31st. This will allow the data to appear in our publication, Children & Change. We remind you that school surveys should be done towards the end of the school year.
Appendix 1 - Learning & Facilitation Visit Report with Guidelines
Cover Sheet Name staff member Aflatoun Secretariat: Partner Organization visited: Name staff member Partner Organization: Country: Places visited: Date of visit: Overall observations/comments: _______________________________ _______________________________ _______________________________ _______________________________ _______________________________ _______________________________
Overall suggestions for partner organization:
Overall suggestions for Secretariat:
Signature staff member Aflatoun Secretariat:
Signature staff member partner organization:
____________________________________ Guidelines for Learning & Facilitation visits
The Aflatoun Network is all about partnership and shared learning. We (the Secretariat) greatly appreciate the opportunity to visit your programme as your programme has gone live in schools. We want to learn about your good practices, the things that have worked well for you, the innovations you have made, the challenges you have faced, the lessons learned and your thoughts on what training you would like to see us provide. To help us both get the most out of our visit we have drafted these guidelines. We hope to see you soon! 1. General Expected Outcomes By the end of the visit, we hope to: □ have gained a better understanding of your programme and organization □ have all your documentation together for Board sign-off for Implementing Partnership □ have seen your programme “live in schools”, and interviewed a mix of participants □ have talked to your staff and understand your good practice, innovations, lessons learned, and training needs □ have assisted you in one or more stakeholder meetings, when needed by you □ have copies of class materials, some stories of change and children’s drawings/artwork 2. General visit overview Though all different, all visits have certain essential elements in common. We suggest a general planning, which can change according to needs. Secretariat and partner staff both are asked to use the document checklist below for your preparation of the visit.
General Scheme Day 1: Meeting with partner staff
What documents to bring? (Partner = P, Secretariat = S) For Board sign-off: □ Completed survey for potential partners (P,S) □ Completed action plan & budget (P,S) □ Partner survey (from partner database) (S) □ Annual report/externally audited accounts of last 2 years (P) □ National legal NGO regist. form (P) □ Third party references of organization (P) Other: □ Own partner’s materials (P) □ Background documents provided by partner (if applicable) (P)
Desired Results/outcomes - Open discussion - Review of survey for potential partners & action plan - Collection of other documents - Overview of training need & other partner expectations Overview of innovations, best practices, lessons learned
Required time 3-4 hrs with partner: - Aflatoun coordin. - Exec. dir. - Other interested/concerned
Day 1: Stakeholder meeting(s)
- Upon the partner’s convenience - Meeting with stakeholders (e.g. educ. dep., Unicef) - 4-5 school visited - Mix of: - Successful/less successful - Rural/urban Interviewed in total: - 2-3 head teachers - 5-6 teachers - 5-6 classes - 10-15 children of different ages
Day 2-4: Visit to schools
□ “Stories of change” (P) □ “Children’s stories” (P) □ Guidelines for stories & child protection (S) □ 6 school surveys* (S) □ 6 class surveys* (S) □ 15 children’s surveys* (predetermine ratio grade 1-3 and 4-8) (S) □ Children’s drawings
- 2-3 school visits p/day if possible - (3 days max.)
* These surveys are meant to guide, not to limit interviews with children, teachers, etc.
3. General tips The school, class and children’s surveys provide guidelines for field interviews. Some extra general tips for preparation and interviewing (both for Secretariat and partner):
Selecting Schools - Try varying between schools that are more & less successful at implementation - Try varying between rural/urban schools - Try varying between social/cultural/ethnic communities Interviewing children in class & individually Example questions other than in surveys: - “Did you participate in camp/culmination event?” - “Who generally decides where you spend your saved money on? - “Can you make a drawing of what Aflatoun means to you?” (you can ask children this question starting a school visit and collect them at the end) Selecting classes - Try varying between classes with individual savings accounts & group accounts - Try including a range of classes with fin./soc. enterprises, and innovations
Interviewing (head) teachers Example questions other than in surveys: - “How do you value the programme?” - “What do you think about the Aflatoun materials; any new ideas?” - “Did you see any unintended effects that we can share with other programmes?” - “What support do you need?” - “Can we go thru class ledger together?” - “Did you receive training? From whom?” - “Did you see any examples of change?”
General tips for interview & visit (IMPORTANT!) - Through language/culture words can be misinterpreted. Be clear, but always remain polite. - Make sure a translator is present who is familiar with the programme - At all times try to avoid prompting answers - If taking photos, recording videos or collecting drawings, get permission* for further use - To fuel learning, partner & Secretariat staff can share notes and agree main advice * Please refer to Aflatoun’s guidelines for child protection
4. Space for key notes This space is available for your notes on key points:
Key good practice/innovations: Follow-up needed: Item: Date & Owner:
Key lessons learned (partner & Secr. staff):
Suggested improvements Secretariat docs:
Appendix 2 - Database Manual This is the Database Manual. It will provide you an introduction to its design, its Database Manual
features and how best to use it. You will receive a user name and password from the Aflatoun Secretariat. When entering this information, remember it is case sensitive, meaning that if certain letters or words are capitalised, then it will not work.
The home section is the first section that you will see once you login. The names of the sections are translated into three languages and a menu with images. To move to these sections, click on the text.
There are two ways to move around the database. - In the Home Section, click on the purple text. - In the navigation bar, click on the appropriate box to move to that section.
There are six core areas in the database. They will be explained in greater detail in the remainder of the manual.
The Introduction Section is where we provide an overview of the AQIS strategy, the database, the surveys, and Social Return of Investment (SROI). Click on the document name to open it up!
Every year there will be changes in the surveys and the database. The News section will provide updates -by year- about new things for the coming year.
There are two places where you can find help. In the “?” you will be able to get help related to that page. In the Manuals-Help section, you will get other tools that provide help or describe the database.
Every year we will load all the relevant results of the annual data collections. We will provide global, national and partner reports to everyone. We are working to a more dynamic program that will allow you to do your own reports/ research.
Once you have entered your information, you will need to send the information to the Secretariat in the Netherlands. You can do this by mail on a CD or through the internet.
The Survey section is the most important area within the database. It contains 4 parts and deals with everything to do with the surveys in the system.
Click on the School Survey and you will see it! You need to fill in a school survey in order to fill in class or child surveys. This means it is the gateway for all your survey data entry.
There are two key types of navigation within the surveys. If the survey is more than one screen, you need to click on next. If you want to add additional surveys, click on the appropriate button of the bottom of the screens.
Press the Partner Checklist Button and you will see the survey! This should be completed only after you have done and entered all your school, class and child surveys.
You will be able to print your survey forms from the database. When you click on the survey, it will open in Microsoft Word. All surveys will be in English, French, Spanish and Arabic.
You will be able to see which surveys you have entered in the view survey screen. You can also add class or child surveys by clicking on the name of the school.
Once you have clicked on the name of the school, you will be able to add a Class or Child Survey from this page.
Once you have finished your work in the database, you need to log out. Click on the Log Out button on either the Navigation Bar or the Home Screen.
Appendix 3 - Suggestions for Training of Surveyors
Overview This document will provide some suggestions on how an Aflatoun partner could train the potential surveyors who will be delivering the surveys. These are suggestions and we hope that you use them as a starting point for your work. Rationale In order to have surveys that are effective, the collection of data has to be as uniform as possible. This means that all individuals who are in schools, interviewing teachers and students, should have the same understanding of how they are to go about surveying. To make sure of this, we suggest that you do training with all your surveyors to go through the data collection process, to outline your requirements, to clarify survey information and respond to any questions they might have. Timing and Logistics We suggest that this process should: - Take approximately 3 hours - Be done in a group with all surveyors if possible - Be done before they go to their first school - Be participatory, within the spirit of the Aflatoun programme
Suggested Exercises and Activities In order to get the best out of your surveyors, we will need to both empower them, motivate them and inform them about what they are supposed to do: - Introductions: Introduce yourself and give an opportunity for all your surveyors to get to know one another! - Overview of Aflatoun and Data Collection Process: What is Aflatoun and why are you doing survey research? It is important for them to know why they are doing this work and what the programme is about. - Survey Logistics: It is important to be very clear with what you expect of your surveyors. Take them through exactly what they should do in each school that they go to. - Survey Testing and Role Playing: It is important that you be able to see people administer the surveys and to let people practice. Have each surveyor ask questions and record answers with another surveyor. Encourage constructive feedback within the group and take some time at the end to determine what has been learned. - Reading the Surveys: Every surveyor should have an identical understanding of the survey questions. This means that you should read through the school, class and student surveys and answers. If people have questions, speak about them as an entire group. - Questions and Opinions: Give your surveyors an opportunity to talk about any questions that they might still have. Also, ask them their opinions about the surveys and the process. What they found helpful and what they think you should change in the future.
Follow Up You should be in regular contact with your surveyors. If one has an important question or comment once the surveys have begun, be sure to provide the answers to all your surveyors!
Appendix 4 - Aflatoun Survey Planning Guide
1. Survey Planning 1. Total Number of Schools 2. How many schools per day will you visit? 3. Time per School Visit (reference for Q.8) 2. Number of Surveys Per School Survey 4. School (required) 5. Class (Optional) 6. Young Child Surveys (Optional) 7. Older Child Surveys (Optional) Time Per Survey/Visit 60 minutes 30 minutes 20 minutes 20 minutes Number of Surveys 1 Total Time (survey time X # of surveys) 60 minutes � One School � 6 hours � Two Schools � 3 Hours � Three Schools � 2 Hours
8. Total Time (Must be equal/less than the Time per School Visit, Question 3) 3. Evaluation Timing 9. Last week you can visit schools 10. Number of weeks required to visit schools (Number of schools/Number of schools per day) 11. Number of Weeks to find/train surveyors and gain permission to enter schools if needed 12. Required start time for evaluation
4. Evaluation Budget Costs 1. Human Resources - Pay for surveyors and partner staff time 2. Training Meetings - Cost of delivering training 3. Material Costs - Cost for creating surveys and printing and disseminating evaluation results 4. Travel Costs - Cost for transport between schools 5. Data Entry Costs - Costs for paying for data entry TOTAL BUDGET
Aflatoun, Child Social & Financial Education Sarphatistraat 7 | PO Box 15991 | 1001 NL Amsterdam ph: +31 20 626 20 25 | fx: +31 20 626 21 18
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.