Workshop on Improving Gender Statistics in Rwanda

Session 5

Statistics’ Quality and Comparability:

Metadata and International Comparisons

Serena Lake Kivu Hotel, Rubavu District

March 25-27, 2014
 Learning Objectives
At the completion of this module, participants should be familiar with:
– What are ‘metadata’ and why are they important?
– Examples of metadata that can be used for gender indicators
– Suggestions for producing the GSF metadata
– Why and How to compare gender statistics internationally
– Current Problems with International Data Comparability
– International mechanisms for keeping informed about
developments on gender statistics

Sources:
Gardner, Jessica. “Importance of metadata.” Workshop on Writing Metadata for Development Indicators
Lusaka, Zambia, 30 July – 1 August 2012, UNECA and African Union
Oakley, Graeme, Australian Bureau of Statistics.
www.unescap.org/stat/apex/2/APEX2_S.4_conference_Statistical%20Metadata%20Standards.pdf;
OECD, Management of Statistical Metadata at the OECD, V/ 2.0, 6/9/2006.
http://www.oecd.org/std/33869551.pdf
UN Statistics Division, Department of Economic and Social Affairs. Millennium Development Goals
Indicators. Series Metadata, http://mdgs.un.org/unsd/mdg/metadata.aspx
World Bank.
2
Part 1. Metadata for
gender Statistics

3
 What are Metadata?

“Metadata: the range of “Metadata provide information on

information, generally textual, that
fosters understanding of the context
in which statistical data have been
collected, processed and analyzed
with the objective of creating
statistical information…”
African Charter for Statistics (2009)
data and about processes of
producing and using data. Metadata
describe statistical data and - to some
extent - processes and tools involved
in the production and usage of
statistical data.”
UNECE, "Guidelines for the Modeling of Statistical
Data and Metadata“ (1995).

• Provide information that defines or describes the data or statistics

• Describe the data collection, production, processing, computation and
analysis process as well as the content and source of the data
• Also discuss the limitations and quality of the data
• Created and used throughout the data production process
• Respond to and inform national standards and systems
4
Metadata include information not only about the “ingredients” (components) but
also about how the data were produced, i.e., the process
 Why Metadata are important for reporting on Gender and
Development Indicators

• Inform users about the source, definition, collection process

and limitations of the data
• Provide users with knowledge and understanding of
statistics availability and use
• Clarify indicators from multiple data sources with different
definitions, data collection process, dates, etc.
• Explain discrepancies in indicators’ estimates--e.g., for the
MDGs
• Guide new data collection and statistics’ and indicators’
production

6
 Example 1. Metadata for Rwanda GSF Indicators: What to look for

• There is no international consensus about what appropriate metadata

should contain -- different producers include different information items

• Example 1: OECD - List of common metadata items:

Item Content

1. Source from which data was submitted or extracted, including original data
source (administrative data, household survey, enterprise/
establishment survey,); periodicity; date when data was last received
2. Characteristics and unit of measure, power code, variables collected, sampling, periodicity,
collection reference period, base period, date last updated, other
3. Statistical population geographic, sector, institutional, item, population and product coverage
and scope of data
4. Statistical concepts and key statistical concepts and classifications
classifications used
5. Manipulation & aggregation and consolidation; estimation, imputation, transformations,
dissemination weights, seasonal adjustment, other manipulation and adjustments;
OECD dissemination format, related publishing
6. Other aspects: recommended uses and limitations, quality comments, other comments
Source: OECD, Management of Statistical Metadata at the OECD, V/ 2.0, 6/9/2006.
http://www.oecd.org/std/33869551.pdf
7
 Example 2: World Bank’s World Development Indicators Metadata

• The metadata for the World Bank’s World Development Indicators includes the
following:
1. Code
2. Development relevance
3. Statistical concept and methodology
4. Indicator Name
5. Long definition
6. Source
7. Topic
8. Periodicity
9. Aggregation method
10. Limitations and exceptions
11. General comments

• Handout 5.1 has an example of the WDI metadata for Under-5 mortality rate
• This same metadata is being developed for the Bank’s Gender Statistics database.

Source: World Bank, World DataBank. World Development Indicators.

http://databank.worldbank.org/data/views/variableselection/selectvariables.aspx?source=world-
development-indicators 8
Example 3. Millennium Development Goals (MDGs) Indicators Metadata

The metadata for the MDGs includes the following items:

1. Contact point in the international agency that produces the data
2. Definition
3. Method of computation
4. Comments and limitations
5. Sources of discrepancies between global and national figures
6. Process of obtaining data
7. Treatment of missing values
8. Data availability
9. Regional and Global estimates
10. Expected time of release

Source: UN Statistics Division, Department of Economic and Social Affairs. Millennium

Development Goals Indicators. Series Metadata, http://mdgs.un.org/unsd/mdg/metadata.aspx

9
 Example 5: Metadata for the UN Minimum Set of Gender
Indicators (MSGI)
• UNSD has compiled the metadata for the set based on information from data
collection agencies: WHO, UNESCO, ILO, etc.
• The data and metadata are provided by international agencies based on national
data or information reported to them
• Some data are old or are estimates or projections produced by the international
agencies
– Rwanda has more recent data that is not included in the MSGI
• Metadata are available for 46 indicators; all but 2 are Tier I indicators (they meet
the 3 criteria discussed yesterday)
• The information is the result of consultation and agreement among the data
collection agencies.
• The metadata highlight the problems and limitations of the data for international
comparability;
– Some problems or limitations highlighted for the international level may not be
relevant at the national data because there may be more detailed information about
the data.

10
 Metadata for the UN Minimum Set of Gender Indicators (2)

• The metadata contain information on 10 areas—similar to MDGs metadata:

1. Indicator Name:
2. Contact point in international agency
3. International agreed definition
4. Method of computation
5. Importance of the indicator in addressing gender issues and its limitation
6. Sources of discrepancies between global and national figures
7. Process of obtaining data
8. Treatment of missing values
9. Data availability and assessment of countries’ capacity
10. Expected time of release

• For Rwanda, the UN MSGI has data for 34 of the 52 indicators, although there are
no metadata for 2 of them – see Handout 5.2
• UN MSGI metadata can serve as a source or example for producing GSF metadata:
• They can help improve the collection and quality of the gender indicators and statistics,
• But, they may not be suitable for the GSF because they were produced for the
international and not the national level – they provide information that is not relevant
at the national level

11
 Rwanda GSF Indicators: examples
Without Metadata –

• Gender parity index for secondary gross enrolment (females to males):

– 1.00 (Rwanda Education Statistics, EMIS, 2010)
– 1.04 (Rwanda Education Statistics, EMIS, 2011)

• Literacy rate among population aged 15-24, by sex (Youth Literacy rate):
– Female: 85% (DHS 2010)
– Male: 83% (DHS 2010)

• Life expectancy at birth (years):

– Female: 54.8 (Rwanda Population Projection, 2009)
– Male: 50.8 (Rwanda Population Projection, 2009)

• Maternal mortality ratio, MMR (maternal deaths per 100,000 live births:
– 476 (DHS 2010)

12
 Metadata for Rwanda GSF Indicators: Gender parity index
for secondary gross enrolment (females to males)

With Metadata
• Gender parity index for secondary gross enrolment:
• Definition: Index is the ratio of female to male gross enrollment ratios in secondary education.
– Gross enrolment ratio, GER: total enrollment in secondary education, regardless of age,
expressed as the percentage of the population of official secondary education age (World
Bank Metadata). Can be calculated separately for females and males.
• Numerator: population enrolled in secondary school
• Denominator: total population of official secondary school age
– GER can exceed 100% due to the inclusion of over-aged and under-aged students
because of early or late school entrance and grade repetition (World Bank WDI
Metadata).
• Unit: ratio
• Computation:
• Numerator: GER Females (enrolled females as a % of all females of secondary school age)
• Denominator: GER males (enrolled males as a % of all males of secondary school age)
• Measurement/Estimation: GER females/GER males.
• Source: Administrative data—Rwanda Education Statistics, January 2011
• When and how were the data actually collected?
• Lead Agency/producer: Ministry of Education?
13
 Metadata for Rwanda GSF Indicators:
Literacy rate among population aged 15-24, by sex (Youth Literacy rate):

With Metadata
• Definition: The percentage of the population aged 15–24 years who can both read and
write with understanding a short simple statement on everyday life. (Source: UIS)
• Unit: %
• Computation: Literacy rates are computed by dividing the number of persons [females
or males] aged 15-24 years who are literate by the total [female or male] population in
the same age group. The result is then multiplied by 100 to yield the literacy rate in per
cent. (Source: UIS)
• Importance of the indicator in addressing gender issues and its limitation: The Youth
Literacy Rate reflects the outcome of primary education over the previous decade. As a
measure of the effectiveness of the education system, it is often seen as a proxy
measure of social progress and economic achievement. Reasons for failing to achieve
the literacy standard may include non-attendance, low quality of schooling or dropping
out before completion of primary education. Differences in literacy levels between
young women and men will often reflect recent inequalities in access to formal
education and persisting inequalities in adult life and the world of work. (Source: UIS)
• Source: DHS 2010
• Lead Agency/producer: Measure DHS?
14
 Metadata for Rwanda GSF Indicators: Life expectancy at birth by sex

With Metadata –
• Life expectancy at birth:
– Definition: Estimate of the average number of years a newborn is expected to live based
on current age-specific mortality rates.
• Life expectancy at birth by sex gives a statistical summary of current differences in
male and female mortality across all ages.
• In areas with high infant and child mortality, the indicator is strongly influenced by
trends and differentials in infant and child mortality (Source: UN DESA, Population
Division, 2011).
– Unit: Number of years
– Year/Date: Calendar year when data were collected -- When and how were the data
actually collected?
– Source: Rwanda Population Projection 2009 – Based on which data sources: census,
survey, administrative records, several sources?
– Assumption: Current age specific death rates/mortality patterns will remain constant in
the future
– Lead Agency/producer: NISR?

15
 Metadata for the Rwanda Maternal Mortality Ratio, MMR
• Definition: Annual number of female deaths from any cause related to or aggravated by pregnancy or
its management (excluding accidental or incidental causes) during pregnancy and childbirth or within 42
days of termination of pregnancy, irrespective of the duration and site of the pregnancy, per 100,000
live births, for a specified year (WHO).
• Unit: Ratio
• Computation:
– Numerator: For Rwanda: Any death that occurred during pregnancy, childbirth, or within two
months after the birth or termination of a pregnancy. Includes all deaths occurring during the
specified period even if due to causes that are not pregnancy related (DHS 2010).
– Denominator: International convention--Number of live births (in 100,000s), based on either a
written record or the mother's recall.
– MMR for Rwanda-DHS 2010: Expressed per 100,000 live births; calculated as the maternal
mortality rate divided by the age-adjusted general fertility rate, which is the average number of live
births per 1,000 women of reproductive age (age 15-44) (DHS 2010).
• Year/date: Fieldwork was conducted September 2010-March 2011 (DHS 2010).
• Measurement/estimation: For Rwanda: Women respondents reported the number of their sisters who
died, and the number who died of maternity-related causes. No definitive procedure for establishing
completeness or accuracy of retrospective data on sibling survivorship (DHS 2010).
• Source: DHS 2010.
• Lead Agency: MEASURE DHS (previously Macro International)
• Limitations of the indicator: Based on recall of deaths and live births by key informant (mother’s sister
or mother). DHS uses a different denominator to overcome the limitations of the recall.
16
 Suggested steps for producing metadata for the GSF indicators:

Not a prescriptive or exhaustive list:

1. Decide who to partner with on this work – e.g.,
• NISR metadata initiative
• Possible pilot with one data collection exercise-e.g., survey
2. Decide which template to use
• Adopt features from different examples and customize them for Rwanda’s
needs
• May involve consultation with data producers and users
• Explore experiences from other countries in Africa or developing countries
3. Decide which indicators to focus on first: main ones, by sector, or other criteria
• GSF has hundreds of indicators so it will take a lot of time and work to produce
metadata for all
• Consider piloting in one sector or one data collection initiative
4. Go to the data sources for Rwanda to find out information about data collection
process, definitions used, limitations and problems, coverage and response rate,
collection dates, etc.
• May involve reviewing data collection manuals, templates, questionnaires and
consulting with staff involved in the process
17
 Suggestions for producing metadata for the GSF indicators

5. Complement with information from international sources when

appropriate
• e.g., internationally agreed definitions, importance from a gender perspective,
limitations, comparability
6. Consult and negotiate with data producers and compilers about feasibility
of using the framework or template in censuses, surveys, administrative
data collection –
• NISR, line ministries, civil registration personnel, international agencies
• This may involve an iterative and continuous process to adjust and improve the
template
7. Agree on and harmonize the metadata with all the data producers
8. Publish and circulate the metadata to all producers and users
9. Train data collectors on how to produce and report the metadata
• Training may need to be provided to data collectors, compilers and reporters
every time a new data collection is started

18
 Disseminated data should always be accompanied by
metadata

• Disseminated data should be accompanied by metadata to help users

understand the data.
– Can be included after the data (each section), as an annex, or as links to the
indicators in electronic format

• Metadata should include, but is not limited to, information on:

– Concepts, definitions and classifications used
– Basic features of the data sources
– Data collection methodology: censuses, surveys, administrative records
– Guidelines on use of the data
– Data quality (e.g. sampling and non-sampling error, non-response rates, data
comparability).

• In some circumstances, it can be useful to release particular types of

metadata in a dedicated publication.
– For example, in Vietnam, the GSO published a Gender Statistical Handbook in 2011.

19
Part 2. Comparing
Gender Statistics
Internationally

20
 Benefits of International Comparability
of Gender Statistics
• Similarities and differences in gender issues between individual countries
and between regions can be studied and relative progress on gender-
related goals can be assessed
– by undertaking data comparisons across countries or regions.

• The overall quality of a country’s statistics can be enhanced

– because producing comparable statistics involves adoption of
international standards and best practice in methodology.

• Gender issues and developments can be analysed in an international

context
– by combining statistics across countries to produce regional and
global aggregates.

21
 Bringing gender statistics from different countries together:
• The World Bank’ Gender Statistics electronic database
– http://datatopics.worldbank.org/gender/
– Allows users to compare statistics on gender for regions and countries in 6
areas: economic structures and access to resources; education; health and
related services; public life and decision-making; and human rights of women
and girl children.
– The data come from the World Development Indicators and additional sources.
– Users can create their own tables and download the data into Excel

• The UNSD publication The World’s Women 2010, Trends and Statistics
– highlights the differences between the status of women and men in various
areas of contemporary life.
– covers 196 countries across the world.
– presents and analyses data at global, regional, and individual country levels.
– http://unstats.un.org/unsd/demographic/products/Worldswomen/Executive%2
0summary.htm
– The following charts from that publication illustrate how country data can be
brought together to inform gender issues in a wider context.
22
 Global sex distribution

Source: UNSD The World’s Women 2010: Trends and Statistics

23
 Regional sex distribution

Source: UNSD The World’s Women 2010: Trends and Statistics

24
 Time spent in domestic work per day, 1999-2008

Source: UNSD The World’s Women 2010: Trends and Statistics

25
Employed persons in vulnerable employment by region and sex, 2004-2007

Source: UNSD The World’s Women 2010: Trends and Statistics

26
 Current Problems with International Data Comparability

• Gaps in the availability of gender statistics in many countries

– For example, in 12 of the 22 topic areas specified in the 2012 UNSD Global
Review of Gender Statistics Programmes, more than a quarter of 126
countries reviewed were not producing any gender statistics.

• Lack of comparability in many of the gender statistics that are available for
individual countries

• International standards for producing comparable gender statistics are not

available or incomplete for some topics

• Existing international standards are not always fully implemented by

countries
– For example, a country may consider that a particular standard classification is
not useful, impractical or inappropriate in its circumstances and therefore not
adopt the standard.

27
 International mechanisms for keeping informed about
developments on gender statistics
• United Nations Statistical Commission (UNSC)annual meetings
– Reports on gender statistics produced and available for all meetings in the
last few years.

• UNSD Global Forum on Gender Statistics biennial meetings

– A range of documents, presentations and reports are publicly available.
– The last Global Forum was held in Jordan in March 2012 and focused on
women’s empowerment;
– Next Global Forum is planned for 2014 and expected to focus on gender
analysis and use of gender data and indicators.

• UNSD regional meetings and training workshops

• UNECA meetings

28
 Exercise 5.1: Group Activity

1. Provide several examples that illustrate the value of

producing gender statistics for Rwanda that are
comparable with those of other countries.

2. What are the main obstacles to producing

internationally comparable data on gender statistics?

3. What actions would need to be taken, and by whom, to

ensure gender statistics in Rwanda are internationally
comparable?

29