Asset-V1 IDBx+IDB17.3x+3T2019+Type@Asset+Block@M3 Evaluation and Information Systems

LEADERS FOR
CITIZEN SECURITY AND

JUSTICE MANAGEMENT
3 INFORMATION AND
EVALUATION SYSTEMS
Inter-American Development Bank (IDB)
Course: Leaders for Citizen Security and Justice Management
Module author
Liliana Manzano Chávez
Researcher and Professor, University of Chile
Editors and contributors to the module

Viviana Vélez-Grajales
Innovation in Citizen Services Division/IDB
Bárbara Paola Cedillo López
Andrés Restrepo Restrepo
Rodrigo Serrano
Andrea Cabezón
Citizen Security Studies Center, University of Chile
Course academic coordinators

Karelia Villa Mar
Alejandra Mohor
Citizen Security Studies Center, University of Chile
Course instructional design

Ana Novinsky Leirner
Inter-American Institute for Economic and Social Development (INDES)/IDB
Design and layout

Manthra Comunicación
Graphic identity and cover

Luna Cammarata y Amalia Alonso
This course was developed by the IDB in collaboration with the Academic Consortium of the University of Chile, John
Jay College of Criminal Justice, and Santa Maria School of Law (FADISMA).
Inter-American
Development Bank
September 2018
1300 New York Ave NW, Washington, DC. 20577
http://www.iadb.org
Copyright © 2018 Inter-American Development Bank. This work is licensed under a Creative Commons IGO 3.0
Attribution-NonCommercial-ShareAlike (CC BY-NC-SA 3.0 IGO) (http://creativecommons.org/licenses/by-nc-nd/3.0/igo/
legalcode) and may be reproduced with attribution to the IDB and for any non-commercial purpose in its original or in any
derivative form, provided that the derivative work is licensed under the same terms as the original.
Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration
pursuant to the UNCITRAL rules. The use of the IDB’s name for any purpose other than for attribution, and the use of IDB’s
logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as a part
of this CC-IGO license.
Note that link provided above includes additional terms and conditions of the license.
The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Inter-
American Development Bank, its Board of Directors, or the countries they represent.
INDEX
1. Presentation........................................................................................................................5
2. Introduction .......................................................................................................................8
3. Information Systems on Citizen Security......................................................................10
3.1. Use of Information Systems..............................................................................................10
3.2. Indicators on Citizen Security..........................................................................................13
3.3. Advantages and Disadvantages of Information Sources........................................14
4. Sources of Information in Latin America and the Caribbean ..................................18
4.1. Criminal Acts or Offenses..................................................................................................18
4.2. Perpetrators............................................................................................................................26
4.3. Victims .....................................................................................................................................34
4.4. Violence ..................................................................................................................................40
4.5. Consequences of Crime and Violence...........................................................................50
5. Evaluation of Citizen Security Policies..........................................................................54
6. Methodologies and Indicators for the Evaluation of
Citizen Security Programs...............................................................................................61
6.1. Ex ante Evaluation ...............................................................................................................61
6.2. Design Evaluation and the Use of the Logical Framework Tool............................63
6.3. Processes Evaluation...........................................................................................................67
6.4. Results Evaluations..............................................................................................................70
6.5. Impact Evaluation.................................................................................................................72
7. Conclusion .........................................................................................................................79
8. Glossary...............................................................................................................................82
9. Bibliography........................................................................................................................87
Optional Reading....................................................................................................................96
1. PRESENTATION
Violence and crime in Latin America have experienced strong

growth in recent decades, positioning themselves as priority issues
on the agendas of the countries of the region. However, important
deficits have been identified in the design, management and evalua-
tion of security and justice policies.
This module discusses the relevance of the use of information for

decision making in the field of public policy on citizen security, the
different types of information sources and the concepts of diagno-
sis, monitoring and evaluation are known. Additionally, the types of
indicators to measure results in the different phases of the imple-
mentation of the programs are explained. And, finally, the charac-
teristics of the different methods of evaluation of citizen security
programs, as well as their suitability and their feasibility of applica-
tion are described.
Module 3:
6 Information and Evaluation Systems
LEARNING OBJECTIVES:
At the end of this module you will be able to:
◊ Reflect upon the usefulness and the use of information

on citizen security.
◊ Identify types of relevant information sources in the

areas of coexistence and citizen security, highlighting
their strengths and weaknesses.
◊ Identify different sources of information available at

international, regional and sub-regional, national and
local (district/community) levels.
◊ Define concepts of diagnosis, monitoring and evaluation.
◊ Identify the different types of indicators to measure

results in the different phases of the implementation of
the programs.
Leaders for Citizen Security
and Justice Management
7
GUIDING QUESTIONS:
◊ What are the purposes and contributions of information

systems for the development of citizen security and coex-
istence policies in the region?
◊ What are the most used sources of information in this

field? What tools and indicators are contemplated within
these information systems?
◊ What are the main attributes and limitations of these

sources of information? How have data collection tools
been perfected, considering high technical standards?
◊ Why evaluate? What are the main functions and uses of

evaluation in the field of coexistence and citizen security
policies?
◊ What does it mean to evaluate within the public policies

cycle? To what extent does evaluation generate value
in the process of decision making and design of public
policies?
◊ How are evaluation models classified? What are the main

characteristics, advantages and disadvantages, as well as
the most used methodologies and indicators in each case?
Module 3:
2. INTRODUCTION
Many recent studies show the growth that the phenomena of violence and crime have
experienced in Latin America in recent decades. As a result, security has been placed
among the priority issues of citizen demands and in the political agendas of the govern-
ments of the region. However, various analysts, both from an academic perspective as
well as from political spheres and public opinion, consider that the response from gov-
ernments and public institutions in the region to this problem has been insufficient, and
sometimes ineffective or with short-term impacts, pointing out important deficits in the
design, management and evaluation of security and justice policies (UNDP, 2013; CAF,
2014; Frühling, 2012; Dammert et al. 2010).
In order to perfect the policies that seek to solve this complex situation, it is necessary
to develop better quality information systems, both administrative registers and other
autonomous measuring instruments that record data on the concept or phenomenon
being studied, which allow making diagnoses according to the regional, national and
local reality, improve management and promote the evaluation of citizen security poli-
cies (Beato, 2005; Frühling, 2012, Dammert et al., 2008; Salazar, 2013).
In particular, to evaluate the results of specific programs or projects it is necessary

to build coherent and feasible systems and methodologies in relation to the available
resources and time, which implies a series of aspects that make this process a highly
complex task.
An effective security policy requires the use of an accumulation of modern methodologi-

cal tools and the application of a set of political and practical criteria to ensure its proper
implementation. In turn, the evaluation highlights the need to combine multiple inter-
ests and face various obstacles. Although the evaluation of citizen security policies in the
region continues to be unsatisfactory, thanks to the recent adoption of a “results-based
management” approach, assessment has improved substantially due to the permanent
monitoring of policies/programs, and the execution of results and impact evaluations.
These not only show compliance with short-term procedures and goals, but also the
achievement of the objectives proposed by the programs and their impact on the direct
beneficiaries.
9
In the first part of this module we will look at the main characteristics of the information
systems developed to collect data and provide useful information for the diagnosis of
the situation, as well as the systematization of experiences and the analysis of results
in the field of citizen security and justice in the region. These systems aim to guide the
design of policies, accompany their implementation and provide information for their
evaluation.
In the second part we will explore the main components of the different models or types
of evaluation, the methodologies and tools that are applied in each type of evaluation,
and the most recommended indicators in each case, particularly with regard to the anal-
ysis of programs and projects related to citizen security policies. These are introductory,
but above all practical, content that will allow the director or public manager in charge of
a specific program or project to propose guidelines for its evaluation, either at the design
stage, during its execution, or once it has ended.
Module 3:
3. INFORMATION SYSTEMS
ON CITIZEN SECURITY
3.1. USE OF INFORMATION SYSTEMS
Public security statistics systems, defined as “the set of statistical information on the
subject, which, for the process of production, management, analysis and dissemination,
requires the interaction and coordination of institutions and organizations that are
linked together” (ECLAC, 2014: 9), allow decisions to be made from a technical perspec-
tive, in interdependent spheres of public administration, and with respect to different
activities, as shown in the following diagram.
Table 1. Statistical Information is Useful in Three Areas
Research in Public
Administration Planning
Policies
◊ Organization of ◊ Decision making ◊ Understanding crim-

resources, effective around a set of meth- inality as a complex
and efficient manage- odologies to achieve a phenomenon, within a
ment of them; goal; legal, social, economic
context, etc.
◊ Monitoring of objec- ◊ Phases: diagnosis,
tives and goals of definition of objec- ◊ Public Policy Designs
institutions; tives, methodologies, based on evidence, use-
and methods, means of ful in ex ante and ex post
◊ Resource targeting verification evaluation
Source: Own elaboration based on the United Nations (2003) and ECLAC (2014)
Information systems can therefore be useful at different stages of the public policy
process, from identification, planning and design, to implementation and evaluation.
However, statistical information records must meet criteria to ensure their quality. The
Office of the United Nations (2014) highlights the following:
11
◊ Integration of different sources of information.
◊ The information must be timely and credible, in order to facilitate collection and
achieve legitimacy.
◊ It must have its financial sustainability guaranteed.
Security policies address complex social phenomena, such as crime and violence. To
facilitate the definition of these policies it is therefore necessary to study these phenom-
ena in depth, considering the different aspects involved in them. It is necessary to study,
for example, criminal acts, perpetrators and victims, violence, and the consequences of
these problems. For information systems, each of these aspects implies an object that
needs to be identified, measured and described. But these “objects of measurement” are,
in turn, the main “objects of intervention” with respect to which security and coexistence
policies are formulated.
These phenomena or measurement objects are usually analyzed from multiple sources
of information, which provide us with various indicators, as listed in Table 2.
Table 2. Information Sources, Objects of Intervention and Key Indicators1
Objects of Source of Information Examples of Indicators1

Intervention
Reports
Police records
Arrests
Criminal acts Admission of cases
Judiciary records
Case Rulings
Victimization surveys The prevalence of victimization
1 The indicators presented in this table are only some examples of the indicators that can be used in each area or subject
of intervention of the security policies; they do not correspond to an exhaustive registry of all the possible indicators,
since these vary between countries and even within a country - between states/regions - according to the availability of
information of each institution.
Module 3:
Police records Detainees (presumed perpetrators)

Judiciary records Charged / Condemned
People serving sentences in peniten-
tiary facilities (by system or type of
Penitentiary system records
center)
Perpetrators Censuses of the prison
People detained according to type of
population
crime
Demographic/social profile of the
perpetrator
Demographic/social profile of the
Self-reports
perpetrator
Police records
Demographic/social profile of victims
Judiciary records
Victims Prevalence of victimization (by crimes)
Victimization surveys
Re-victimization (by crimes)
Public opinion surveys
Reaction to crime (consequences)
Reports of violent crimes
Police records
Arrests for violent crimes
Causes for violent crimes (admissions
Judiciary records
Violence and rulings, sentenced and convicted)
Health system records
Expertise in homicide and sexual
Forensic science services
assault
Surveys on violence
Perception of insecurity or fear of
Victimization surveys crime
Public opinion surveys Trust in criminal system institutions
Consequences
of crime and Political studies Evaluation of criminal system
violence institutions
Percentage of GDP spent on com-
Studies of the cost of crime bating crime, violence and its
consequences.
Source: Own formulation.

13
3.2. INDICATORS ON CITIZEN SECURITY
What do we mean by indicator? How are they formulated and classified? These are
some of the topics that we review in this section.
“An indicator is a numerical relationship that is established between two or

more pieces of significant data and that provides information about the state
of a system or its variations” (Mohor, 2007: 2).
The indicators, in terms of citizen security, can have a variety of sources and be con-
structed by different procedures, but, undoubtedly, the ones used most are those from
administrative records (police, judicial and penitentiary system), as well as victimiza-
tion surveys. The indicators allow us to understand the magnitude, distribution (ter-
ritorial and population) and evolution (temporal) of phenomena such as those already
mentioned.
Worth noting among the indicators of crime and violence from administrative records are:
◊ Homicide rate per 100,000 inhabitants.
◊ Suicide rate per 100,000 inhabitants.
◊ Death rate by traffic accident per 100,000 inhabitants.
◊ Death rate by firearm per 100,000 inhabitants.
◊ Rate of reports of sexual crimes per 100,000 inhabitants.
◊ Rate of reports of intrafamily/family violence per 100,000 inhabitants.
◊ Rate of reports of child and adolescent abuse per 100,000 inhabitants.
◊ Rate of theft per 100,000 inhabitants.
◊ Rate of robbery per 100,000 inhabitants.
◊ Rate of kidnapping per 100,000 inhabitants.
◊ Rate of robbery and theft for every 100,000 registered cars.

Module 3:
Besides these, the following indicators can be obtained from surveys on the subjec:
◊ Prevalence of criminal victimization in people over 18 years of age.
◊ Prevalence of intrafamily, family and domestic violence.
◊ Prevalence of sexual violence.
◊ Levels (percentages) of perception of insecurity in people over 18 years of age.
◊ Levels (percentages) of trust in public institutions, especially police and justice.
Quantitative information for the assessment of the management of public agencies, such
as performance indicators and effectiveness of institutional work, satisfaction and pub-
lic trust, etc., should also be added to these indicators. Similarly, it is possible to measure
economic indicators, which allow us to evaluate the cost of the programs or projects that
are implemented. For example: public spending on security programs, cost effectiveness
and cost of crime, among others.
To find out more about the construction of indicators, consult the reading Construction
of Indicators (optional reading).
3.3. ADVANTAGES AND DISADVANTAGES OF

INFORMATION SOURCES
Among the sources of information available to analyze phenomena in the field of security,
such as those already mentioned, we can consider: victimization surveys and statistics
recorded by the police, the judicial system and the prison system, among others. These
sources allow us to analyze the occurrence of crime and the different stages related to
its detection and prosecution: from the experience of victimization, reporting, deten-
tion, to the possible conviction within the prison system.
As illustrated in the figure proposed by Dammert et al. (2010), when going from one stage
to another, the volume of information about crimes and the actors involved decreases,
while the complexity of recorded information increases.
15
Figure 1. Information on the Occurrence, Detection and Prosecution

of the Crime
Estimated Crimes Victimization Polls
Registered Crimes Police Statistics

Causes Entered
Public Prosecutor
Causes Terminated Judicial Power
Type of
Termination
Conviction Judicial Power

Convicts Penitentiary System
Source: Dammert et al., 2010
Each of the sources of information has advantages and disadvantages, which should be
considered when using this type of data. The advantages and disadvantages presented
by administrative information records are set out below.
Among the advantages or strengths of official records are the following (UNDP, 2013;
Maguire, 2012; Dammert et al., 2010).
◊ Public institutions act as guarantors of the rigor of the data.
◊ The fields and concepts used are usually standardized and vary little over time,
which allows the analysis of trends and comparisons in the area to be made.
◊ There is a systematic record of information. This enables the analysis of time series
or trends.
◊ They collect information about a large number of variables.
◊ There is information available regarding different geographical levels (national,

provincial, municipal).
Module 3:
However, this type of source also has disadvantages or limitations that must be taken
into consideration for its proper use. Among the weaknesses, the following stand out.
(UNDP, 2013; Maguire, 2012; Dammert et al., 2010):
◊ The records of the different organisms may not be comparable.
◊ Not all crimes committed are reported, meaning there is a dark (hidden) figure2 of
criminality.
◊ Access to this type of information may be limited or partial.
◊ The data can be used for political purposes.
◊ Not all the actors involved in the collection and analysis of this type of data have
the same criteria.
◊ The processing of this type of data tends to take time.
◊ The trust that exists in the police and in the Public Ministry determines the volume
of information collected.
Ultimately, for purposes of comparability between different sources of information at

national and international levels, data analysis faces a set of challenges. As a way of sum-
mary, Frühling and Mohor (2009) present the following table, in which the difficulties of
comparative studies are set out.
2 Dark figures are those crimes that occur in a given area and time, but do not become known to the police because they
are not reported or detected in flagrante delicto.
17
Table 3. Matrix of Factors that Affect the Comparability of Crime Statistics
Problem of continuity Problem of congruence
Differences (in form) between

Legal terms and Changes (in form) in laws, legal countries in laws, legal processes
conditions processes and practices over time. and differences (informal) in their
application and practices.
Changes in the (formal) rules of Differences between countries in

Statistical statistics collection and produc- the (formal) rules for the collec-
conditions tion and (informal) application of tion and production of statistics
those rules over time. and their (informal) application.
Changes in the propensity to com- Differences between countries in

mit crimes, the structure of oppor- the propensity to commit crimes,
Real-world
tunities, probability of detection, the structure of opportunities,
conditions
propensity to report and similar probability of detection, propen-
factors. sity to report and the like.
Source: Westfelt and Estrada (2005), cited in Frühling and Mohor (2009: 6).
Module 3:
4. SOURCES OF INFORMATION IN LATIN

AMERICA AND THE CARIBBEAN
The following are the main sources of information available in Latin America and the
Caribbean (LAC), which provide information on the following aspects: a) criminal acts,
b) perpetrators, c) victims, d) violence and e) consequences of crime and violence. For
each object of intervention, one or two of the most relevant information sources are
described, and their main characteristics, indicators, purposes, usefulness, advantages
and disadvantages are analyzed.
4.1. CRIMINAL ACTS OR OFFENSES
When talking about crime, the debate usually focuses on the criminal acts, those that
correspond to “an omission or intentional act in violation of criminal law, committed
without defense or justification and sanctioned by the state as a felony or misdemeanor”
(Tappan, 1947: 100). This definition considers criminal acts as those classified as felonies
in each country, i.e., that violate legal provisions defined and classified within each coun-
try or each state (in the case of federated countries).
To organize the available data and carry out the desired analyzes, the crimes must be
classified. At national level, a uniform classification of crimes based on legal codes facil-
itates the comparative analysis of data and the exchange of information between the
different institutions of the criminal justice system. At regional level, standardized defi-
nitions and classifications are necessary for the comparability of records and for analyz-
ing regional trends.
Example of international classification of crimes, proposed by the United

Nations Office on Drugs and Crime (UNODC) (2015):
◊ Acts that lead to death or intend to cause death.
◊ Acts that cause harm to the person.
◊ Injurious acts of a sexual nature.

19
◊ Acts against property that involve violence against a person.
◊ Acts against property only.
◊ Acts that involve controlled psychoactive substances or other drugs.
◊ Acts that involve fraud, deception or corruption.
◊ Acts against public order or authority.
◊ Acts against public safety.
◊ Acts against the natural environment.
◊ Other criminal acts not classified elsewhere.
Information regarding criminal acts or felonies can be obtained through multiple sources
according to the specific object or indicator to be analyzed. The main sources used in the
countries of the region to analyze crimes are: police records, judicial system records and
victimization surveys. In this section, we will address the first two and the third will be
looked at later.
ADMINISTRATIVE RECORDS OF POLICE SOURCES
The administrative records from police institutions correspond to the most widely used
sources of information in the field of security policies, because they present multiple
advantages (Dammert et al., 2008). They have stable indicators over time, collected
under standardized systems, applied routinely and they respond to broad territorial
coverage.
From reports filed by the community, as well as their own procedures (controls, arrests,
investigations), the police register a large amount of information regarding allegedly
criminal acts. These registers usually include types of crimes, characteristics of the event
(time, place, use of weapons, etc.), actors involved (victims and potential perpetrators),
as well as offences and disorders reported by the community.
Nevertheless, most police institutions only publish aggregate statistics of reports and
arrests, and these records have a dark figure. Dark figures are those crimes that occur in
Module 3:
a given area and time, but that do not become known to the police because they are not
reported or detected in flagrante delicto. As an example, homicides and theft of property
show a higher level of reporting than other crimes and, therefore, a lower dark figure
(Dammert et al., 2008).
There are four factors that can affect, to a greater or lesser extent, the administrative
records of police sources: the list of reportable crimes, the rules for counting crimes, the
conduct of police reporting and the conduct of citizen reporting.
Factors that Affect Police Records
List of reportable crimes: defines what crimes are included when compiling
statistical registers. This list may be modified according to legal adjustments,
which may result in decriminalizing an offense or defining another one as a
crime.
Rules for counting crimes: indicate how to proceed to count offenses repeated
by the same offender, how many and which of the offenses should be recorded.
Police reporting conduct: police officers have a certain degree of discretion

regarding which facts observed or reported allegedly correspond to a crime.
The information that is recorded may be subject to police objectives or criteria,
which define how the information is recorded, what information is recorded
and where the arrests are recorded. Additionally, the police may have a biased
approach to a certain profile of individuals, which could be due to their social
status or their age group, among other aspects.
Citizen reporting conduct: whether citizens report the criminal acts of which
they witness, or victims depends on a set of variables. Included in these are
trust in police institutions, the belief that the perpetrator will be found, or the
property lost will be recovered, reports for insurance claims.
Source: Maguire, 2012.

21
Usually, the information collected by the police is published according to two indicators:
reports and arrests. This information is presented, according to certain time intervals
(annual, quarterly) and certain territorial divisions (country, states, regions, districts,
etc.). The reports are usually restricted to crimes of greater volume and impact for the
community, including robbery, property theft, violent robbery, injury and homicide.
These crime indicators are usually expressed in terms of absolute frequencies or in rates
per 100,000 inhabitants. Using rates makes it possible to compare territorial units that
have different population sizes.
For example, in the Region of Maule (Chile) the amount or frequency (absolute) of police
cases for crimes of Greater Social3 Connotation for 2015 was ten times greater than the
3 According to the Undersecretary for Crime Prevention, interpersonal or violent crimes, such as robbery with violence,
robbery with intimidation, robbery with surprise, injuries, homicide and rape, are considered among the most socially
charged crimes. In addition, crimes against property are included: theft of a vehicle, theft of a car object or an object
from inside the vehicle, theft in an inhabited place, and other robberies using force and theft.
Module 3:
number of police cases for the same group of crimes in the Aysén Region. However, when con-
sidering the size of the regional population it should be noted that the rate of police cases per
100,000 inhabitants is practically the same for both regions. This means that the magnitude of
the criminal phenomenon is similar in both regions.
Table 4. Frequency and Rate of Police Cases for Crimes of Greater Social
Connotation in Chile, in 2015 and 2016.
2015 2016 % of rate

POLICE CASES
Frequency Rate w/10K Frequency Rate w/10K variation
Region of Africa and
5.881 3.340,2 6.120 3.520,4 5,4%
Parinacota
Region of Tarapaca 14.568 4.156,3 14.297 3.999,2 -3,8%
Region of Antofagasta 25.982 4.277,5 23.041 3.758,2 -12,1%
Region of Atacama 9.654 3.320,8 9.739 3.329,0 0,2%
Region of Coquimbo 22.163 2.879,0 22.878 2.935,2 2,0%
Region of Valparaiso 64.879 3.505,7 63.460 3.398,1 -3,1%
Region of Lib.Bernando
25.437 2.748,9 25.611 2.744,6 -0,2%
O´Higgins
Region of Maule 27.111 2.588,2 25.754 2.442,0 -5,7%
Region of Biobio 61.230 2.916,9 57.378 2.719,1 -6,8%
Region of La Araucanía 28.897 2.860,1 26.914 2.644,8 -7,5%
Region of Los Rios 9.799 2.547,1 10.095 2.621,6 2,9%
Region of los Lagos 23.987 2.701,0 22.577 2.514,5 -6,9%
Region of Aysen 2.838 2.580,7 2.485 2.242,5 -13,1%
Region of Magallanes 3.532 2.191,4 3.425 2.120,0 -3.3%
Metropolitan Region 269.724 3.749,4 248.037 3.422,6 -8,7%
Unpecified 195 - 407 - -
COUNTRY TOTAL 595.877 3.335,4 562.218 3.123,1 -6,4%
Note: includes official data from Carabiniers of Chile [Chilean national police force] and Investigations Police of Chile
Source: Center for Crime Studies and Analysis (CEAD), Ministry of the Interior and Public Security of Chile.
Additionally, the analysis of criminal acts based on police statistics offers the possibility of geo-
referencing. Georeferencing techniques include the geographical location, based on coordinates,
of the crimes that occurred. It identifies places with a high incidence of crime and focuses police
resources, as well as crime prevention strategies. For more information, access the Geographic
Information System (GIS) reading (optional reading).
23
ADMINISTRATIVE RECORDS OF JUDICIAL SOURCES
A second source of information on criminal acts are the administrative records of the
judicial system, from the Public Prosecutor’s Office, the National Prosecutor’s Office or
the National Attorney General’s Office, according its name in each country. In addition,
in some countries information can also be obtained from the Public Defender’s Office,
the Ombudsman, courts and tribunals that may differ (labor courts, family courts, courts
of appeals, local courts, etc.), forensic institutions and the prison system.
Institutions such as the Public Prosecutor’s Offices or District Attorney’s Offices in

those countries where a modern criminal procedure system predominates, usually have
powers to collect both reports of crimes from police forces and reports filed directly by
citizens at their national or local offices. In this way, these entities collect a large volume
of information.
Such information is usually more complete than that contained in police sources with the
full process of a case in the criminal justice system: ranging from admission, the study
of the characteristics of the specific act and those of the victim and the perpetrator,
right through to the prosecution, its ruling and types of rulings (absolution, sentences
and types of penalties) 4. Additionally, judicial records enable the study of the effective-
ness and efficiency of procedural management carried out by the justice system (Mohor,
2007). For example, the efficiency of judicial proceedings can be monitored through the
average duration of the cases.
Despite these attributes, judicial records also have limitations. One of the main draw-
backs is that the withdrawal by the victims can impact the recorded numbers. For exam-
ple, excessive bureaucracy and slow progress of the processes make the victims drop
the criminal charges. Additionally, in cases of violent crimes, for example, child abuse
or sexual crimes, the system does not have mechanisms to avoid repeating statements
and offers few protection mechanisms, which often also results in the withdrawal of the
charges (Mendes de Souza et al., 2014).
4 This type of information is available when the bodies in charge of the criminal process are adequately organized to pro-
duce this type of data.
Module 3:
On the other hand, in many countries of the region, judicial registries continue to respond
to administrative or institutional criteria that limit their systematization in an accessible
and comprehensible manner. For example, the volume of cases entered into the judicial
system is specified, but the types of crimes or other relevant characteristics of the cases
or the accused are not always stated, which would allow for better criminal and crimino-
logical analyzes. Worse still, in most cases, judicial statistics are simply not available to
the public (Dammert et al., 2010).
Access to public information has several advantages with the main ones being its func-
tioning as an instrument of democratic control of public institutions, and its impact on
the way that individuals relate to public administration. From an overall perspective,
this serves for purposes of transparency, accountability and public policy design. From
an individual perspective, it allows citizens to obtain and use data for various personal
interests, such as academic research (Síndic de Greuges de Catalunya, 2012).
As an example, it is worth highlighting the volume, quality and periodicity of the informa-
tion disseminated by the Public Ministry of Chile and the Public Ministry of Peru. These
institutions have information which includes the cases distributed geographically, types
of perpetrator, victim profiles, types of crimes, applied sentences and the process man-
agement, etc. This information is of high quality insofar as it is standardized information
at a national level, responds to the legal typology of each crime and is delivered peri-
odically. These standards allow the information to be used for multiple purposes, both
academically and for public policy.
25
Table 5. Indicators Available in the Statistics of the Public Ministry of

Chile and Peru
Country Institution Main published indicators Website

Number of admissions
Number of rulings
Number of crimes admitted
Number of crimes sentenced
Number of known defendants
Number of custodial sentences
Average processing time
Number of oral trials conducted National Prosecutor’s

Office of Chile
Public Number of precautionary measures
Chile
Ministry Number of women victims http://www.fiscaliade-
chile.cl/Fiscalia/estadis-
Number of men victims ticas/index.do
Number of admissions under juvenile
criminal responsibility law
Number of rulings under juvenile crimi-

nal responsibility law
Number of admissions under domestic

violence law
Number of rulings under domestic vio-

lence law
Number of Public Prosecutor’s Offices
Public Ministry of Peru
Number of Prosecutors
Public
Peru Number of Forensic Services http://www.mpfn.gob.
Ministry
pe/publicaciones_esta-
Number of crimes according to type and disticas/
province
Module 3:
The administrative records of judicial institutions are usually more complete than those
of police sources, since they have the full flow of a criminal case entered into the jus-
tice system. However, excessive bureaucracy and slowness of the processes, as well as
the scarce protection mechanisms, leads to the withdrawal by victims of crime that can
impact the recorded numbers.
4.2. PERPETRATORS
In addition to the criminal acts, the perpetrators are also an important object of inter-
vention for citizen security. Throughout history, much of the development of the crim-
inological discipline, the construction of theories and the performance of empirical
studies have sought to find out and understand the characteristics and motivations of
the law offenders or perpetrators.
In particular, the intended goal is to understand and describe the demographic profile
of the perpetrators, the type of crime committed and the circumstances under which
the event occurred, and especially the causes or motivations behind these events.
Independent of the theoretical perspective under which the criminal behavior is inter-
preted, at present most of the academics and designers of public policies coincide in the
fact that the causes and motivations to commit a crime are multiple and, therefore, it is
necessary to have diverse sources of information (and multiple indicators) that contrib-
ute to a more detailed characterization of the perpetrators.
Only through analyzing the demographic, social and psychosocial characteristics of the
condemned subject, as well as their criminal history (criminal career) and the elements
involved in the crime, is it possible to come closer to, in some measure, a greater under-
standing of the possible factors that led this person to commit a crime, continue his/her
criminal career or to desist from it, initiating a process of reintegration into society.
Official records from the police and judicial systems provide information about alleged
perpetrators or defendants, since when speaking of perpetrators, they must have been
convicted of a crime.
To this extent, prison systems, in their different areas or subsystems (charged or con-
victed incarcerated population, population condemned by type of conviction or by type
27
of prison), are the only institutional source that allows for the analysis of the perpetra-
tor’s profile.
Below, we will discuss the type of information provided by the penitentiary systems.
STATISTICS OF THE PENITENTIARY SYSTEM
The statistics of the penitentiary system allow for the characterization of the prison
population in global terms and with respect to specific dimensions, including the prison
population according to crimes which they are serving sentences for, types of sentences,
the system in which the sentence is served (closed or semi-open), the population with
precautionary measures, the population with prison privileges, sentence time, previ-
ous criminal history, and demographic characteristics (sex, age, nationality, occupation,
education).
The main indicator for diagnosing the phenomenon of crime and violence in the region,
which can be extracted from these sources, is the prison population per 100,000 inhab-
itants, which not only accounts for the levels of crime and violence in the country, but
also the rate of incarceration. However, other indicators, such as overpopulation of the
system or admission to serve time with new sentences are also relevant.
In Latin America, most countries make administrative records about their prison sys-
tems. However, access to this information is not always public. Another weakness that
records of the penitentiary systems present is that the registration criteria are not stan-
dardized in most cases, which prevents comparative analysis at regional level.
However, international organizations such as the Institute for Criminal Policy Research
(ICPR) contribute towards greater standardization, comparison and dissemination of
prison system indicators worldwide, representing a useful source of information for the
countries in our region.
Module 3:
The ICPR provides an online database with information about the world’s
prison systems. On its website, it is possible to find an updated report for each
of the countries in the region with indicators such as:
◊ Total criminal population (including pre-trial prisoners).
◊ Rate of people serving sentences for every 100,000 inhabitants.
◊ Percentage of the pre-trial population.
◊ Percentage of the female population.
◊ Percentage of underage population.
◊ Percentage of foreign population.
◊ Number of prisons.
◊ Capacity of prison system.
◊ Occupancy level of the penitentiary system.
◊ Prison population trends according to year, population and rate.
Source: http://www.prisonstudies.org/
ALTERNATIVES TO PENITENTIARY SYSTEM STATISTICS
Given the deficiencies of the official prison records, and the difficulties that exist to
access this information and to learn about the profile of the perpetrators serving sen-
tences, there are several quantitative and qualitative studies on this population, which
allow access to more detailed and complex information about those who have committed
crimes. Among them are the census of the prison population and the self-report surveys.
The prison population census reports information on people held in penitentiary cen-
ters such as the prison population, distribution by sex and age, nationality, marital status,
fertility, educational level, peer group, health, alcohol and drug use, type of crime, ethnic
origin, disability, religion, employment, criminal behavior, living conditions in prisons and
inmate population by custodian-guardian, among other data.
29
First National Penitentiary Census (Peru, 2016)
This was held in 67 prisons nationwide, registering a total of 77,086 people

serving sentences in prisons in the country. Some of the fields of information
available from this census are: the size of the prison population, the distribu-
tion by sex and age, nationality, marital status, fertility, educational level, peer
group, health, alcohol and drug consumption, ethnic origin, disability, religion,
employment, criminal behavior, living conditions in prisons, and relations with
institutions.
The National Institute of Statistics and Computing of Peru publishes some data
on the perpetrator profile on its website, which was obtained from the First
National Penitentiary Census:
◊ 132% overpopulation.
◊ The average age of the penitentiary population is 36 years old.
◊ 70% of the prison population was born in the province.
◊ 39.8% of the penitentiary population has completed secondary school

studies.
◊ 24.6% of the prison population has a diagnosed disease.
◊ 24.6% of the penitentiary population consumed drugs during the last year.
◊ 29.8% of those surveyed had a relative in a prison.
◊ 29.5% of the penitentiary population committed the crime of aggravated

robbery; 8.9%, illicit drug trafficking and 8.7%, rape of a minor.
◊ 76.5% of inmates used firearms to commit their crime.
More information about the census in:

https://www.inei.gob.pe/media/MenuRecursivo/publicaciones_digitales/Est/Lib1364/libro.pdf
Module 3:
The self-report surveys correspond to questionnaires in which the respondents give an

account of their own delinquent behavior, even if this was reported or known by the police
or judicial system, with the purpose of overcoming the biases of the official records. The
self-report surveys seek to collect information on prevalence and frequency of the com-
mission of crimes, on the details about criminal behavior (initiation, motivation, criminal
career) and the profile, and context of the perpetrator (McAra and Mcvie, 2012; Perez-
Luco et al., 2012).
Information gathering programs in the prison system have been promoted in the United
States and some European countries. For example, about every six years the BJS (Bureau
of Justice Statistics) of the U.S. Government Department of Justice (DOJ) carries out the
survey on the inmates in detention - one for sentenced prisoners and another one for
prisoners on trial - with 10,000 inmates in each survey (www.bjs.gov). A survey about
the social climate in prisons is also carried out annually (Prison Social Climate Survey
-PSCS-) on inmates and, separately, on prison staff, in order to evaluate the management
of each prison unit.
In LAC, there is limited experience in applying this methodology. In addition, these have
not been institutionalized as a system for collecting information which is regularly
applied, mainly because the application of these instruments entails high financial costs,
but also because of the ethical implications presented by studies carried out with the
incarcerated population. In 2013, for the citizen security report of the United Nations
Development Program (UNDP), penitentiary surveys were managed in the province of
Buenos Aires (Argentina), the state of São Paulo (Brazil), the central regions of Chile, and
national surveys in Peru and El Salvador. Also, since 2002 there have been four surveys
on inmates in prisons in the Federal District (D.F.) and the State of Mexico (2002, 2005,
2009, and 2013), and a survey on inmates in federal prisons in Mexico (2013). (Bergman
et al., 2014).
31
Offending, Crime and Justice Survey: a Self-report Survey
At the international level, an example is the survey on Offending, Crimes and

Justice that was applied from 2003 to 2006 by the Research and Statistics
Directorate of the Home Office (Ministry of the Interior) and the National
Center for Social Research (NatCen) in the United Kingdom. The main objec-
tive of the application of this instrument was to measure the prevalence of
crime and drug use in the youth population of England and Wales.
The sample considered people between 10 and 25 years old from households
in England and Wales, which is distinct from most surveys of this type in the
United Kingdom which are usually applied to people serving a sentence for one
or more crimes.
The questionnaire considered the following topics: characterization of the

household, respondent and neighborhood; contact with and attitude to the
criminal justice system; victimization; antisocial behavior; consumption of
alcohol and drugs, commission of crime and its characterization. From the
information obtained, Hales et al. (2009) identified the following findings:
◊ Number (proportion) of prolific offenders who commit a large propor-

tion of crimes.
◊ Women are less likely to commit crimes than men.
◊ Antisocial behavior can be a precursor to committing crimes.
◊ Family, peers and schooling have a significant impact on the pathway

towards youth delinquency.
For more information about the survey visit: :

http://natcen.ac.uk/our-research/research/longitudinal-analysis-of-the-offending-crime-and-justice-survey/
and database at: https://discover.ukdataservice.ac.uk/series/?sn=2000042
Module 3:
Penitentiary Surveys in Latin America
General Objective
To produce reliable information for identifying and quantifying the following

sets of characteristics of the prison population in Latin America:
◊ Demographic and socioeconomic details.

◊ Family context during childhood, prior to detention, and at present.
◊ Conditions of community social cohesion of the place of residence.
◊ Type and circumstances of the offence.
◊ Use of psychotropic substances and weapons.
◊ Operation of the criminal justice system.
◊ Living conditions and health in prison.
Pre-requisites
To obtain basic information on which the different instruments will be designed.

The greater the availability of this data, the greater the accuracy of the results
that will be obtained. A minimum list of data is the following:
◊ Distribution of prisoners by federative entity and by prison unit.

◊ Distribution by gender.
◊ Distribution by legal status (sentenced, prosecuted, without penal respon-
sibility, etc.).
◊ Administrative staff per unit, by region and totals.
◊ Yearly inflows and outflows of the prison system.
◊ Distribution by generic types of crimes (robbery, homicide, etc.).
◊ Total beds or available spaces according to international conventions.
◊ Total youth in detention systems for juvenile offenders.
◊ Budgetary information.
33
Operational Planning Stage
To consider some central aspects that can determine the success or limitations
of a survey and especially a census. For example, if it is not possible to access a
penitentiary facility when a census is taken, valuable information is lost, and no
other information can be used in its place.
To guarantee the implementation of data collection it is necessary to ensure the

following steps in the planning stage.
◊ Collect the necessary and detailed information in the “Prerequisites”

subsection in the methodology section of this chapter to determine the
sample.
◊ Obtain the explicit and written authorization of the highest prison author-
ity to carry out the work (this is important because there might be a direc-
tor who is not willing to cooperate).
◊ Ensure that regional and penal institution directors grant permits to
access prisons.
◊ That they assign a contact person in each prison to be responsible for
ensuring the work of the pollsters and survey supervisors.
◊ To specify supervisors, they must obtain details about the characteris-
tics of each prison (visiting days, security protocols for entering, security
specifications, etc.).
◊ To coordinate with the contact people on the physical facilities for the
administration of the survey (classrooms, open spaces, etc.), and ensure
that they are sufficiently similar in all prisons.
◊ To ensure that the inmates’ lists to determine the samples are ready at
least 24 hours before starting the survey.
◊ To have the support of criminal security personnel to locate the inmates
selected for the survey.
Source: Bergman et al. (2014). Penitentiary surveys in Latin America.

Conceptual and methodological tools for the design, implementation and evaluation.
Module 3:
4.3. VICTIMS
Victims are “people who, individually or collectively, have suffered damages,

including physical or mental injuries, emotional distress, financial loss or
substantial impairment of their fundamental rights, as a result of actions or
omissions that are in violation of criminal laws in force in the Member States,
including the one proscribing criminal abuse of power” (UNO, 2007: 313).
Victimology is the branch of criminology that focuses on the study of victims. It charac-
terizes them, evaluates the causes and consequences of victimization, the interaction of
victims with the justice system, and studies the prevention of crime (Daigle, 2017).
Nowadays, the sources of information regarding victims in LAC come mainly from offi-
cial records (police, judicial and forensics) and victimization surveys.
VICTIMS AND OFFICIAL RECORDS

(POLICE, JUDICIAL AND FORENSICS)
Police records in LAC have always featured information concerning the victims of crimes.
However, for a long time, this information has been restricted to basic aspects, e.g., the
demographic profile, aspects related to the reporting of the crime without going into
depth.
Also, this information was rarely systematized and distributed because, to a large extent,
police and judicial work focused on the persecution of the offender and not on knowl-
edge about victims, their reality, or on the avoidance of going through their victimization
experience again (Pasara, 2015; Riego, 2014).
During the mid-2000s, after the beginning of the of Criminal Procedure Reform process
in most of the countries in the region, the role of the victim in the judicial process was
brought to attention. This led to the recognition of victims’ rights, the creation of victim
assistance units, and the systematization of statistics regarding them (Riego, 2014).
35
Presently, several countries in the region have judicial and forensic records of crime vic-
tims. However, although in some cases this information is published on a regular basis,
its systematization and standardization are still limited, since in most case the reports
are, very technical and unintelligible. For example, monthly and annual newsletters
on performed procedures are issued at the National Institute of Forensic Sciences of
Guatemala 5, but these are based on medical definitions. The National Institute of Legal
Medicine and Forensic Sciences of Colombia, on the other hand, publishes reports con-
taining information about common criminal definitions which is summarized with easily
understandable graphs.
Table 6. Statistics on Violent Deaths in Colombia. Monthly newsletter

(August 2017)
2016 2017
Manner of
death
Male Female Total Male Female Total
Homicide 6.291 586 6.877 6.292 597 6.889
Transportation
3.572 841 4.413 3.395 780 4.175
accident
Accident 1.392 329 1.721 1.717 531 2.248
Suicide 1.073 241 1.314 1.263 288 1.551
Total 12.328 1.997 14.325 12.667 2.196 14.863
Source: Monthly statistical newsletters, Instituto Nacional de Medicina Legal y Ciencias Forenses
[National Institute of Legal Medicine and Forensic Sciences].
http://www.medicinalegal.gov.co/boletines-estadisticos-mensuales
Despite the fact that official records have information about the victims of crime, in
terms of volume, this information remains insufficient since most of the people who suf-
fer from crimes fail to report the facts to the authorities.
5 http://www.inacif.gob.gt/index.php?option=com_content&view=article&id=96&Itemid=2
Module 3:
VICTIMS IN THE VICTIMIZATION SURVEYS
Between the 1970s and the early 1980s, the first victimization surveys were imple-
mented as a mechanism to fill the information gaps created by the non-reporting of vio-
lence. Between mid and late 1990s, these surveys were implemented in Latin America.
According to the “Inventory of Victimization Surveys in Latin America and the Caribbean”
(CDEUNODC, 2013), out of 23 Latin American countries consulted, 17 countries and
two cities 6 implemented at least one victimization survey, but only ten countries apply
this type of surveys on a regular basis at national level.
The survey of victimization is a collection of quantitative information, measuring the

experience of having suffered a crime, either personally, or by any member of the house-
hold, between six and 12 months prior to the implementation of the instrument (INE,
2011). This instrument is used through an interview conducted on a sample of the pop-
ulation consulting topics such as: victimization by different types of offenses (both the
reported and unreported acts), repeated victimization, willingness to report and the
reasons for not reporting, fear of crime, changes in routine activities, confidence in
the police and other institutions of the criminal justice system, among other subjects
(Aromaa, 2012). In this way, victimization surveys provide a more comprehensive and
detailed picture of delinquency and its consequences, complementing the information
provided by administrative records.
With regard to methodologies, the inventory optimistically acknowledges that given the
similarity of recorded crimes and the methods applied, it is possible to conclude that
progress has been made in the region towards a greater standardization of these instru-
ments for generating comparable data.
However, in order to extend the production and use of this type of instrument a num-
ber of obstacles present in some of the countries of the region will have to be overcome
(Aromaa, 2012; Lagos and Dammert, 2012):
◊ Budgetary: lack of permanent national budgets.
◊ Technical specialization: public officials with a specialized training are required.
6 Bolivia, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Guyana, Honduras, Jamaica, Mexico,
Panama, Peru, Dominican Republic, Uruguay, and Venezuela. Cities of Buenos Aires and Quito.
37
◊ Distrust of system stakeholders: these stakeholders usually see these new sources
as a threat.
◊ Manipulation of results: it is recommended that these instruments be conducted

by and their results disseminated by independent institutions
Victimization surveys, as well as other sources of information, have strengths and weak-
nesses. The main ones are:
Table 7. Victimization Surveys
Advantages Disadvantages
◊ They allow for the recording of ◊ They are not very sensitive at measuring
a greater number of crimes than violent crimes (domestic violence, sexual
official records. crimes), especially when these are commit-
ted by acquaintances.
◊ They are necessary to capture
crimes suffered by individuals ◊ They are not sensitive at measuring crimes
and members of the household and offences that affect groups/communities.
and are applicable to the entire
population. ◊ They are an inadequate measure of orga-
nized crime (human trafficking, drug traffick-
◊ They are adequate instruments ing and other crimes).
for measuring crimes on the
public streets and committed by ◊ They involve methodological biases derived
strangers. from sampling, the instrument, field work,
response rate, inaccurate counting and cod-
◊ These instruments capture a wide ing, among other factors.
range of information including:
victimization, characterization of ◊ They have margins of error, so in less fre-
the victim, fear of crime and trust quent crimes or in subsampled groups, less
in police institutions. accurate results are obtained.
◊ The processing and analysis of ◊ Costs involved in the application of this

this type of instrument is usually instrument are high, so they cannot be con-
fast. ducted frequently.
Source: Own elaboration from UNDP (2013); Aromaa (2012); Maguire (2012).
Module 3:
As highlighted in the above-mentioned report, so far only two countries in the region
have managed to overcome these obstacles and institutionalize their victimization
surveys at the national level: Chile and Mexico. Chile’s National Urban Safety Survey
(ENUSC, according to its Spanish acronym) is the only one that has implemented the
instrument on a yearly basis in the region. From 2003 to 2016, 12 versions have been
applied. Only in 2004, as an exception, the survey was not conducted. For more infor-
mation regarding Chile’s experience, you can read “Buenas prácticas en los sistemas de
información” [Good Practices in Information Systems] (optional reading).
In Mexico, the National Survey of Victimization and Perception of Public Safety (ENVIPE,
according to its Spanish acronym) has been implemented on a yearly basis from 2011 to
the present day. The previous version of this instrument, the National Insecurity Survey,
was implemented seven times between 2002 and 2010. (CDEUNODC, 2013).
The following graph shows the countries which carry out national or regional surveys
on citizen security or victimization, highlighting the year in which the implementation
process began.
39
Table 8. Victimization Surveys in Latin America and the Caribbean
Non-periodical Regional surveys,

National surveys of
national either periodical, or Victimization modules
periodic deployment
surveys (1 or 2 deployed at specific within other surveys
(annual or biannual)
deployments) times
Buenos Aires and Brazil (National
Barbados (since Argentina
other large cities Household Survey,
2002). (2017).
(since 1995). 2009).
Bogotá, Cali and
Peru (National Survey of
Chile Brazil Medellín (since 2003,
Budget Programs, since
(since 2003). (2010-2012). deployed several
2010).
times).
Dominican Republic
Colombia Costa Rica Large cities of Bolivia
(National Household
(since 2012). (2008-2010). (2011).
Survey, since 2005).
El Salvador Quito, Ecuador (2003,
Guatemala
(2004 and 2004, 2008 and Guyana (LAPOP, 2014).
(since 2004).
2009). 2011).
Jamaica (2006,
Ecuador (2008 Large cities of Peru
2009, 2012 and
and 2011). (2005), Lima (2009).
2013).
Mexico (2002- Honduras (2014
2010/2011-2017). and 2016).
Nicaragua (since
Paraguay (2009).
2007).
Panama Venezuela (2006
(since 2010). and 2009).
Uruguay
(2009-2016 / 2017).
Source: Own preparation based on information provided by the Center of Excellence for Statistical Information on Government Public
Safety, Victimization and Justice (CDEUNODC); the Hemispheric Security Observatory of OAS, and various official sites (electronic links
in the bibliographical references are at the end of the reading).
Module 3:
However, even in countries where victimization surveys are institutionalized, accord-

ing to Aromaa (2012) there has been no systematic use of their information for several
reasons. Firstly, because governments do not spread the results nor offer convincing
interpretations of their information. Secondly, because researchers do not provide any
analysis that is plausible and accessible for the common public. And finally, because aca-
demic researchers have not been able to build communication bridges between the aca-
demic activity and policy makers (Aromaa, 2012).
For the region, the challenges of implementing periodically the national victimization
surveys, ensuring they maintain consistency of methodology over time which allows
them to obtain quality and representative information from each country still remain.
Along with these efforts, work should be done towards the construction of standardized
instruments that address crime categories and similar issues. This will make the com-
parability of data between countries in the region easier, identifying similar patterns of
behavior, levels of victimization, particular uses of violence, among other aspects. For
more information, consult the reading [Criteria of quality or technical rigor] (optional
reading).
Additionally, it is worth highlighting the periodic deployment at the regional level of two
public opinion surveys: Latinobarómetro and Proyecto de Opinión Pública de América
Latina [Latin American Public Opinion Project] of Vanderbilt University (LAPOP). These
surveys facilitate the comparative analysis of topics such as victimization, fear percep-
tion and trust in the criminal justice system institutions, although because of the reduced
size of their national samples, they are not useful for an analysis at national/local level.
For more information about these surveys, consult the reading [Examples of public opin-
ion surveys] (optional reading).
4.4. VIOLENCE
In Latin America, between the 1990s and the 2000s, studies on violence focused on
the analysis of homicide or lethal violence, with high rates of this crime observed in the
region, particularly in countries that had faced (or were still facing) internal armed con-
flicts—e.g., Nicaragua, El Salvador, Guatemala and Colombia—but also in countries with
high levels of criminal violence, such as Venezuela. (Fajnzylber et al., 1998; Arriagada and
Godoy, 1999). In these analyzes, an epidemiological approach was adopted using con-
cepts from public health, which considered this phenomenon as an epidemic plaguing
the region (Concha-Eastman and Guerrero, 1999; Londoño and Guerrero, 1999).
41
In studies of that period, but also in more recent ones, the most commonly-used indi-
cators were frequencies and rates of complaints and apprehensions for crimes such as
homicides, injuries, injuries with the use of weapons, crimes associated with trafficking,
and carrying weapons.
Most of these studies have used data from police and judicial records, despite all the
problems they faced, such as those mentioned previously (low levels of reporting, lack of
standardization in indicators, lack of comparability between sources, lack of periodicity
and distribution of information). Only on a few occasions was it attempted to supple-
ment such information with ad hoc survey data, given the lack of periodic victimization
surveys. This has changed in the last decade with the expansion of information sources
(UNDP, 2013, CAF, 2014, Vilalta, Castillo and Torres, 2016).
Below, we will explore three of the main indicators for measuring violence in LAC: the
homicide rate, the intrafamily and gender violence rate, and the femicide rate.
HOMICIDE RATE: THE PREFERRED INDICATOR TO

MEASURE VIOLENCE IN THE REGION
The homicide rate has been the main indicator used to analyze both the magnitude and
characteristics of this phenomenon within a country and to compare the reality among
countries in the region (UNDP, 2013). It is expected that on the basis of these analyzes,
the potential causes of lethal violence will be established, and public policy strategies
will be proposed at regional and local levels (CAF, 2014).
In this context, reports issued by international organizations, such as the “World Reports
on Homicide”, published by the United Nations Office on Drugs and Crime (UNODC),
which compare homicide rates worldwide should be mentioned. Nevertheless, the focus
is mainly on Latin America’s situation. In the most recent report (2013), data from offi-
cial, police, judicial or forensic sources are presented. The difference between sources
of information is relevant to the extent that each institution has different systems of
records and there is no similarity in the measurement criteria, so the comparison of
homicide rates between countries might not be accurate.
Module 3:
Figure 2. Map of Homicide Rates in the World (data for 2012 or last
available year)
Source: UNODC (2013: 23).

https://www.unodc.org/documents/gsh/pdfs/2014_GLOBAL_HOMICIDE_BOOK_web.pdf
The UNODC report of 2013 reported a worldwide homicide rate of 6.2 cases per
100,000 inhabitants. Considering this world average, the previous map shows that sev-
eral countries in the region have a homicide rate of over 30 cases per 100,000 inhabi-
tants—Venezuela, Colombia, Guatemala and El Salvador—, while countries such as Chile,
Peru and Uruguay have a rate that does not exceed 5 cases per 100,000 inhabitants.
According to the report by UNDP (2013), over the last decade the deadly violence in
the region increased by 12%, and in 11 of the 18 countries in the region discussed in the
report, the deadly violence can be described as an “epidemic”, according to the crite-
ria of the World Health Organization (WHO), with a rate higher than 10 homicides per
100,000 inhabitants.
Another source of information that offers updated and compared data on homicides
worldwide, also expressed in georeferenced maps, is the Homicide Observatory of the
Brazilian Igarapé Institute. The most recent information reported by the site indicates
that in 2017 of the ten countries with the highest world ranking in terms of homicide
rates, eight are in the region.
43
Figure 3. Map of Homicides Worldwide (Homicide Observatory,

Igarapé Institute)
Source: Map downloaded in April 2018, from the Homicide Observatory site of the Brazilian Igarapé Institute
https://homicide.igarape.org.br/
Despite the extensive use of the homicide indicator, it should be noted that it is not the
only nor the best way to represent the real magnitude of a problem so complex as vio-
lence, especially in our region, where there are still many deficits in terms of quality of
the information provided by official sources. In the diagnosis made by Ribeiro, Borges
and Cano (2015) it was found that homicide figures are not always comparable between
countries, and not even within the same country. This is due to differences in the defini-
tions of a homicide and the unit of analysis, under-reporting and a coverage gap in police
institutions, deficiencies in the systematization of the data, in the handling of residual
categories and lost cases, among other factors.
In order to develop this diagnosis and seek solutions, the “Conference on the Quality
of Homicide Data in Latin America and the Caribbean” was held in the city of Bogota,
Colombia, in September 2015. The conference brought together members of criminal
Module 3:
justice, statistical and academic institutes with civil society from 12 countries in the
region. The main result was the drafting of a Protocol on Homicide Data Quality. The
document defined criteria and thresholds which were grouped into eight thematic areas:
a) registration unit, b) definition of a homicide, c) minimum fields to record, d) lost data
and indeterminate cases, e) convergence between sources, f) mechanisms of verification
and validation of data, g) disclosure and transparency, and h) periodicity and opportunity.
For more information regarding the Protocol, you can access the available video
(http://conferenciahomicidiosbogota2015.org)(optional resource).
NEW INDICATORS OF VIOLENCE:

INTRAFAMILIAL AND GENDER
Another indicator of violence is domestic or intrafamily violence, which is defined as

“violence exercised in the field of family coexistence or assimilated, by one of the mem-
bers against another, against some of the others (members) or against all of them” (Mora
Chamorro, 2008: 90) 7. In this section we will focus on the tools and indicators that are
used to measure and study this phenomenon and its consequences.
As addressed previously, with the objective of improving knowledge about the victims
of crime, in the middle of the 2000’s the first surveys of victimization were implemented
in some countries of the region. The data generated through these instruments allows
us to learn in greater volume the amount of unreported crimes. However, as we have
seen, several authors have claimed that these standardized questionnaires are rela-
tively insensitive when it comes to measuring crimes committed by relatives or someone
known by the victim, or that occur in the context of an ongoing process, and not just at a
specific moment sometime, as happens in cases of domestic violence and sexual crimes
(Maguire, 2012; Hoyle, 2012).
Recognizing these shortcomings, countries in the region have recently developed and
implemented data collection instruments that seek to measure, various manifestations
of domestic violence and/or violence against women in a more sensitive way. These
instruments seek, on one hand, to provide a more complete figure for the occurrence of
7 This issue will be addressed in more detail in the “Prevention of Youth and Violence Against Women” module.
45
these crimes and, on the other, to describe the socio-demographic profile of the house-
holds/women who suffer these acts, to investigate the nature of the violence (physical,
emotional, psychological and sexual), the characteristics of these acts (periodicity, repe-
tition), and the link between victim and aggressor.
Unfortunately, only a few countries in the region have implemented this type of instru-
ment (Argentina, Chile, Ecuador, Peru, Mexico, and Uruguay), and in most of these cases
they are single studies and not regular ones. Below, two examples of this type are high-
lighted; the first one is of intrafamily violence and the second one is of gender violence.
Module 3:
National Survey on the Dynamics of Relationships in Households, ENDIREH
The ENDIREH is a survey resulting from the joint work of the National
Institute of Women (INMUJERES) and the National Institute of Statistics
and Geography (INEGI) in Mexico. It has been implemented on four occa-
sions (2003, 2006, 2011, and 2016) and its purpose is to scale and charac-
terize the dynamics of couple relationships in households, as well as the
experiences of women with different types of violence, in school, work, and
the community.
In 2016, the ENDIREH was applied to 142,363 women over 15 years old
in private households, using a probabilistic, three-phase, stratified and con-
glomerate sampling. The survey has urban national representativeness for
the three populations under study: married women or in union, women once
in union (now divorced, separated or widowed), and women never having
been in union (unmarried).
Among the topics addressed by the survey are the following:
◊ Characterization of the household and the housing.
◊ Characterization of the respondent.
◊ Violence in the school context.
◊ Violence in the workplace.
◊ Violence in the community sphere.
◊ Obstetric violence.
◊ Intimate partner violence.
The survey considers specific forms of violence, such as having been kicked
or hit with a fist, being attacked with a weapon, being a victim of sexual
harassment, offensive comments, pinching, pulling of hair, slapping, touch-
ing, and other incidents of violence.
47
The results of ENDIREH, in its 2016 version, indicate that:
◊ 66.1% of women over 15 years old have suffered incidents of vio-

lence by any aggressor.
◊ 43.9% of women reported having suffered incidents of violence by

their current or last partner, husband, or boyfriend.
◊ 38.7% of those surveyed were victims of acts of violence in public

spaces (streets, parks, transport).
◊ 26.6% of working women have experienced violence in the workplace.
◊ 25.3% of the women who attended school faced situations of

violence.
Source: http://www.beta.inegi.org.mx/proyectos/enchogares/especiales/endireh/2016/
Module 3:
National Survey on Violence Based on Gender and Generations - Uruguay

This survey was applied in Uruguay, by the National Advisory Council to
Combat Domestic Violence and the Integrated System for the Protection of
Children and Adolescents against Violence, in 2013. The instrument offers
information on violence against women in the domestic, educational, labor,
and social spheres. It was implemented on women over 15 years old residing
in urban areas. All five regions of Uruguay were covered. The objectives of the
survey were:
◊ To generate information on the incidence and severity of violent behav-

ior in relationships.
◊ To quantify the different types of violence (psychological, physical, pat-

rimonial and sexual) as well as their frequency.
◊ To generate specific information about the magnitude, specific char-

acteristics and frequency of violence experienced by girls and older
women.
◊ To generate information about the sociodemographic profile of the

aggressors and the victims.
Based on the data gathered by the survey, it can be observed that 68.8% of
Uruguayan women, aged 15 or older, have experienced a situation of gender
violence throughout their lives. This prevalence is even greater in the case of
large cities such as Montevideo (76.9%). The National Council expects this
information will promote the design of more effective social policies in the
field of the prevention of gender-based violence and assistance to victims.
For more information, consult the Results Report of the First National Prevalence Survey on Violence,
based on Gender and Generations, p. 23
http://www.mysu.org.uy/wp-content/uploads/2015/08/PENPVBBG.pdf#page=9&zoom=auto,-107,713
49
MEASUREMENT OF FEMINICIDE IN LATIN AMERICA

AND THE CARIBBEAN
According to the study of Matthias Nowak (2012), 14 out of the 25 countries with the
highest rates of feminicides (also known as femicides) in the world are Latin American
countries. This manifestation of violence has therefore been considered a key indicator
for analyzing the magnitude and forms in which this phenomenon is expressed in the
countries of the region.
Although in most of the legislation of the region the concept of “femicide” is understood
as the murder of a woman in the context of a couple’s relationship, this definition differs
depending on the regulatory framework of each country. For example, while in Chile only
feminicide is recognized when it occurs in the context of marriage or formal cohabitation
(courtship is excluded), in Peru this crime is recognized in cases of homicide that occurred
in the context of intrafamily violence. According to Nowak (2012), in order to confront
the phenomenon with more effective public policies, it is necessary to first establish a
consensual definition at the regional level and, second, to make progress in addressing
the lack of registers and/or the poor quality of the existing data within each country.
Although only 15 countries in the region had laws on “feminicide” by 2014, the Gender
Equality Observatory in Latin America and the Caribbean (http://oig.cepal.org/es)
gathered information from the administrative records of the police, judicial and health
authorities of all countries in the region, concluding that at least 1,678 women were
murdered for gender reasons in that period.
Although the Observatory’s effort to account for these cases is remarkable, it is clear
that additional efforts are needed to put the problem at the top of the priorities to be
addressed by security policies, for which it is not only necessary to improve legislation
and public policies, but also to improve the mechanisms for its standardized, periodic,
comparable and public access measurement.
Module 3:
Figure 4. Amount (frequency) of Femicides in Latin America and the

Caribbean (2014)
(Gender Equality Observatory in Latin America and the Caribbean)
As shown in the graph,

CARIBBEAN TOTAL DEATHS OF WOMEN
femicides are not evenly
COUNTRIES
distributed among the
CAUSED BY THEIR PARTNER
OR EX-PARTNER
Jamaica (2015) 9
Trinidad and Tabago (2015) 5
México a
Santa Lucia (2015)
Barbados (2014)
1
2
countries of the region.
2 289
In 2014, Mexico was the
St. Vincent and the Granadines (2013) 1
Guatemala Granada (2013) 1
217 Suriname (2014) 7
Honduras
Dominica (2014) 1
country with the highest
531
Dominican Republic
188
number of femicides,
Puerto Rico
44
with 2,389, followed by
El Salvador
183
Panamá
26 Honduras, El Salvador,
Nicaragua
36 Costa Rica
the Dominican Republic
14
Colombia
Venezuela (Rep.Bol.de) and Guatemala.
74
145
Brasil (Sin información oficial) However, as the Central
COUNTRIES WITH THE
Ecuador
97
American countries have
HIGHEST RATE OF
FEMINICIDES IN THE REGION
a smaller population,
Perú
90 they have higher rates
HONDURAS than those of Mexico.
EL SALVADOR Bolivia (Est.Plur. de)b On the contrary, coun-
96
tries such as Uruguay
Paraguay
DOMINICAN REPUBLIC 32 NUMBER OF
GUATEMALA Uruguay FEMINICIDES (24 cases) and Costa
24
Chile
40 2014 Rica (14 cases) have the
lowest number of femi-
Argentina Feminicides
225 Female deaths apparently
with presumption by homicide
cide cases in the region.
Source: ECLAC, 2014
https://www.cepal.org/sites/default/files/infographic/files/3_feminicidio_final_20_oct_2016_correcta.pdf
* Mexico’s number is not comparable with the other countries in the region.
* Brazil’s number corresponds to 2015
4.5. CONSEQUENCES OF CRIME AND VIOLENCE
We have already analyzed the subjects of investigation that refer to committing crimes
or violent acts and the actors involved in them. These subjects deal with the objec-
tive aspects of citizen security, which correspond to the levels of violence and crime.
However, as we have seen, there is also another dimension to the problem of citizen
security: subjective security. This considers “both the emotions and the representations
and behaviors of the public towards insecurity” (Otamendi, 2016: 62).
51
Violence and crime have many consequences for people, ranging from the effects on
their behaviors and freedoms, how they relate to others (known and unknown people),
how they use public spaces, even the way in which state institutions are evaluated. Also,
beyond the effects on people and households, violence and crime have effects on the
economy of a country and even on democratic governance.
The epidemic situation of violence and crime in Latin America leads to the consequent
perception that public institutions are not effective which results in low confidence in
the institutions, particularly in the police and justice. As a consequence of this situation
of low confidence in the police, there is a deterioration in the relations between these
institutions and the community, making dialogue between both parties difficult, reduc-
ing the rates of reported cases and making police work more difficult.
The paradox between objective and subjective security in Latin America is that indica-
tors for both are not directly correlated. As an example, homicide rates for each country
are not directly correlated with the perception of security in those countries.
PERCEPTION OF INSECURITY OR FEAR OF CRIME
There are multiple definitions of fear of crime, but we can understand it as the emotional
response to the possibility of being a victim of crimes or violent acts. This is one of the
subjective aspects of insecurity. Fear restricts activities in public spaces and social activ-
ities, can cause psychological problems, changes in commercial activities and corruption.
This concept has been the subject of study since the seventies, thanks to the implemen-
tation of victimization surveys.
In Latin America, based on the information provided by the victimization surveys, it has
been observed that victimization is not perfectly correlated with the fear of crime. In
other words, inasmuch as victimization decreases, the fear of crime does not necessar-
ily diminish. This is clearly illustrated in the results of the Americas Barometer 2013, a
public opinion survey that includes questions about victimization and fear, as indicated
above.
Module 3:
Figure 5. Perception of Insecurity in Relation to Victimization and Intentions

to Migrate. Survey of the Americas Barometer (2013)
Intentions to migrate, victimization per crime and
perception of insecurity in Central America, 2012
El Salvador
Intentions to migrate 23,5

Perception of insecurity 42,5
Victimization per crime 17,4
Intentions to migrate, victimization per crime and
Nicaragua
perception of insecurity in Central America
40
35 Guatemala

Percentage (%)

30
25 Honduras
20
Victimization per crime
18,9
15 Costa Rica
2004 2006 2008 2010 2012 Intentions to migrate 10,1
Intentions to migrate Perception of insecurity Victimization per crime Victimization per crime 17,5
Source: © Barómetro de las Américas por LAPOP
Panama

0 10 20 30 40 50
95% conidence interval (incorporated design effect)

Source: © Barómetro de las Américas por LAPOP, 2012
This graph shows the evolution in time of This graph shows the levels of victimization
both the victimization and the perception and perception of insecurity in six coun-
of insecurity, indicating that between tries of Central America. This allows us to
2004 and 2008 the decrease in the per- observe that those countries with higher
ception of insecurity was not associated levels of perception of insecurity are not
with a decrease in victimization levels. necessarily those with the highest victimiza-
tion rates.
Source: LAPOP (2014: 2-3). http://www.vanderbilt.edu/lapop/insights/IO901es.pdf
COSTS OF CRIME AND VIOLENCE
Knowledge about the costs of crime allows for the implementation of public security pol-
icies with measurable results based on their economic value. The costs of crime can be
classified into two types: those detectable for measuring and being monetarily calculated,
53
and those that cannot be estimated this way. Isolated studies analyze this problem in Latin
America. As an example, UNDP (2013) analyzed the costs of crime in two ways, it first
calculates the accounting costs by estimating the monetary expenses directly associated
with insecurity, among which are the losses and damages associated with the crime and
the costs of containment, attention and prevention. Second, it estimates the years of life
lost due to homicides and their impact on Gross Domestic Product (GDP).
In Uruguay, the Economic Research Center carried out a study of this type in 2013, which
estimates the costs of crime prevention and the justice system, the monetary value of
stolen goods, health costs and the costs of deprivation of liberty. Based on this analysis,
the study concludes that the cost of crime in Uruguay rises to 3.96% of GDP, with homi-
cides and domestic violence representing the highest costs.
Table 9. Accounting Costs of Crime
Anticipation Direct consequences Indirect consequences

Stolen or damaged property
Police officers
Expenditure on security Emotional and psychological
impact Public prosecutor’s office
Expenditure on insurance
Health services Prisons
Preventive expenditure Victim support expenditure
Magistrates and courts
Reparations to victims
Expenditure on crime
Legal defense costs
prevention Decrease in the accumulation
of physical capital, human Research costs
Expenditure on private capital and social capital
security Cost of rehabilitation and
Deterioration of the physical
reintegration
infrastructure
Sources: Cafferata (2011), Ospina and Giménez (2009), Brand and Price (2000), systematized by UNDP (2013: 102).
http://www.undp.org/content/dam/rblac/img/IDH/IDH-AL%20Informe%20completo.pdf
Module 3:
5. EVALUATION OF CITIZEN
SECURITY POLICIES
As acknowledged by authors such as Frühling (2012) and Sherman (2012), to ensure bet-
ter design and implementation of citizen security programs, particularly in the field of
crime prevention, it is necessary to have evidence on the effectiveness of interventions.
An evaluation corresponds to the systematic analysis based on the social research

methodologies, activities, components and results of a policy, program or project. The
evaluation of social policies and programs should be considered as a part of the process
of executing a program or project, insofar as it is a strategic tool to evaluate the process
of designing and implementing a policy, as well as the results of the intervention. .
“To evaluate is to order and measure the reality, to organize it into categories
or variables that are then measured, either quantitatively or qualitatively. In
this sense, to evaluate is also to investigate a dynamic exercise that goes from
the theoretical-deductive (generation of categories or variables from existing
knowledge) to the empirical-inductive (quantification and qualification of cate-
gories or variables from data collected)” (Tocornal et al., 2005: 3).
55
EVALUATION FUNCTIONS AND TYPOLOGIES
Those who make decisions require quality information to ensure that their actions are
adequate and that, based on these, the expected objective will be achieved. This require-
ment is solved through the evaluation of social policies or programs, which will enable
planning, designing and implementing policies and programs that reach, the problem of
intervention effectively and efficiently. With evaluation strategies it is possible to imple-
ment programs of greater coherence based on evidence, which are sustainable in the
long term and more effective.
Possible questions that guide an evaluation (Olavarría, 2007: 11):
◊ What are the problems requiring new interventions, or expanding or

modifying the existing social programs? What is the nature and scope
of these social problems? Where are they located and who do they
affect?
◊ What social interventions would significantly reduce the social

problem?
◊ What is the appropriate target population for a particular interven-

tion? Does the intervention reach its target population?
◊ Is the intervention implemented in the intended manner? Is the inter-

vention effective?
◊ What is the cost of it? How are the costs related to the effectiveness of
the program and its benefits?
◊ What results and impacts has the intervention produced?
Although the aims of the evaluation can be multiple, the following table highlights the
main purposes from the point of view of their usefulness, feedback or improvement in
the programs or projects.
Module 3:
Table 10. Function or Contribution of Evaluation
Function Description
To choose between different options or possible solu-
tions, allocating resources to those that are more effi-
Decisions on public policy cient and effective.
regarding the implementation
To establish the position of the actors in the debate that
of a program will revolve around in the program announce a possible
vote in the National Congress, in the Communal Council
or other collegiate body.
During implementation, the evaluation monitors the
processes involved in the programs and guides efforts
to improve them.
Rectification or modifications
Given that the evaluations provide their findings in a
within the program (learning from
documented form and often identify the causes of the
the implementers) problems that are affecting their performance, the
results of the evaluation studies are used by the people
who formulate the public policy to implement rectifica-
tions or reforms in the programs.
An evaluation at the end of a program will give an
account of the degree of compliance with the objec-
Decisions on the continuity,
tives, the effectiveness of it, and possible reasons for
termination or replicability of a
its success and/or failure. These elements will help to
program
decide if the program should be maintained, modified or
eliminated, and if it can be replicated in other contexts.
The information provided by the evaluation of pro-
grams will allow the authorities, citizens and represen-
Legitimization and accountability tatives to learn about how the interventions have been
towards the community implemented, if they reach their target population, and
the impact, results, benefits and effectiveness achieved
by them.
Source: Own elaboration based on Manzano and Mellado, 2016
The contents presented below are structured based on one of the possible classifica-
tions of the evaluation typologies, which is related to the time of the project in which it
is implemented, added to the main purposes of these evaluations. In this way, the text
is organized into the ex ante evaluation (design, diagnosis), and the intermediate (pro-
cesses) and ex post (results and impact) evaluations.
57
Table 11. Types of Evaluations
Project Evaluation
Definition Considerations
stage type
Analyze the coherence ◊ Identification, description

and the context in and conceptualization of the
Design which the intervention problem.
evaluation will take place and the ◊ Analysis of the consistency
internal organization of between the problem and the
the program. design of the intervention.
Ex ante Establishes the rela- ◊ Cost-benefit or cost-effective-

evaluation tionship between the ness evaluation.
Economic
(before the expected results and ◊ Identifies the intervention option
evaluation
start of the the resources invested that maximizes results, minimiz-
intervention) in the program. ing costs..
It describes in detail ◊ It leads to better knowledge

the problem situation about the problem, its causes
Diagnostic which the interven- and consequences in the target
evaluation tion will be applied to, population.
in order to define a ◊ It defines the baseline for a sub-
baseline.. sequent impact evaluation.
◊ It evaluates to what extent the
work processes implemented
within the framework of the pro-
gram allow the products to be
Mid-term delivered and reach the estab-
Activity of internal
evaluation lished objectives.
analysis of the pro-
(during the Processes
cesses carried out ◊ It checks the progress status,
implemen- evaluation
during the execution of identifies critical areas and situ-
tation of the
the project or program. ations, corrects problems and, if
intervention)
necessary, changes course.
◊ Allows for decision-making on
implementing appropriate and
necessary rectifications.
Module 3:
Establishes the rela- ◊ Cost-benefit evaluations and

tionship between the cost-effectiveness evaluation.
Economic
obtained results and
evaluation ◊ Analyze the relationship of costs
the resources invested
in the program. and results.
◊ It compares the goals which the

project or program was expected
It identifies the degree to achieve with the indicators that
Ex post of compliance with express the degree of effective
Results compliance or results achieved in
evaluation the objectives estab-
evaluation reality by the intervention.
(concluded lished for the project or
the execu- program. ◊ It identifies the lessons and learn-
tion of the ing obtained from the project or
intervention) program.
◊ It considers what would have
happened if the project or pro-
It aims to identify
gram had not been implemented.
which of the changes
Impact
that occurred are ◊ It analyzes the expected and
evaluation
associated with the unforeseen impact
intervention.
◊ It identifies learning about what
works and what does not.
Source: Own elaboration based on Tocornal et al. (2005) and Olavarría (2007)
EVALUATION OF THE DESIGN, PROCESSES AND RESULTS OF

A PROGRAM IN PANAMA
For example, the program “Improving citizen security in Panama: towards the social con-
struction of a culture of peace”, with the general objective of strengthening the installed
capacities for the management of public policies on citizen security (Arbulú, 2013) has
several types of evaluations. Between 2012 and 2013 the MDG Achievement Fund
carried out a comprehensive evaluation of the program aiming to establish the extent
to which the activities had been adequately executed, the products delivered, and the
expected results obtained. To accomplish this, three types of evaluation were devel-
oped: design, processes and results. The information-gathering techniques developed
for the evaluation were: documentary analysis, bilateral interviews, focus groups and
semi-structured interviews.
59
In the design evaluation, the relevance of the program is established. In other words, it
determines the extent to which the program is consistent with the defined objectives.
The aforesaid report identified the following weaknesses of the program in terms of its
design:
◊ Problems in the proposed indicators.
◊ Lack of definition of intersectoral work.
◊ Programming problems.
◊ Insufficient human resources.
◊ Insufficient national diagnosis.
The process evaluation (also known as monitoring evaluation) determines to what

extent the operation of the program is efficient and how far the management model
brings results. In this area, the report concluded:
◊ The stakeholders involved are coordinated, with their roles defined appropriately.
◊ The financial returns did not have the required regularity.
◊ The programmed activities are carried out: diplomas, research, institutional

strengthening and community participation, care for young people in conflict with
the law.
◊ The administrative processes of the program were an obstacle to the expeditious

functioning of the activities and the coordination of the actors.
◊ There is no internal monitoring system that adequately monitors program

activities.
The evaluation of results establishes the extent to which the objectives have been
achieved (effectiveness) and the probability of maintaining the benefits of the program
in the long term (sustainability). The results of the evaluation in this area revealed the
following:
◊ A citizen security observatory is created enabling the strengthening of informa-

tion needs on the reality of crime in Panama.
◊ Seminars, training activities, manuals and guides are undertaken to improve the
Module 3:
capacities of the involved stakeholders in the management of citizen security.
◊ Prevention, training and institutional capacity building activities are carried out on
issues related to children and youth.
◊ Institutions are strengthened at the national and municipal levels.
◊ The definition of a joint work strategy is missing.
◊ A strategy focused on rights and gender perspective is implemented.
Finally, the evaluation process identifies a set of good practices of the program, includ-
ing: a) the key role of intersectoral coordination, b) empowerment of local governments,
c) promotion of community participation, d) association of the program with private sec-
tor institutions. (For more information, consult Arbulú, 2013)
61
6. METHODOLOGIES AND INDICATORS FOR

THE EVALUATION OF CITIZEN SECURITY
PROGRAMS
For the purposes of this course, we have defined four key evaluation processes in the
implementation of citizen security policies or programs which we will explore below: ex
ante evaluation (diagnosis, design evaluation), process evaluation or monitoring, evalua-
tion of results and impact evaluation.
6.1. EX ANTE EVALUATION
As we have seen, there are two basic types of ex ante evaluation: the security diagnosis
and the design evaluation. The economic evaluation is also a type of ex ante evaluation,
but it will not be explored in this module since it is outside the scope of the course.
DIAGNOSIS OF CITIZEN SECURITY
Every project or social program starts with the following questions: What is the social
problem? Is a social condition really a problem? What are the characteristics of this
problem? What is the seriousness of the problem? All these questions must be resolved
with an appropriate diagnosis of the issue that will be intervened, one of the main tools
for the design and implementation of effective public policies. Therefore, the diagnostic
phase should be considered as a foundational part of the intervention process.
Developing a social diagnosis involves the collection, processing and systematization of

information in order to understand a specific phenomenon in its context. This implies
analyzing its causes, its evolution, its conditioning and risk factors, and its trends (Aguilar
and Ander-Egg, 2001).
A diagnosis must analyze the magnitude, manifestations and characteristics of a phe-

nomenon, considering both objective and subjective components. To do this, qualitative
Module 3:
and quantitative analysis tools must be used. The in-depth knowledge of the problem
allows us to define priorities in the intervention strategies, their viability, the available
means and the actors involved.
There are several topics that we can analyze during the diagnostic evaluation, as summa-
rized in the table below:
Table 12. Issues to Analyze in the Framework of a Diagnosis in Citizen Security
Diagnosis of situations of shortage, deprivation, unmet

Definition of the problem to be needs, conflict, needs for change, social demands, con-
intervened with sidering the context and the population in which they
are observed.
Identification of causal factors
Review of the causes of the problem to guide the inter-
or determinants, conditioning
vention towards those aspects that bring a solution.
factors and risk factors
Estimation of the evolution of the problem or conditions
Forecast of the situation
and how they will be manifested in the future.
Identification of resources and Analysis of human, institutional, economic, legal and
means of action communication resources available for the intervention.
Determination of priorities, in
Prioritization of some problems with respect to others
relation to the needs and prob-
from the diagnostic analysis.
lems detected
Definition of actions that the intervention will carry out,
Establishment of action strategies
considering the elements already analyzed.
Review various scenarios, foreseeing difficulties, facili-
Contingency analysis
tating and impeding elements.
Source: Own elaboration from Araya (2009)
For example, in Chile, the Communal Plan for Public Security Program of the Under sec-
retariat of Crime Prevention (SPD, according to its Spanish acronym) of the Ministry
of the Interior and Public Security aims to promote planning and management capaci-
ties in the field of security in the municipalities. Its purpose is to identify and prioritize
the security problems that most afflict the inhabitants of each community and, in this
way, establish a plan to meet these needs; the community teams have to develop citizen
security diagnoses. In accordance with the technical guidelines provided by the SPD, the
diagnoses have the following main objectives:
63
◊ To characterize the context of the security situation in the community.
◊ To identify the problems of citizen security that exist in the community.
◊ Incorporate the opinion of citizens and local stakeholders to gain a comprehensive

vision of the main problems of citizen security.
◊ Identify the resources available to the community to respond to security problems.
◊ Prioritize and organize those problems into a hierarchy in order to define the
objectives of the Communal Public Security Plan.
◊ To understand and describe the main situations or phenomena that generate per-
ception of insecurity among citizens.
Complementing qualitative and quantitative methodology, the communal teams collect

information about these issues and based on this they define a set of community needs,
prioritizing those problems of greater relevance. In this way, a range of programs suitable
for these problems is designed, focusing on those areas of greater criminal complexity.
6.2. DESIGN EVALUATION AND THE USE OF THE

LOGICAL FRAMEWORK TOOL
Another type of ex ante evaluation is the design evaluation, which consists in the assess-
ment of the formulation of the intervention project. According to Olavarría (2007),
before implementing the project, the design evaluation analyzes if it can effectively
address the social phenomenon which it is addressing.
This type of analysis takes into account the coherence between the problem identified
from the diagnosis, the population to be intervened with, the objectives, the activities
and their organization, and the planning of the processes. Additionally, the viability of
the intervention can be estimated on the basis of a design evaluation.
One of the main design evaluation tools is Logical Framework. The Logical Framework
approach consists in the application of an analysis matrix that makes it possible to for-
mulate a project, as well as identify and review its constituent elements and coherence.
Module 3:
From this analysis, identification and prioritization of problems, formulation and evalu-
ation are facilitated ex ante while operational planning, monitoring and evaluation, and
evaluation are facilitated ex post. The aim of the Logical Framework matrix is to structure
the planning process, clearly explaining the following elements:
◊ Its purpose, aims, components and activities (rows).
◊ Intervention strategy, indicators, means of verification and assumptions (columns).
INTERVENTION MEANS OF
INDICATORS ASSUMPTIONS
STRATEGY VERIFICATION
Purpose Purpose indicators
Aim Aim indicators

Monitoring and
Context
Components Component indicators evaluation system
Activity indicators
Activities
Budget
The analysis of the information provided in the logical framework matrix is based on a
double logic: horizontal and vertical. A project or program must meet both criteria to
confirm that its design is consistent.
◊ Vertical logic of the Logical Framework: analyzes the cause-effect relationships of

the elements arranged in the columns. That is to say, if the component is effectively
achieved from the activities, if the purpose is really achieved from the component
and, finally, if the objective is reached through all these elements.
◊ Horizontal logic of the Logical Framework: This evaluates whether the proposed
indicators adequately measure the results obtained at purpose, components and
activities levels.
65
Logical Framework of the Project: Cauce Ciudadano-Cure Violence in

Ecatepec, State of Mexico.
In Mexico, the municipality of Ecatepec, Morelos, is an area where territorial

disputes between rival groups and gangs occur, which has deteriorated the
social fabric. The case of Ciudad Cuauhtémoc particularly stands out, which
is an area populated by approximately 200,000 people, where criminal vio-
lence has considerably increased, especially homicides. The work of Cauce
Ciudadano and Cure Violence focuses on secondary and tertiary prevention,
which has allowed the projects to contribute to caring for the damage and
the rehabilitation of those people who have been victims and victimizers in
high-risk communities. Both organizations conceptualize violence as a public
health problem, for which they seek to implement methodologies to stop the
“contagion”.
Module 3:
Table 13. Cauce Ciudadano-Cure Violence in Ecatepec Logical Model
Resources Activities Products Results Impact
◊ Interrupters/ Mediation ◊ Number of prison ◊ Decrease of ◊ Contribute to

Mediators visits violence nor- the decrease in
◊ Personnel selection
◊ Facilitators ◊ Number of malization in the risk perception
and recruitment community
◊ Neutral physical workshops ◊ Decrease of
◊ Mapping of actors ◊ Decrease of violet acts:
space (base) ◊ Number of
◊ Community diagnosis activities violent behav- shootouts, fire-
◊ Computers iors: self-care, arms, knives,
◊ Training (initial/update) ◊ Number of life expectations, arrests
◊ Basic office
Equipment (sta- ◊ Territory identification trained abilities to solve ◊ Decrease in
tionary, water, ◊ Interrupters/ problems and the percep-
coffee) ◊ Walk the streets conflicts
facilitators tion of violent
◊ Transportation ◊ Elaboration of daily ◊ Number of ◊ Activities with behaviors in
resource reports (interrupters/ trainings high-risk youth the community
facilitators) ◊ Solved conflicts (schools, pris-
◊ Telephone ◊ Number of ons, church,
(credit) ◊ Visit to prisons processes ◊ Identification of business, lead-
◊ Logistical Interruption ◊ Number or transformation ers, community
expenses alternatives in leaders, busi-
◊ Personnel selection ◊ Intermediations/ the community nesspeople)
◊ Training equip- and recruitment mediations (interrupters´
ment, training and facilitators´ ◊ Development
◊ Mapping of actors ◊ Number of man- of abilities to
◊ Supervisor/ agements (ser- work)
◊ Community diagnosis make a living
Field Coord vices, programs,
◊ Program coord ◊ Training (initial/update) funeral home)
◊ Financing ◊ Territory identification

◊ Publication ◊ Walk the streets
materials
◊ Elaboration of daily
◊ Data/registers reports
◊ Digital platform
Community activities
◊ Workshop/activity
planning
◊ Weekly coordination
meetings
◊ Elaboration of daily
reports
Management activities
◊ Referrals
◊ Social
◊ Education
◊ Health
◊ Personal
Source: EASE, (2017).
67
For more information regarding the logical framework, consult [The Logical Framework
Matrix for Formulating and Evaluating Projects] (optional reading).
6.3. PROCESSES EVALUATION
The processes evaluation, also known as monitoring or follow-up evaluation, seeks

to ascertain whether the intervention is carried out according to the design and in an
appropriate manner, and if it reaches certain objectives. It consists in the systematic
evaluation of the processes under development during the intervention, the coverage
of the project or program, the consistency of the activities with the design and the eco-
nomic management (Tocornal et al., 2005; Straits and Singleton, 2011).
Based on this analysis, public policy makers and decision makers can evaluate the devel-
opment of their activity, the structure of its components, the dynamics it generates,
among other issues relevant to achieving the objective of the intervention. The review of
the processes involved in the implementation of a citizen security program allows public
policy makers to:
◊ Determine which program components contribute to the intended objective.
◊ Identify difficulties in programming, organization and operation of the program.
◊ Evaluate the efficiency in terms of program costs.
◊ Generating learning about the program procedures.
◊ Reprogram intermediate processes.
◊ Provide feedback to the work of the involved actors.
Process evaluations can be carried out using qualitative and quantitative methods, the
main components of which are: a) comparison of the operation of the project in relation
to the design, b) description and analysis of activities, components and processes and c)
recommendations to improve the intervention (Manzano and Mellado, 2016). From this
and other information, process indicators to monitor the activities, their beneficiaries
and intermediate products of the project or program can be constructed.
Module 3:
Process indicators are aimed at analyzing the specific activities in order to meet the
objectives of the security plan or program.
Process indicators:
◊ Size of the target population (coverage).
◊ Number of community assemblies held.
◊ Number of program activities.
◊ Number of professionals in the program.
◊ Number of benefits.
◊ Number of derivations.
◊ Profile of the professionals in charge of the initiative.
Source: Manzano and Mellado, 2016

69
The Office of the High Commissioner of the United Nations for Human Rights
in Mexico (OHCHR) and the Ministry of Public Security of Mexico City (SSPDF
according to its Spanish acronym) (2014), propose a set of indicators on citizen
security and human rights. The document states that indicators “enable the evalu-
ation of the way in which a state complies with its obligations” (OHCHR and SSPDF,
2014: 23), with those processes based on budgetary allocations, coverage of cer-
tain population groups, complaints regarding reparation, the proportion of human
resources destined for citizen security tasks, etc. In particular, on the subject of
human rights, the OHCHR and the SSPDF (2014) propose the following process
indicators:
◊ Number of complaints received by the SSPDF about violations of human rights.
◊ Number of cases which were considered by the Office of the Internal
Controller and the General Directorate of Police Inspection.
◊ Percentage of SSPDF personnel trained in the field of human rights.
◊ Number of human rights training courses given by the SSPDF.
◊ Number of activities and events related to human rights at the SSPDF.
◊ Rate of participation of SSPDF staff in events related to human rights.
◊ Number of SSPDF staff.
◊ Total budget of SSPDF.
◊ Budget allocated to different items.
The Center for Research for Development A.C. (CIDAC, 2016) designed a methodology for
the evaluation of the first responder police, which contains elements of process evaluation.
The methodological proposal is constructed in order to evaluate the performance of the
police as the first authority that takes cognizance of the criminal acts in the accusatory crim-
inal system. The proposed evaluation methodology considers the analysis of the following
sources of information:
◊ Police records: knowledge of criminal act, assistance to people, protection of the place
of crime, inspection of people, evidence processing, detention, interviews.
◊ Surveys on police offices: quality and adequacy of equipment and work tools, police
training, inter-institutional coordination, police performance.
◊ Surveys on public ministries: quality and adequacy of equipment and work tools, train-
ing of police, inter-institutional coordination, police performance.
◊ Evaluation guide on compliance with detention regulations.
Module 3:
6.4. RESULTS EVALUATIONS
The evaluation of results can be defined as “an evaluation of the effects of the immediate
products of the project. The products are closely related to the objectives of the project
since they are precisely those that correspond to the means to achieve the objectives.”
(Tocornal et al., 2005).
The results evaluations are ex post assessments, i.e., they are applied once the imple-
mentation phase of the project or program has been completed. This is the most widely
used type of evaluation, insofar as it provides information regarding the results, effec-
tiveness, efficiency and benefits of an intervention, generating learning that can be
applied to the design of new programs.
To perform the required analyzes and to determine whether a program has achieved the
planned objective, as well as to examine the relationship between costs and benefits,
quantitative methodologies are traditionally used. These are based on indicators, from
which the expected and obtained results can be compared numerically.
Therefore, result indicators observe the products, services and/or achievements

obtained from the activities undertaken by the actors, which are defined based on some
of the objectives or components of the security program or project.
Examples of Performance Indicators
◊ Number of users successfully graduated from the program.
◊ Number of people who evaluate the intervention as effective.
◊ Level of user satisfaction with the program.
◊ Variation in recidivist population.
◊ Variation in reports of intrafamily violence.
Source: Own elaboration based on Manzano and Mellado, 2016

71
For the comparison between the achieved results and the objectives established in the
design of the program, different sources of information should be used depending on the
objectives, coverage and activities defined in the design of the program. The collection
of information must at least include information on: a) the activities performed, b) use
of resources, c) achievement indicators, d) users’ impressions, community and program
staff (Olavarría, 2007).
Evaluation of the National Program for the Prevention of Crime 2014 in

Chihuahua, Mexico
To illustrate these types of indicators, the evaluation developed by the

Government of Chihuahua regarding the National Crime Prevention Program
(PRONAPRED) 2014 is presented below. The objective is to implement proj-
ects which inhibit the factors that generate violence and delinquency, benefit-
ing direct or indirect victims of violence and delinquency. To provide an account
of the results obtained from the execution of the program, the following indi-
cators are used:
◊ Percentage of projects implemented with PRONAPRED resources.
◊ Percentage of people who benefited from PRONAPRED projects.
◊ Percentage of people who benefited from projects to increase Citizen

Co-responsibility.
◊ Percentage of people who benefited from projects of vulnerability reduc-

tion against violence and crime.
◊ Percentage of people who benefited from the creation of environments

of coexistence and citizen security.
◊ People who benefited through the strengthening of institutional

capacities.
◊ Percentage of concluded agreements.
Source: INTEGRAM (2015)

Module 3:
6.5. IMPACT EVALUATION
An impact evaluation aims to identify the desired and undesired effects on the imple-
mentation of a project or program, to verify whether these changes correspond to the
objectives set out in the original program design and, in particular, to determine if the
observed effects are attributable to the intervention. The analysis of this information
allows us to learn lessons from the intervention, which will help to redirect and refine
new implementations of the program.
It is important to highlight the difference between the evaluation of results and the
impact evaluation. On one hand, the evaluation of results provides information on
changes in the variables that measure the results of an intervention, which are not nec-
essarily due to the program. On the other, the impact evaluation assesses the extent to
which the results are attributable to the intervention.
For example, in a program whose general objective is to reduce the participation of ado-
lescents (between 14 and 18 years old) in violent crimes in a certain territory, the main
lines of intervention have been defined: reintegration of adolescents into the school sys-
tem, training of leaders in mediating conflicts and treatment and reintegration in cases of
abuse or dependence on drugs or alcohol. The results evaluation of this program would
analyze:
◊ The performance of the activities and services established for each line of
intervention.
◊ The achievement of the program regarding: number of young people who re-en-
tered the school system, number of leaders trained as mediators and number of
young people who paid for an abusive drug use habit.
◊ Achievement of the overall objective of the program: decrease in the percentage

of young people who have been arrested for violent crimes in the territory, com-
pared to the situation prior to the program.
The impact evaluation of the same program would only seek to determine compliance
with the general objective of the program within the target group of the intervention
but trying to establish unequivocally that the changes perceived in the target group at
the end of the program are directly attributable to it. To this effect, the evaluation must:
73
◊ Measure the total number of adolescents from 14 to 18 years old, and the propor-
tion of those who have committed crimes within the territory to be intervened.
◊ Randomly assign the adolescents into two groups, one as a control group and the
other as an experimental group. In each group, the personal and family charac-
teristics of young people should be measured, as well as variables related to their
schooling, work-life balance or conflict resolution skills, and substance use.
◊ During the program, a detailed follow-up should be made of each group, recording
the changes, particularly with respect to the key variables.
◊ At the end of the program, both groups should be measured again in relation to the
variable of participation in violent crimes, and to the other key variables.
The questions that guide the impact evaluation are: To what extent did the project affect
the beneficiaries? Were any of the improvements the direct result of the project or
would it have occurred anyway? Also, the results of the impact evaluation are useful to
establish the extent to which program costs were justified (Baker, 2000).
The answers to the formulated questions in an impact evaluation require an analysis

that establishes causality between the intervention and the results. To establish these
types of relationships experimental or quasi-experimental models are employed, using
mainly quantitative research methods. For this type of analysis, it is necessary to define
control groups (without intervention), whose results are compared with those of the
group that have received the intervention, the treatment group.
Once the groups are established, the researchers manipulate one or more independent
variables to observe how these affect a dependent variable. Examples of dependent vari-
ables are the impact indicators which include: rate of complaints, perception of insecu-
rity, homicide rate, victimization due to family violence. The independent variables are
those that explain the behavior of the dependent variables, for example: age, schooling,
place of residence, criminal record, etc.
In an evaluation with experimental design, the groups are randomly selected from a
sample of eligible individuals or intervention units eligible for the benefits of the program.
Module 3:
Impact Evaluation of the Program: “USAID Community-Based Strategy

for the Prevention of Crime and Violence in Central America: El Salvador,
Guatemala, Honduras, and Panama”.
This evaluation was implemented by a team from the Americas Barometer in

order to determine the effectiveness of community-based violence prevention
in contrast to traditional hardline strategies. This evaluation was implemented
in four Central American countries (El Salvador, Guatemala, Honduras, and
Panama), using both quantitative and qualitative techniques for the collection of
information. The following activities were defined within the intervention model:
◊ Planning of committees at municipal level.
◊ Crime observatories and data collection.
◊ Prevention of crime based on environmental design.
◊ Programs for young people at risk.
◊ Community police.
The design of the study considered a randomized experiment conducted by
conglomerates, which consists in the application of the program in selected
neighborhoods, while also selecting neighborhoods of similar characteristics
at the same time to serve as control groups. From each neighborhood repre-
sentative samples of cases are taken that make it possible to make the com-
parison between the groups. From the comparative analysis of groups, the
following was concluded:
◊ The residents from the neighborhoods with interventions know that
the following exists:
19% fewer thefts.
25% decrease in the sale of illegal drugs.
51% fewer murders.
◊ Residents of neighborhoods with interventions show 11% less fear than
residents of neighborhoods without interventions.
◊ Residents of neighborhoods with interventions have confidence levels
in the police that are 9% higher than those residents of neighborhoods
without intervention.
For more information on this impact assessment, consult Impact Evaluation of
USAID’s Community-Based Crime and Violence Prevention Approach in Central
America: Regional Report for El Salvador, Guatemala, Honduras and Panama].
(Berk-Seligson et al., 2014) (optional reading).
75
An alternative to experimental designs are the quasi-experimental designs with more

flexible methodologies of application. These propose to replicate experiments with sta-
tistical tools. In other words, the selection of control and treatment groups is not ran-
dom, but groups with similar characteristics are constructed.
Impact Evaluation of a Social Inclusion and Student Violence Prevention

Program
Through a quasi-experimental design, the impact of the “Together for a

Community without Violence” program was measured. The latter was carried
out in educational establishments located in districts at risk in the Republic of
Panama. The aim of the program is to contribute to the prevention of student
violence and increase the levels of social inclusion of young people by promot-
ing and strengthening attitudes and values for personal growth and peaceful
coexistence in the student milieu and community.
The treatment group is made up of 43 schools intervened in 2011, 2012 and

2013: Colón (10), David (4), Panamá Centro (21) and San Miguelito (8). The con-
trol group consists of 27 schools that were not part of the project but located in
the same districts: Colon (5), David (5), Central Panama (14) and San Miguelito
(3). The fact that the schools in the control group are from the same districts as
the schools in the treatment group is essential to ensure comparability between
groups.
It was found that the program decreased the rate of pregnancies in the student
population, mainly in eighth, eleventh and twelfth grade students.
Source: Gibbons M. & M. Rossi (2015)
Indicators that measure the specific progress of the agreed targets and the changes
resulting from the intervention must be constructed to evaluate the impact of the inter-
ventions. In this sense, the impact indicators seek to measure the effect of a policy or
program on an issue.
Module 3:
For example, if the objective were to reduce crime in a given territory, one possible indi-
cator is the variation in complaints over the last year (Mohor, 2007). Since they compare
the current situation with the initial situation, baseline information and a measurement
to the evaluation date are required prior to the intervention of the program.
In Latin America, impact evaluations are scarce and lack the necessary rigor to obtain
relevant information about what works and what does not (Frühling, 2012). However,
in recent years efforts have been made to evaluate the impact of numerous public secu-
rity programs that are underway in the region. This is the case of the Quadrant Plan of
Carabineros of Chile; the Quadrant Plan of the Colombian National Police; the evalua-
tion of the Fica Vivo (Stay Alive) program in Brazil; the Comprehensive Citizen Security
Management Program in Montevideo, Uruguay; the community-based strategy of
the United States Agency for International Development (USAID) for the prevention
of crime and violence in Central America: El Salvador, Guatemala, Honduras, Panama
(Frühling, 2012).
77
Generation of Evidence
For almost a decade, IDB has been developing various strategies in order to
strengthen the evaluation of security programs and to promote the designing
of programs based on evidence. Therefore, as an example, it has supported the
efforts of academics to systematize programs evaluated in areas such as: fam-
ily and school (Hein, 2010); intrafamily violence and child maltreatment (Larraín,
2010); situational prevention (Rau and Prosser, 2010); prevention of violence
through local interventions: multi-agencies, drugs and gangs (Oviedo, 2010);
police (Frühling, 2010); and the criminal justice system (Olavarría y Pantoja,
2010)8. Another program worth highlighting is the “Regional System for Impact
Evaluation of Public Security Policies for Latin America”, which began in 2012
aiming to create a regional network that systematically promotes the perfor-
mance of impact evaluations on citizen security programs with the highest qual-
ity standards in the field in Latin America. The national governments of Mexico, El
Salvador, Honduras, Costa Rica, Uruguay and Chile, the Mayor’s Office of Bogota
(Colombia) and the government of the Province of Cordoba (Argentina) partici-
pate in this program, with the support of the National University of Villa María
and the Center for Studies in Citizen Security of the University of Chile.
Source: Frühling, 2012
8 The texts referenced by Frühling (2012) are the following:

• Frühling, Hugo (2010). “La Policía y el control del delito” “Police and Crime Control”. Programs directed at reducing
crime: A Systematic Review on the Literature. IDB.
• Hein, Andreas (2010). “Family and School Centered Prevention”. Programs directed at reducing crime: A Systematic
Review on the Literature. IDB.
• Larraín, Soledad (2010) “Prevention Program Systematization on Intrafamily Violence and Child Abuse”, Programs
directed at reducing crime: A Systematic Review on the Literature. IDB.
• Olavarría, Mauricio and Rodrigo, Pantoja (2010). “ Criminal Justice System and Crime and Violence Prevention”.
Programs directed at reducing crime: A Systematic Review on the Literature. IDB
• Oviedo, Enrique, “Violence Prevention Programs Through Local Interventions: Multi-agencies, Drugs and Mobs”,
Programs directed at reducing crime: A Systematic Review on the Literature. IDB
• Rau, Macarena and Paola, Prosser (2010). “Evaluation Study of Strategy Impact on Situational Prevention” Programs
directed at reducing crime: A Systematic Review on the Literature. IDB
For more information on standards and evaluation standards suggested by the United Nations, see the document at:
http://www.unevaluation.org/document/detail/1914
Module 3:
TOWARDS BETTER STANDARDS IN CITIZEN SECURITY

ASSESSMENTS
As we have seen throughout this unit, evaluations of security programs in the region are
scarce and are infrequently disseminated. Nevertheless, it is evident that over the last
decade evaluations have increased both in volume and seen improvement in the appli-
cation of better technical standards. In this process, academic centers aimed at promot-
ing public policies in these matters, as well as international cooperation agencies, have
played a key role.
In this context, the United Nations Evaluation Group (UNEG) launched a document
named: “Standards and Evaluation Standards” in 2016. This is an updated version of the
guide developed in 2005, “Evaluation Standards in the United Nations System”, which
was intended to strengthen evaluation practices and become a key reference for evalu-
ators around the world.
79
7. CONCLUSION
Various studies and sources of information highlight the fact that the phenomena of vio-
lence and crime in Latin America have experienced a strong growth in recent decades. In
particular, the high levels and persistence of lethal violence (homicide rate) in the region
have been highlighted, in contrast with the downward trends observed in other parts of
the world.
At the same time, several studies reflect on the fact that the governments and public
institutions of the region have so far not been able to provide effective and far-reaching
solutions to these problems, which is largely due to shortcomings in the design, manage-
ment and evaluation of citizen security and coexistence policies.
In order to enhance public policies in this area, it is claimed that better quality informa-
tion systems must be developed, both administrative records and victimization or secu-
rity surveys. The existence of better information systems would certainly contribute to
the realization of security diagnoses, according to the regional, national and local reality,
as well as improving the management and evaluation of citizen security policies.
Statistical systems of public security, for the process of production, management, anal-
ysis and dissemination of their results, require coordination between institutions and
agencies that are linked together. The information provided by these systems allows
decisions to be made from a technical perspective in interdependent spheres of public
administration.
Considering the complexity of the problems of crime and violence that afflict the region,
such as the ones mentioned above, the evaluation of policies or programs in this area
presents obstacles and challenges greater than those observed in other areas of social
policies.
The multicausality of the phenomena (violence, crime, fear, among others), the diver-
sity of public and private agencies that execute programs in this area and, finally, the
Module 3:
implementation of programs at different levels of government (national, regional and

local), implies greater methodological demands in the formulation of evaluation strate-
gies and indicators. These elements must be coherent with each other and in accordance
with the evaluated program or project in order to obtain better evaluation mechanisms.
As it was seen in the module, the different evaluation models that are implemented fulfill
different purposes and have different characteristics according to the time of its appli-
cation and its objectives.
In this sense, the development of ex ante evaluations for diagnosis facilitate the identi-
fication, characterization and prioritization of problems, as well as the targeting needs
of certain territories/groups. The evaluation of design, on the other hand, analyzes the
coherence between the identified problem, the population which will be intervened with
and the proposed program, its objectives, activities and, in general, the planning of the
processes. The design evaluation also establishes whether the technical and financial
conditions exist for the intervention to be implemented (or its viability). The evaluation
of processes gives an account of the functioning of the programs, the execution of activ-
ities and the budgets involved, enabling policy makers to correct those activities that
mark a departure from the original design and do not show results in the short term.
On the other hand, once the implementation of the program has been completed, the
evaluation of results offers information regarding the correct execution of the activi-
ties, the attainment of products and/or achievements, and the fulfillment of objectives
according to the initially proposed goals. Finally, the evaluation of impact reveals if there
were changes in the target group with respect to the problem situation, and if they are
attributable to the executed program. Both the evaluation of results and the impact
allow us to make decisions about the continuity, term or replicability of a program.
In LAC, the implementation of evaluative strategies is still incipient because they face
multiple obstacles, from political and organizational to technical problems derived from
the lack of access to data/indicators on the subject. As a result of these difficulties, most
of the citizen security programs that have been implemented in the last two to three
decades in the region have had no evaluation which would allow the stakeholders to
establish, in an empirical, rigorous and objective manner, whether such intervention
strategies and actions had an effect on the reduction of the “situation-problem”. Due
to this lack of evidence, most of the programs that are designed are not coherent and
81
do not offer pertinent answers to the security problems that afflict the region, both at
national and local levels.
In spite of this, it is evident that over the last decade these evaluations have increased
both in volume and better technical standards have been applied. This process has been
possible, on the one hand, thanks to the greater emphasis given by national and local
governments towards better management of citizen security and crime prevention poli-
cies, and, on the other hand, due to the improvement of information systems. In this pro-
cess of improving the standards of the region’s evaluations, academic centers aimed at
promoting public policies in these matters, as well as international cooperation agencies,
have played a key role.
Undoubtedly, there is still a long way to go and this requires that managers and public
managers make greater efforts to develop integrated evaluation practices from the out-
set in the formulation of security and crime prevention policies. Only through a more
in-depth and systematic knowledge about which interventions work and which ones do
not, will it be possible to make further progress in the development of public policies
for efficient and effective citizen security, but above all based on a constant process of
learning and feedback.
Module 3:
8. GLOSSARY
Administrative Records – Official: these correspond to the statistical information reg-

istered systematically and periodically by public bodies in order to monitor trends in the
occurrence of social phenomena. In addition, they enable decision-making on the design,
monitoring and evaluation of public policies. Dark (or hidden) figure: term used to refer
to criminal acts that are not known by the system due to lack of reporting.
Data Gathering Instruments - Surveys: standardized instrument for the collection of

information on a research object. The surveys or questionnaires are composed of struc-
tured questions that measure perceptions, attitudes, interests, among other data. Insofar
as this instrument is applied to a representative sample of the population, selected ran-
domly, it allows for the characterization of large segments of the population through the
use of statistical analysis procedures.
Effectiveness: refers to the ability to balance between achieving effectiveness and

efficiency in the implementation of a public program. In other words, to achieve the
expected results through the efficient use of available resources. The cost-effectiveness
evaluation can be carried out to choose between two program alternatives (before the
intervention) or to evaluate the results and impact of a program (ex post).
Efficiency: refers to the execution, management and organization of activities carried

out in such a way that it implies the lowest possible cost to maximize the delivery of
goods and/or services.
Efficacy: refers to the degree of compliance with the objectives of a project, not neces-
sarily measured by the resources allocated for this, but through the percentage of cov-
erage of the target population.
Evidence-Based Programs: those programs that are designed based on the empirical
results of other similar experiences (what works and what does not), that is, through the
systematic collection of observations that answer the question of whether a program or
public policy has achieved the expected results.
83
Georeferencing: spatial positioning methodology that geographically locates different

phenomena through the use of standardized maps and databases. GISs (Geographic
Information Systems) therefore make it possible to analyze the distribution, concentra-
tion, trends and displacement of the object of study within the framework of analysis
units of different scales (streets, neighborhoods, cities, countries).
Indicator: a quantitative expression that reflects a relationship between two dimensions

of a certain phenomenon or reality that is used to measure the achievement of a goal
sought by a project or program. Indicators allow for the comparison of the state of real-
ity, identified by the indicator, at different moments of time, or with realities close to it.
Integrated Information Systems: an organized and interconnected set of statistical

information from different sources that therefore requires a fluid interaction and coor-
dination between the institutions and the involved organizations. Although each record
of information is created for a certain objective, when it is complemented with other
sources, new uses can be applied to it to make complex decisions.
Logical Framework Methodology: management tool designed to facilitate the process of

identification, formulation or design, execution and evaluation of projects. Its objective
is to structure the planning process and allow essential project information to be clearly
communicated (Virtual Training Course on Evaluation of Programs for the Prevention of
Crime and Violence of the CESC).
Results-Based Management : methodology that is part of the New Public Management

(NPM) approach, which proposes ordering the management of public bodies based on
strategic planning, the definition of quantifiable (measurable) results, the evaluation of
performance and the establishment of contractual relationships between public agents.
Statistics: refers to the collection, interpretation and analysis of numerical data that are
presented in an orderly and systematic manner. The purpose of statistics is to describe
numerical information and make inferences regarding the occurrence of certain phe-
nomena, enabling the analysis of high volumes of information.
Social Evaluation: a systematic application of procedures and methods of social research

aimed at establishing a value judgment regarding the design, implementation, efficiency
and effectiveness of a project, program or public policy.
Module 3:
Victimization and perception survey: a quantitative information gathering tool that

seeks to characterize the victims, offenders, context and circumstances of the crimes in
order to get a closer picture of the real levels of criminality (objective insecurity) and to
investigate citizens’ perceptions of the crimes (subjective insecurity).
ABBREVATIONS AND ACRONYMS
Banco Interamericano de Desarrollo - Inter-American

BID- IDB
Development Bank
BJS Oficina de Estadísticas Judiciales – Bureau of Justice Statistics
Banco de Desarrollo de América Latina - Latin American

CAF
Development Bank
Centro de Excelencia para Información Estadística de Gobierno,
Seguridad Pública, Victimización y Justicia - Center of Excellence
CDEUNODC
in Statistical Information on Government, Crime, Victimization and
Justice (CdE)
CEAD Center for Crime Studies and Analysis
Comisión Económica para América Latina y el Caribe - Economic

ECLAC
Commission for Latin America and the Caribbean
CESC Center of Studies in Citizen Security (Chile)
CIDAC Research Center for Development AC (Mexico)
DOJ U.S. Department of Justice
Encuesta Nacional Urbana de Victimización de Chile - Urban

ENUSC
National Survey Victimization of Chile
85
National Survey on the Dynamics of Relationships in Mexican

ENDIREH
Households
Encuesta Nacional de Victimización y Percepción sobre Seguridad
ENVIPE Pública de México - National Survey on the Dynamics of
Household Relationships
Proyecto de Opinión Pública de América Latina de la Universidad
LAPOP de Vanderbilt - Latin American Public Opinion Project of
Vanderbilt University
Instituto de Investigación de Política Criminal - Institute for
ICPR
Criminal Policy Research
Instituto Nacional de Estadísticas de Chile - National Institute of
INE
Statistics of Chile.
Instituto Nacional de Estadística y Geografía de México - National
INEGI
Institute of Statistics and Geography of Mexico
Instituto Nacional de las Mujeres de México - National Institute of
INMUJERES
Women of Mexico
Centro Nacional de Investigación Social de Inglaterra - National
NatCen
Social Research Center of England
NGP New Public Management
Oficina del Alto Comisionado de las Naciones Unidas para

OACNUDH
los Derechos Humanos - Office of the United Nations High
- OHCHR
Commissioner for Human Rights (OHCHR)
Organización de los Estados Americanos - Organization of
OEA - OAS
American States.
WHO World Health Organization
ONU – UN Organización de las Naciones Unidas- United Nations
GDP Producto Interno Bruto - Gross Domestic Product

Module 3:
Programa de las Naciones Unidas para el Desarrollo - United

PNUD-UNDP
Nations Development Programme
Programa Nacional de Prevención del Delito de México - National
PRONAPRED
Program for the Prevention of Crime in Mexico
PSCS Prison Social Climate Survey
GIS Geographic Information Systems
Subsecretaría de Prevención del Delito de Chile - Undersecretary

SPD
of Crime Prevention of Chile
Secretaría de Seguridad Pública del Distrito Federal de México -
SSPDF
Secretariat of Public Security for the Federal District of Mexico
Grupo de Evaluación de las Naciones Unidas - United Nations
UNEG
Evaluation Group
Oficina de Naciones Unidas contra la Droga y el Delito - United
UNODC
Nations Office on Drugs and Crime
Agencia de los Estados Unidos para el Desarrollo Internacional -
USAID
United States Agency for International Development
87
9. BIBLIOGRAPHY
Aguilar, M. and Ander-Egg, E. (2001). Diagnóstico Social. Conceptos y Metodología [Social
Diagnosis. Concepts and Methodology]. Argentina: Grupo Editorial Lumen.
Análisis y Desarrollo Social Consultores. (2010). Guía de evaluación de programas y proyec-
tos sociales [Evaluation of programs and projects guide] . Madrid: Plataforma de ONG
Acción Social [Platform of Social Action NGO].
Araya, J. (2009). Documento de Apoyo Docente: Diagnóstico y Gestión Pública en Políticas de
Seguridad [Educational support document: diagnosis and public management of secu-
rity policies] . Santiago de Chile: Centro de Estudios en Seguridad Ciudadana.
Arbulú, A. (2013). Evaluación final del programa Mejorando la Seguridad Ciudadana en
Panamá: Hacia la Construcción Social de una Cultura de Paz. Fondo para el Logro
de los Objetivos de Desarrollo del Milenio. Naciones Unidas [Final evaluation of
the Improving Citizen Security in Panama program: Towards the social construc-
tion of a culture of peace. Funds
Aromaa, K. (2012). Victimization Survey - What are they Good for? Viktimologija: Teorija,
Praksa, Aktivizam, Jun 2012: 85-94. Serbia.
Arriagada, I. and Godoy, L. (1999). Seguridad ciudadana y violencia en América Latina:
Diagnóstico y políticas en los años noventa [Citizen security and violence in Latin
America: Diagnosis and policies in the 90s]. Serie políticas sociales. Santiago de
Chile. CEPAL. Baker, J. (2000). Evaluación de Impacto de los Proyectos de Desarrollo
en la Pobreza [Evaluation of the impact of poverty development projects] . Washington
D. C.: World Bank.
Beato, C. (2005). Producción y Uso de Informaciones y Diagnósticos en Seguridad Urbana
[Production and use of information and diagnosis of urban security]. Washington D.C.:
World Bank.
Bergman, M., Flom, H., Vilalta, C. and Masello, D. (2014). Informe sobre Encuestas
Penitenciarias en América Latina. Herramientas conceptuales y metodológicas para su
diseño, ejecución, evaluación [Report on Prison surveys in Latin America. Conceptual
and methodological tools for thier design, implementation, evaluation]. Washington
D.C.: Inter-American Development Bank (IDB).
Berk-Seligson S., D. Orcés, G. Pizzolitto, M. Seligson and C. Wilson (2014). [Impact
Evaluation of USAID’s Community-Based Crime and Violence Prevention Approach in
Central America: Regional Report for El Salvador, Guatemala, Honduras and Panama].
Module 3:
Latin American Public Opinion Project (LAPOP) and USAID. Vanderbilt University.
Brookman, F. y Robinson, A. (2012). Violent Crime. In Maguire, M., Morgan, R. and Reiner,
R., The Oxford Handbook of Criminology. Oxford: Oxford University Press: 563-594.
Center of Excellence in Statistical Information on Government, Seguridad Pública,
Victimización y Justicia [Public Security, Victimization and Justice] (CDEUNODC).
(2012). Available at https://cdeunodc.wordpress.com/2012/12/18/encuestas-de-
victimizacion-un-recorrido-breve-a-lo-largo-de-la-historia-lo-bueno-y-lo-malo/
Center of Excellence in Statistical Information on Government [Public Security,
Victimization and Justice]a (CDEUNODC). (2013). Inventario de Encuestas de
Victimización en América Latina y el Caribe [Inventory of victimization surveys in
Latin America and the Caribbean]. UNODC.
Cohen, E. and Franco, R. (2000). Evaluación de Proyectos Sociales [Evaluation of social proj-
ects]. Mexico: Siglo XXI Editores.
Comisión Económica para América Latina y el Caribe (CEPAL) [Economic Commission
for Latin America and the Caribbean (ECLAC)]. (2014). Documento conceptual de
los sistemas estadísticos de seguridad pública y justicia en los países de América
Latina y el Caribe [Concept paper on Statistical Systems Citizen Security and
Justice in Latin America and the Caribbean]. Statistical Conference of the Americas.
Mexico: United Nations, ECLAC and Center of Excellence in Statistical Information
on Government, Public Security,, Victimization and Justice.
Concha-Eastman, A. and Guerrero, R. (1999). Vigilancia epidemiológica para la pre-
vención y el control de la violencia en las ciudades [Epidemiological surveillance
for the prevention and control of violence in cities]. Panameña de Salud Pública/
Panama Journal of Public Health (4/5): p.322. Washington D.C.: Organización
Panamericana de la Salud [Pan American Health Organization].
Daigle, L. (2017). Victimology: A text/reader. London: Sage Publications.
Dammert, L., Ruz, F. and Salazar, F. (2008). ¿Políticas de seguridad a ciegas? Desafíos para
la construcción de sistemas de información en América Latina [Blind security policies?
Challenges for the construction of information systems in Latin America]. Santiago de
Chile: FLACSO-CHILE.
Dammert, L., Salazar, L., Montt, C. and González, P. (2010). Crimen e Inseguridad. Indicadores
para las Américas [Crime and Insecurity. Indicators for the Americas]. Santiago de Chile:
FLACSO-Chile/Banco Inter-American Development Bank (IDB).
Gibbons M. and M. Rossi (2015). Evaluación de Impacto de un Programa de Inclusión
Social y Prevención de la violencia Estudiantil [Impact Evaluation of a Social
89
Inclusion and Student Violence Prevention]. IDB Working paper No. IDB-WP-624.
Washington DC.
Estrategias de Acompañamiento y Servicios Educativos S de RL de CV (EASE) (2017).
-Report entitled Acompañamiento y Evaluación de la intervención de Cauce Ciudadano-
Cure Violence en Ecatepec, Estado de México. Teoría del cambio, modelo de marco lógico
e indicadores [Accompaniment and evaluation of the Cauce Ciudadano-Cure Violence
intervention in Ecatepec, State of Mexico.Theory of change, logical framework model
and indicators]. Washington D.C.: Inter-American Development Bank (IDB).
Fajnzylber, P., Lederman, D. and Loayza, N. (1998). Determinants of crime rates in Latin
America and the world. Washington D. C.: World Bank.
Frühling, H. (2003). Police Reform and the Process of Democratization. In Frühling,
H., Tulchin, J. and Golding, H., Crime and Violence in Latin America: Citizen Security,
Democracy and the State. Washington D. C.: Woodrow Wilson Center Press: 15-44.
Frühling, H. and Mohor, A. (2009). Medición de análisis de la criminalidad en los países
de América Latina. Lecciones aprendidas [Measurement and analysis of criminal-
ity in Latin America. Lessons learned]. Round table on criminality. Cali, Colombia:
Proyecto Sistema Regional de Indicadores Estandarizados de Convivencia y
Seguridad Ciudadana [Regional System of Standardized Indicators in Peaceful
Coexistence and Citizen Security Project].
Frühling, H. (2012). Discussion paper: La Eficacia de las Políticas Públicas de Seguridad
Ciudadana en América Latina y el Caribe: Cómo Medirla y Cómo Mejorarla [Efficacy
of public policies in citizen security in Latin America and the Caribbean: how to
measure it and how to improve it]. Diálogo de Política Regional [Dialogue on Regional
Policy]. VENUE: Inter-American Development Bank (IDB).
Hales, J., Nevill, C., Pudney, S. and Tipping, S. (2009). Research report 19: Longitudinal anal-
ysis of the offending, crime and justice survey 2003-06. London: Home Office. United
Kingdom.
Hernández Sampieri, R., Fernández Collado, C. and Baptista Lucio, P. (2007). Fundamentos
de Metodología de la Investigación [Fundamentals of the Methodology of
Research]. Madrid: McGraw Hill.
Hiskey, J., Malone, M. and Orcés, D. (2014). Violencia y migración en Centroamérica
[Violence and Migration in Central America]. Vanderbilt University: Barómetro de las
Américas [The Americas Barometer].
Hoyle, C. (2012). Victims, Victimisation and Restorative Justice. In M. Maguire, R. Morgan,
and R. Reiner (eds.), The Oxford Handbook of Criminology. Oxford University Press.
Module 3:
INTEGRAM (2015). Informe final de la Evaluación específica del desempeño del

Programa Nacional para la Prevención del Delito 2014 [Final report on the specific
evaluation on the performance of the crime prevention national program 2014].
INTEGRAM Administración y Finanzas, S.A. of C.V., Government of the state of
Chihuahua, Chihuahua, Mexico.
IDB. (2017). Documento de Marco Sectorial de Seguridad Ciudadana y Justicia [Sector
Framework of Citizen Security and Justice Document]. Washington D.C.: Inter-
American Development Bank (IDB).
Inter-American Development Bank (IDB). (2010). Prevención de delito y la violencia en
América Latina y el Caribe: Evidencia de las Intervenciones del BID [Crime and Violence
Prevention in Latin America and the Caribbean: Evidence from IDB’s Interventions].
Washington D. C.: Inter-American Development Bank (IDB).
Institute for Economics & Peace. (2016). Global Peace Index. Sydney, New York, Brussels
and Mexico: Institute for Economics & Peace.
Krug, E., Dahlberg, L., Mercy, J., Zwi, A. and Lozano, R. (2003). Informe mundial sobre
la violencia y la salud [World Report on Violence and Health]. Washington D. C.:
Organización Panamericana de la Salud [Pan American Health Organization].
Lagos, M. Y Dammert, L. (2012). La Seguridad Ciudadana. El Problema Principal de
América Latina [Citizen Security: Major problem of Latin America] . Lima: Corporación
Latinobarómetro.
Londoño, J. and Guerrero, R. (1999). Violencia en América Latina. Epidemiología y costos
[Violence in Latin America. Epidemiology and Cost]. Washington D.C.: Inter-American
Development Bank (IDB).
LAPOP (2014). Perspectivas desde el Barómetro de las Américas: 2014 [Perspectives
from the Americas Barometer: 2014]. Number 101. Co-edited by Jonathan H., M. A.
Seligson and E. J. Zechmeister cwith the support of the LAPOP team at Vanderbilt
University. www.AmericasBarometro.org
Latin American Development Bank (CAF). (2014). Por una América Latina más segura. Una
nueva perspectiva para prevenir y controlar el delito [Towards a safer Latin America.
A new perspective to prevent and control crime]. Bogotá: Corporación Andina de
Fomento.
Maguire, M. (2012). Criminal statistics and the construction of crime. In Maguire, M.,
Morgan, R. and Reiner, R., The Oxford Handbook of Criminology. Oxford: Oxford
University Press.
91
Manzano, L. (2009). Violencia en Barrios Críticos: Explicaciones Teóricas y Estrategias de

Intervención Basadas en el Papel de la Comunidad [Violence in Critical Neighborhoods:
Theoretical Explanations and Intervention Strategies Based on the Role of the
Community]. Santiago de Chile: Ril Editores.
Manzano, L. and Mellado, C. (2016). Modelos y Herramientas para la Evaluación de
Políticas o Programas de Seguridad Ciudadana [Models and Tools for Evaluating
Citizen Security Policies and Programs]. Curso de Metodologías Aplicadas a la
Investigación Social en Seguridad y Justicia [Course on Applied Methodologies in Social
Research on Security and Justice]. Thematic Unit 3. University of Chile: Instituto de
Asuntos Públicos [Institute of Public Affairs].
McAra, L. and McVie, S. (2012). Critical debates in developmental and life-course
criminology. In Maguire, M., Morgan, R. and Reiner, R., The Oxford Handbook of
Criminology Oxford: The Oxford University Press: 531-560.
Mendes de Souza, E., M. Carvalho, R. Gomes, L. Pereira and D. Monique Carlos (2014).
Justice system and secondary victimization of children and or adolescents victi-
mas of sexual violence in the family. Saude e Sociedade, vol. 23 n.3, Sao Paulo, July/
September 2014.
Ministerio Público de la República del Perú [Public Ministry of the Republic of Peru].
(2016). Anuario Estadístico 2015 [Statistical yearbook 2015]. Lima: Ministerio
Público de la República del Perú [Public Ministry of the Republic of Peru.
Mohor, A. (2007). Uso de Indicadores para Evaluar el Funcionamiento Policial [Use of inidcators
to evaluate police functioning]. Santiago de Chile: Centro de Estudios en Seguridad
Ciudadana del Instituto de Asuntos Públicos [Centre of Studies in Citizen Security
of the Instistitue of Public Affairs]. University de Chile.
Mora Chamorro, H. (2008). Manual de protección a víctimas de violencia de género
[Manual for the protection of victims of gender-based violence] . Spain: Editorial Club
Universitario.
Murria, M. and González, C. (2010). La seguridad ciudadana: Instrumentos de análisis
[Citizen security: analysis instruments]. Barcelona: Instituto de Estudios Regionales
[Institute of Regional Studies and Metropolitanos de Barcelona.
National Institute of Statistics -NIS- Chile (2011). Memoria de la Encuesta Nacional Urbana
de Seguridad Ciudadana [Report on the National Urban Survey of Citizen Security].
Available at: http://historico.ine.cl/canales/chile_estadistico/encuestas_seguri-
dadciudadana/pdf/memoria_enusc.pdf
Module 3:
Newburn, T. (2013). Criminology. Oxon: Routledge.

Nowak, M. (2012). Femicide: A global problem. Small Arms Survey Research Notes, No. 14.
Switzerland: Graduate Institute of International and Development Studies.
OACNUDH and SSPDF (2014). Indicadores sobre Seguridad Ciudadana y Derechos
Humanos de la Secretaría de Seguridad Pública del Distrito Federal [Indicators on
citizen security and human rights of the Ministry of Public Security of the Federal
District . Mexico office of the High Commissioner of the United Nations for Human
Rights and the Federal District Public Ministry. Published by SSPDF, Mexico D.F.
Dominican Republic Citizen Security Observatory (OSC, according to its Spanish acro-
nym). (2017). Informe Estadístico sobre Seguridad Ciudadana 2016 [Statistical Report
on Citizen Security 2016]. Santo Domingo: Dominican Republic Citizen Security
Observato (OSC).
United Nations Office on Drugs and Crime (UNODC). (2007). Compendium of United
Nations standards and norms in crime prevention and criminal justicel. Vienna: UNODC.
UNODC. (2013). Global Study on Homicide. Viena: UNODC.
UNODC. (2015). [International Classification of Crime for Statistical Purposes. Vienna:
UNODC.
Olavarría, M. (2007). Documento de Apoyo Docente Nº17: Nociones de Evaluación de
Programas.[Educational support document Nº17: Evaluation notions on programs]
Santiago de Chile: Instituto de Asuntos Públicos de la Universidad de Chile
[Institute of Public Affairs of the University of Chile].
Ortegón, E., Pacheco, J. and Prieto, A. (2005). Metodología del marco lógico para la plan-
ificación, el seguimiento y la evaluación de proyectos y programas.[Methodology
of logical framework for the planning, monitoring and evaluation of projects and
programs] Santiago de Chile: CEPAL.
Otamendi, M. (2016). Seguridad objetiva y subjetiva en América Latina: Aclarando la
paradoja [Objective and subjective security in Latin America: Clearing up the para-
dox]. Brasileña de Seguridad Pública journal, São Paulo v. 10, n. 1: 56-87.
Pasara, L. (2015). Las víctimas en el sistema procesal penal reformado [Victims in the reformed
penal process system]. Revista de la Facultad de Derecho PUCP, No. 75: 317-331.
Perez-Luco, E., Lagos, L.and Báez, C. (2012). Reincidencia y desistimiento en adoles-
centes infractores: Análisis de trayectorias delictivas a partir de autoreporte de
delitos, consumo de sustancias y juicio profesional [Recidivism and withdrawal in
juvenile offenders: Analysis of crime pathways based on crime self-reporting, sub-
stance use and professional assessment. Universitas Psychologica journal, Vol. 11,
Núm. 4 : 1209-1225.
93
Public Ministry of Chile. (2017). Boletin estadístico anual. Enero - diciembre 2016 [Annual
statistics bulletin. January - December 2016]. Santiago de Chile: Public Ministry of
Chile.
United Nations Development Program (UNDP). (2013). a Citizen Security with a Human
Face: Evidence and Proposals for Latin America. New York: UNDP.
Ribeiro, E., Borges, D. and Cano, I. (2015). Calidad de los datos de homicidio en América
Latina [Quality of homicide data in Latin America]. [Document prepared by the
“Conference on the quality of homicide data in Latin America and the Caribbean”].
Bogotá: Open Society Foundations.
Riego, C. (2014). La expansión de las facultades de la víctima en la Reforma Procesal
Penal y con posterioridad a ella. [The expansion of the victim’s faculties in the penal
process reform and subsequent to it] Política Criminal, Vol. 9,18: 668-690.
Salazar, F. (2013). Estado del arte de definición de indicadores en América Latina y el Caribe
[State-of-the-Art in the Use of Indicators in Latin America and the Caribbean]. Quito:
FLACSO-Ecuador.
Sherman, L. (2012). Development and Evaluation of Citizen Security Programs in Latin
America. Protocol for Crime Prevention Based on the Evidence. Washington D.C.: Inter-
American Development Bank (IDB)
Síndic de Greuges de Catalunya. (2012). Informe sobre el derecho de acceso a la información
pública [Report on the right of access to public information]. 1a edición. Barcelona:
Síndic de Greuges de Catalunya
Straits, B. y Singleton, R. (2011). Social Research: Approaches and Fundamentals. Oxford:
Oxford University Press.
Tappan, P. (1947). Who is the criminal? American Sociological Review, Vol. 12, No. 1:
96-102.
Tocornal, X., Viano, C. and Zuloaga, L. (2005). Los Qué, Cómo y Cuándo de la Evaluación:
Recomendaciones para el Diseño de Evaluaciones de Proyectos de Prevención
Comunitaria del Delito [The What, How and When of Evaluation: Recommendations
for evaluation design for Community Prevention of Crime Projects]. Boletín
+Comunidad +Prevención. Santiago, Chile: CESC.
Vilalta, C., Castillo, J. and Torres, J. (2016). Delitos violentos en ciudades de América
Latina [Violent Crime in Latin American cities] . Washington D. C.: Inter-American
Development Bank (IDB).
Module 3:
VICTIMIZATION SURVEYS, CONSULTED SITES:
http://www.cdeunodc.inegi.org.mx/articulos/doc/1_inv_
encuesvic.pdf
Regional references: spe-
cialized centers or security http://www.cdeunodc.inegi.org.mx/index.php/mapa/
observatories
http://www.oas.org/dsp/espanol/cpo_observatorio_enlaces_
victimizacion.asp#AR
Argentina (2017)
http://www.jus.gob.ar/areas-tematicas/estadisticas-de-polit-
Buenos Aires and other cities ica-criminal/estudios-de-victimizacion.aspx
(since 1995)
Barbados (since 2002)
victimizacion.asp#AR
Bolivia, large cities (2011) http://www.onsc.gob.bo/
Brazil (2010-2012)
Brazil (National Household www.ibge.gov.brwww.justica.gov.br

Survey, 2009)
Chile (since 2003) http://www.seguridadpublica.gov.cl/encuestas/
http://www.dane.gov.co/index.php/esta-
Colombia (since 2012)
disticas-por-tema/seguridad-y-defensa/
Bogotá, Cali and Medellín encuesta-de-convivencia-y-seguridad-ciudadana-ecsc
(since 2003, implemented
http://www.dane.gov.co/index.php/estadisticas-por-tema/
several times)
seguridad-y-defensa/encuesta-de-victimizacion
Costa Rica (2008-2010) victimizacion.asp#AR
www.inec.go.cr
http://www.ecuadorencifras.gob.ec/encuesta-de-victim-
izacion-y-percepcion-de-inseguridad-2011/
Ecuador - Quito (2003, 2004 http://www.oas.org/dsp/documents/victimization_surveys/
and 2008, 2011) ecuador/ecuador%20-%20encuestas%20de%20victimazi-
Ecuador (2008 and 2011) cion.pdf
http://omsc.quito.gob.ec/index.php/12-contenido-ejem-
plo/40-encuesta-de-victimizacion.html
95
http://www.oas.org/dsp/documents/victimization_surveys/
El Salvador (2004 and 2009)
el_salvador/victimizaci%C3%B3n_el_Salvador_2009.pdf
http://www.cdeunodc.inegi.org.mx/index.php/mapa/
Guatemala (since 2004) www.poderjudicial.gob.ni/genero/ovgn/informe_final_
febrero_2015.pdf
Guyana (LAPOP, 2014) http://www.cdeunodc.inegi.org.mx/index.php/mapa/
https://presencia.unah.edu.hn/seguridad/articulo/iud-
Honduras (2014 and 2016) pas-aplicara-encuesta-de-percepcion-de-inseguridad-y-vic-
timizacion-en-honduras
Jamaica (2006, 2009 and
http://statinja.gov.jm/projects.aspx
2012-2013)
http://www.inegi.org.mx/est/contenidos/proyectos/encues-
tas/hogares/regulares/envipe/http://www.cdeunodc.inegi.
Mexico org.mx/index.php/mapa/
(2002-2010/2011-2017)
http://www.beta.inegi.org.mx/proyectos/enchogares/
regulares/envipe/2017/
Nicaragua (since 2007) https://www.ieepp.org/media/files/publicacion-2-509.pdf
http://www.pa.undp.org/content/panama/en/home/library/
Panama (since 2010) democratic_governance/viii_informe_seguidad_ciudad-
ana_2017.html
http://www.cdeunodc.inegi.org.mx/index.php/mapa/
Paraguay (2010)
http://www.dgeec.gov.py/
Peru (National Survey of http://www.inei.gob.pe/media/MenuRecursivo/publica-
Budget Programs, since 2010) ciones_digitales/Est/Lib1442/libro.pdf
Peru, large cities (2005), Lima https://www.inei.gob.pe/media/MenuRecursivo/publica-

(2009) ciones_digitales/Est/Lib1427/libro.pdf
Dominican Republic (National
www.one.gob.do/Estadisticas/216/encuesta-nacional-de-ho-
Household Survey, since
gares-de-propositos-multiples-(ENHOGAR)
2005)
Uruguay (2009-2016 / 2017) http://www.ine.gub.uy/web/guest/victimizacion
https://www.oas.org/dsp/pdfs/encuestavictimizacion2009.
Venezuela (2006 and 2009)
pdf
Module 3:
OPTIONAL READING
HOW TO BUILD INDICATORS?
Two key steps must be followed to build indicators: conceptualization and operational-
ization (or transforming the concept into an indicator).
Conceptualization
Define theoretically the concept
that will be measured
Operationalization
Transform the abstract concept
into objective categories
CONCEPTUALIZATION
The indicator must have a solid theoretical basis, which supports the decisions that are
taken in its construction and the theoretical construct it intends to represent. In this first
stage, it is necessary to clearly define what the indicator is expected to measure, the con-
cept that will be measured, what the limits of the concept are, and what the indicator will
emphasize, if anything at all. The process of constructing the indicator consists of moving
from the more abstract to the more concrete so that at the end of the conceptual discus-
sion, the result will be a clear and concise theoretical definition to be disaggregated and
operational in subsequent phases.
Example of the Result of the Conceptualization: Global Impunity Index 2015

(CESIJ, 2015)
Concept Theoretical Definition

The impossibility, de jure or de facto, of bringing the perpetrators of
violations to account—whether in criminal, civil, administrative or
disciplinary proceedings—since they are not subject to any inquiry
Impunity
that might lead to their being accused, arrested, tried and, if found
guilty, sentenced to appropriate penalties, and to making reparations
to their victims.
97
OPERATIONALIZATION
The process of operationalization of the theoretical definition has two components: the
definition of dimensions and, based on this, the definition of indicators .
A. Defining the Dimensions
The vast majority of phenomena under study—from the fields of citizen security and
criminal justice to many other social sciences—are clearly multidimensional. For this
reason, it is necessary to extract from the conceptual development in the previous phase
some specific dimensions that explain the phenomenon under study in its complexity.
Example of Defining the Dimensions: Global Impunity Index 2015 (CESIJ,

2015)
Concept Theoretical definition Dimensions
Structural: installed capacities that states

have to punish those who violate the rule
The impossibility, de jure of law in accordance with the rules of due
or de facto, of bringing the process.
perpetrators of violations to
account—whether in criminal, Functional: the way in which government
civil, administrative or disci- areas responsible for punishing those who
plinary proceedings—since violate the rule of law operate, regardless of
Impunity they are not subject to any their policy framework, or of their capabilities
inquiry that might lead to their and institutional infrastructure.
being accused, arrested, tried
and, if found guilty, sentenced Human rights: situation of human rights at
to appropriate penalties, and national level enables the Global Impunity
to making reparations to their Index to present a comprehensive proposal
victims. on the subject and to refer to the crimes com-
mitted by the State itself, which also remains
unpunished.
Module 3:
B. Definition of Indicators
At this stage, it is necessary to select the indicators, which are numerical measurements
that capture the meaning of the defined dimensions. In many cases, more than one indica-
tor will be required to give an account of the concept, and instead a cluster of indicators
from the same or different sources of information may be needed. It is recommended
that a list is prepared of all the indicators that could provide the most complete informa-
tion about the dimensions of the concept for each dimension, and then to evaluate the
relevance of including the indicator: in terms of its strengths and weaknesses; the infor-
mation source; the quality of the indicator; and the replicability of the data (for example,
if the same information will be available in two or three years).
Example of Results from Constructing Indicators: Global Impunity Index

2015 (CESIJ, 2015)
Concept Theoretical definition Dimensions Indicators
The impossibility, de jure ◊ Police personnel per 100,000

or de facto, of bringing inhabitants.
the perpetrators of
◊ Prison personnel per peniten-
violations to account—
whether in criminal, tiary capacity.
civil, administrative or ◊ Prisoners in prisons, peniten-
disciplinary proceed- tiary or correction institutions
ings—since they are not
Impunity Structures per penitentiary capacity.
subject to any inquiry
that might lead to their ◊ Number of judges and profes-
being accused, arrested, sional magistrates per 100,000
tried and, if found guilty, inhabitants (security system).
sentenced to appro-
◊ Number of judges per 100,000
priate penalties, and to
inhabitants (justice system).
making reparations to
their victims.
99
GEOGRAPHICAL INFORMATION SYSTEM (GIS)
Criminology suggests that criminal activity tends to be concentrated in particular places

and times. Criminal phenomena occur in specific locations, just as the perpetrators have
a place of origin. For this reason, spatiality and temporality are components of the phe-
nomenon of crime that are relevant for the analysis. With this objective in mind, the use
of Geographic Information Systems (GIS) started in the 70s, which allows the events to
be displayed on a map, graphically illustrating crime patterns, their concentration and
spatial distribution.
GIS can play an important role in decision-making in public policies and in the reduction
of crime by providing information for the diagnosis, monitoring and evaluating the inter-
vention object.
Georeferencing of crime: refers to locating the criminal facts on a map, and in

this way observing the distribution and concentration of events or their
displacement.
In recent decades the development of technologies has facilitated the use of georef-
erencing techniques. The use of digital maps and databases standardized in the same
format allows for the display of a crime phenomenon in its temporal-spatial context in
greater depth, making it possible to access small units of analysis. This methodology can
be used on different scales to observe trends at a global, regional, national or local level.
Module 3:
Components of a GIS:
◊ Information available on events with standardized geographic informa-

tion fields.
◊ Computer equipment with good capacity (hardware).
◊ Specialized software (ArcView 3.2 - 8x; arcGis; ArcInfo 8.02; statistical
analysis software, etc.).
◊ Specialized professionals (cartographers, geographers).
◊ Digital maps of the areas to be studied with the necessary technical
characteristics.
◊ Other sources of georeferenced information that enables basic social or
demographic analysis.
The georeferencing of crimes is a technique widely used by the police, and one of the
main sources of information to perform this type of analysis are police administrative
records since they have information on the location of reports and arrests made by the
police. From this and other sources of information, the main analyses that are performed
observe points or frequencies, areas of concentration and the analysis of hotspots.
◊ Analysis of points or frequencies: the criminal facts registered at a point are posi-
tioned on a map. As the frequency of criminal acts increases in the same place, the
size or color of this point is modified.
◊ Analysis of concentration: The map is divided into grids, corresponding to seg-

ments of urban space, blocks, neighborhoods, or other sections. The accumulation
of criminal acts in each grid is calculated by frequencies, assigning colors to each of
them according to concentration categories.
◊ Hotspots: These facilitate the identification of the areas with the highest density
of criminal activity occurrence, making it possible to observe the displacement of
these facts. This methodology is used by the police for the organization of police
activities, identifying those areas where their impact can be greater. While sev-
eral studies support the results of this strategy in Latin America, very few police
departments deploy it.
101
USE OF GIS FOR CRIME DIAGNOSIS AT LOCAL LEVEL
For the intervention of problem areas associated with security and justice at local level,
it is essential to know the criminal dynamics, the spatial distribution of criminal or vio-
lent acts and their causes. With an adequate diagnosis, local governments can design
public policies that respond effectively to reality, focusing their interventions on those
territories of greater complexity.
For local diagnosis, it is possible to georeference administrative police information,

qualitative information and data on local government programs. As an example of the
application of this type of methodology, the experience of Ciudad Juarez, Mexico is given
below.
In Mexico, the implementation of the Program for Citizen Coexistence began in 2012
as part of the Merida initiative—an agreement between the governments of Mexico
and the United States to reduce violence in border states. In this framework, a group
of industrial sites in Ciudad Juarez were selected for the program. Before designing the
Master Plan for Prevention, a local diagnosis was developed, identifying the major prob-
lems that required intervention. These diagnostics integrated information from official
records, qualitative data obtained from the discourse of the community, demographic
data and georeferenced information. This made it possible to make a diagnosis that cov-
ered the local situation in terms of socio-demographic characteristics, education, access
to health, housing conditions, equipment, income, population distribution and crime
context. The mapping of criminal information allows for the construction of a local diag-
nosis that takes into account the spatiality of the phenomenon. By way of example, the
diagnosis made for the city of Juarez presented georeferenced information on the teen-
agers’ place of residence who had violated the law in the period 2007-2010.
Based on this information, the diagnosis could conclude that there was a concentration
of adolescents who broke the law in some areas of the city. The diagnosis suggested that
this distribution was associated with areas with few educational services on offer, as
well as few recreational areas. This is illustrated more clearly in the chart below, which
uses the hotspots methodology to georeference the location of young people who have
broken the law.
Module 3:
Figure 1. Georeferencing of juvenile delinquency, Ciudad Juárez, 2010
Source: Program for Citizen Coexistence (2012: 27)

http://mexicosos.org/descargas/dossier/estudios/plan_maestro_juarez_fco_i_madero.pdf
103
Figure 2. Critical areas of juvenile delinquency per colony,

Ciudad Juárez, 2010
Source: Program for Citizen Coexistence (2012: 28)

http://mexicosos.org/descargas/dossier/estudios/plan_maestro_juarez_fco_i_madero.pdf
Module 3:
QUALITY OR TECHNICAL RIGOR CRITERIA
The use of measuring instruments on violence and crime, such as victimization surveys,
only makes sense if they have coherent and quality information (Dammert et al., 2010).
This implies that this information meets certain criteria of technical rigor, such as valid-
ity, reliability, objectivity, standardization and comparability. These criteria are based on
the definitions proposed by Trochim and Donelly (2008) and Hernandez Sampieri et al.
(2007), although they can be found in other methodological manuals.
Validity: refers to the measuring capacity of an instrument; that is to say, a valid instru-
ment is one that, through the variables to be investigated collects information about the
phenomenon under study and nothing else. An instrument can have content, criterion
and construct validity.
For an instrument to comply with content validity, it must be based on a process of

operationalization of variables that provides a solid theoretical foundation. To prove
the criterion validity, the extent to which the instrument’s parameters behave in a sim-
ilar way to another information parameter which collects accurate information on the
same study object should be evaluated. Finally, every instrument should aim for con-
struct validity, which shows the relationship with the degree to which inferences can be
performed based on the measurements made by the instrument, i.e. to what extent the
observations made by the instrument approach the theory.
Reliability: refers to the consistency between different measurements taken by the same
instrument, which is achieved through a coherent instrument that is stable over time.
This implies that the error induced by a particular measurement—either by chance, by
the observer’s application of the instrument or by distortions of the instrument itself—
are the least possible. Different strategies, including the test-retest of the instrument,
the parallel forms method or internal consistency tests are used to measure reliability.
See Figure 3 for an illustration of validity and reliability, in which the red point is the
concept to be measured. First of all, there is an instrument that is reliable since the mea-
surements are consistent; however, it is not valid, as the measurement does not repre-
sent the measured concept properly. In second place, there is an instrument that is not
reliable due to the dispersion that the observations show, although it is valid because it
refers to the concept measured. In third place, we can see an instrument that does not
105
meet the standards of validity or reliability as it presents scattered observations that

do not represent the concept. In summary, an instrument should be as close as possible
to the final figure, in which the observations are consistent and cluster around the con-
struct that it is expected to measure.
Figure 3: Validity and Reliability of an Instrument
Reliable Valid Unreliable Reliable

Invalid Unreliable Invalid Valid
Source: Trochim and Donelly (2008). Own translation.
Objectivity: an instrument is objective to the extent that the observation of the phe-
nomenon is impartial, without it being committed to obtain certain results or distort
the information collected. As long as an instrument is valid and reliable, its objectivity
improves. However, the design and administration of an instrument by expert and inde-
pendent bodies undoubtedly ensures higher levels of objectivity.
Standardization: refers to the establishment of definitions and categories that are pre-
cise, exclusive, mutually coherent and that remain consistent in time. This facilitates the
comparability of the data over time (analysis of trends), independent of any changes that
may occur in the teams applying the instruments.
Periodicity: Maintaining a measuring instrument over time, but also applying it in fixed
and continuous time divisions—for example, semi-annual, annual, biannual—is essential
to ensure the quality of the information being collected and enable the trend analysis
of the criminal phenomenon. Furthermore, maintaining periodicity in the dissemination
of data promotes the legitimacy of the instruments. Therefore, while administrative
records are usually collected routinely and disseminated regularly, in terms of victimiza-
tion surveys this remains a challenge for most of the countries in the region.
Module 3:
Comparability: In addition to the temporal comparability of data (trends), designing

instruments that are based on international standards that include concepts such as
types of questions, categories and application methodologies, make it possible to obtain
comparable data of countries with different realities. Only regional victimization and
public opinion surveys, such as Latinobarómetro and LAPOP, now meet criteria for
regional standardization and comparability.
Examples of Public Opinion Polls
Public Opinion Studies at Regional Level
Latinobarómetro is a public opinion instrument which was designed by Latinobarómetro

Corporation and is implemented in each country by collaborating companies., From 1995
to 2017 it was applied annually. In 2016 it was implemented in 18 countries in the region:
Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala,
Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru, Dominican Republic, Uruguay and
Venezuela.
On the methodological aspects of its implementation, the questionnaire applies to people

over the age of 18, conducting face to face interviews and considers a sample of 1,000 to
1,200 cases by country, which represent the total national population, resulting in a total
sample of 20,000 cases, approximately. The sample design is defined independently by each
country.
The instrument collects information on topics such as:
◊ Support and satisfaction with democracy, abidance to norms, freedom of expression,

interpersonal trust and trust towards institutions.
◊ Approval of the government, evaluation of the policy, the national economic situation
and the media.
◊ Satisfaction with life and the economy, and with work.
◊ Victimization for delinquency and domestic violence, fear of crime, and corruption.
The databank is available at: www.latinobarometro.org

107
The AmericasBarometer (LAPOP) is a public opinion instrument that was designed at

Vanderbilt University, although it is applied at a national level by collaborating institutions.
It was implemented every 2 years between 2004 and 2017. In 2014 it was applied to 28
countries in the three regions of the Americas—South America, Central America and the
Caribbean—as well as North America.
Regarding methodological aspects of information collection, the questionnaire is applied to

adults of voting age in face to face interviews, with the exception of the United States and
Canada where the survey is conducted online. The questionnaire considers a sample of 1,500
to 4,000 cases per country, which represents the total national population, coming to 50,000
surveys. The sample design is probabilistic in each country.
This instrument collects information on topics such as:
◊ Political legitimacy and political tolerance.
◊ Support for stable democracy and electoral behavior.
◊ Respect for the rule of law.
◊ Evaluation of local governments, participation of civil society and social capital.
◊ Victimization by crime, victimization by corruption, insecurity and its effects..
The database is available at: https://www.vanderbilt.edu/lapop/
THE LOGICAL FRAMEWORK MATRIX FOR THE

FORMULATION AND EVALUATION OF PROJECTS
The Logical Framework is a tool designed to facilitate the process of identification, for-
mulation or design, execution and evaluation of projects, with the objective of structur-
ing the planning process so that the essential information of the project is communicated
clearly (Ortegon et al., 2005; Manzano and Mellado, 2016).
The Logical Framework is used during all phases of the project cycle, enabling the follow-
ing actions:
Module 3:
Carried out on the basis of an analysis of the popula-

Identification and prioritization
tion’s problems and their possible solutions.
Performed by specifying and quantitatively estimating

Ex-ante formulation and evaluation
the benefits and costs involved in a project.
Undertaken by specifying exactly which, activities and

Operations planning
resources are necessary for the execution of a project.
Monitoring and evaluation Performed on the basis of a set of indicators.
Ex-post evaluation and analysis Carried out in order to determine its specific contribu-
of the social impact of a project tion in the context which the project was designed for.
As stated previously, the Logical Framework is presented as a matrix of four columns by

four rows. The columns provide the following information:
◊ Intervention strategy or a narrative summary of the different levels of objectives

and activities.
◊ Indicators (specific results to be achieved).
◊ Means of verification.
◊ Assumptions (external factors that involve risks).
The rows of the matrix present information about the objectives, indicators, verification
means and assumptions at four different moments in the life of the project:
◊ The goal which the project contributes to once it is underway.
◊ Purpose achieved when the project has been executed.
◊ Components/Results completed during the execution of the project.
◊ Activities required to produce the components/results.

109
INTERVENTION VERIFICATION
STRATEGY MEANS
Goal Goal Indicators
Purpose Purpose indicators

Monitoring and
Context
Component indicators evaluation system
Components
Activities Budget
Module 3:
GOAL
◊ Every project responds to a problem or obstacle that has been detected. The goal
expresses the solution to the diagnosed problem.
◊ The goal, therefore, is a higher-level objective which the project attempts to contribute to.
◊ This is the description of how the project contributes, to the long-term solution of the
identified problem, which does not mean that the project itself will be sufficient to
achieve the goal and that there may be other projects and/or programs that also con-
tribute to its achievement.
◊ The indicators at the goal level measure the overall impact that the project will have.
Therefore, they are specific in terms of quantity, quality and time.
◊ The assumptions indicate the events, conditions or important and necessary decisions
for the sustainability (continuity in time) of the benefits generated by the project.
PURPOSE
◊ This is the general objective of the project and constitutes the concrete and direct
effect or impact, which will be achieved as a result of the use of the components pro-
duced by the project during its execution.
◊ It is a hypothesis about the desired impact or benefit which can and should be measured.
◊ The achievement of the purpose will contribute to achieving the goal, i.e. the specific
contribution to the solution of the problem diagnosed.
◊ It is expressed as a result, for example: “It improves the quality of teaching”, and not, “It
will improve”.
◊ It is what we expect to happen when the beneficiaries of the project use the products
we deliver them.
◊ It must be taken into account that for the purposes of the Logical Framework each proj-
ect is required to have only one purpose or general objective: two purposes are two
projects.
◊ The indicators at purpose level describe the impact achieved at the end of the project
and include goals that reflect the situation on the completion of the project. Each indi-
cator specifies quantity, quality and time of the results to be achieved.
111
COMPONENTS
◊ These are the specific results of the project, i.e. the products (goods and/or services)
produced or delivered by the program to fulfill its purpose.
◊ A component is a good and/or service aimed at the end beneficiary or, in some cases, at
intermediate beneficiaries: it is not a stage in the production process of the component.
◊ Each component must be necessary to achieve the purpose and therefore it is reason-
able to assume that if all the components are produced as planned, the purpose will be
fulfilled: the set of components allows for the achievement of the purpose.
◊ The components must be written clearly and as results, actions, finished work, or final
products; for example: alarms installed, workshops held, training carried out, etc.
◊ Its production mode should also be noted (enforcement mechanisms or ways to pro-
vide products and/or services), specifying who is responsible for the execution of the
components.
ACTIVITIES
◊ These are the detailed tasks that must be performed to achieve each of the compo-
nents of the project, and they involve costs.
◊ To determine them, a list of activities is prepared in chronological order for each com-
ponent and the time and resources required to carry it out are estimated. This detailed
list of activities will be the basis for the preparation of the Execution Plan, Calendar,
Gantt Chart or Operational Plan of the Project.
◊ In some cases, the activities indicator is the budget for each component, which will be
produced by the project.
◊ The means of verification indicate where an evaluator can get information to verify if
the budget was spent as planned and is normally the accounting record of the execut-
ing unit.
Module 3:
INDICATORS
◊ The indicator is the quantitative or qualitative specification used to measure the

achievement of a goal. It must be accepted collectively by those involved as adequate
to measure the achievements of the project.
◊ The indicators specify precisely each objective at the component, purpose and goal
level. At activity level, it contains the costs of each activity and, overall, the cost of each
component, which constitutes the project budget.
◊ At the purpose level, the indicators measure the direct effect after the completion of
the project.
◊ The indicators set the basis for monitoring performance and evaluation and show how
the success of a project can be measured.
◊ The indicators must have three critical attributes:
◊ Be practical: it must measure what is important, with the minimum number of
indicators needed to measure each goal.
◊ Be independent: it reflects the result specified in the objective and not the
means used to achieve that result.
◊ Be objectively verifiable: in the “means of verification” column there should be
agreed sources of information that can be examined objectively to verify if a cer-
tain objective has been achieved.
◊ On the other hand, it is not enough to identify the value of an indicator for the beginning
and end of the project, we must also count on intermediate indicators which are indis-
pensable for monitoring the project: this applies especially to component indicators.
◊ It may be important to include in the project at least one qualitative indicator, espe-
cially the opinion of the beneficiaries on the performance of the project, since this fos-
ters the beneficiary’s sense of belonging to the project. Therefore, we should not rule
out the use of primary sources of information.
113
MEANS OF VERIFICATION
◊ These correspond to the sources of information that will be used to obtain the val-
ues of the indicators. They must be identified in the matrix for each of the indicators
presented.
◊ This requires that the project designers identify sources of information or arrange for
information to be collected, possibly as a project activity, with its corresponding cost.
◊ Not all information has to be statistical. The production of components can be verified
through a visual inspection of the specialist and can include published material, sur-
veys, information records, statistical reports, etc.
ASSUMPTIONS
◊ They are the external factors that are beyond the control of the person responsible for
the project, but which affect its success or failure. In other words, the assumptions are
related to the risks involved in a project, be they environmental, financial, institutional,
social, political, climatological or otherwise.
◊ The risk is expressed as an assumption that has to be fulfilled to advance to the next
level in the hierarchy of objectives. The reasoning is as follows: if we carry out the indi-
cated activities and certain assumptions are met, then we will produce the indicated
components. At the same time, if we produce the indicated components and other
assumptions are met, we will then achieve the purpose of the project. Finally, if we
achieve the purpose of the project, and further assumptions still keep showing up, then
we will contribute to the achievement of the purpose.
◊ The logical framework therefore requires that the risks be identified at each stage:
activity, component, purpose and goal. However, the objective is not to record every
eventuality that can be conceived, but to identify possibilities with a reasonable degree
of probability.
◊ The assumptions column plays an important role, both in planning and in the imple-
mentation of the project. In the first case, it serves to identify the risks that can be
avoided by incorporating additional components in the project itself, while, during exe-
cution, the assumptions indicate the factors that must be anticipated, influenced, and/
or confronted with adequate emergency plans.
◊ Assumptions can also be expressed in quantitative terms, including the minimum val-
ues of certain external factors that guarantee the normal development of the project.
Module 3:
THE LOGIC OF A PROJECT:

VERTICALLY AND HORIZONTALLY
A Logical Framework matrix has a double logic: horizontal and vertical. Only if both are
met, will the design of the matrix be consistent.
◊ Vertical Logic of the Logical Framework Matrix
This shows the cause-effect relations between objectives at different levels: in this way,
to achieve the goal it is necessary for the purpose to be fulfilled; to accomplish the pur-
pose it is necessary to produce the results (components or products); and to comply with
the products it is necessary for the activities to be carried out (to perform the activities
it is necessary to have the inputs).
The purpose, together with the assumptions at this level, describe the necessary condi-
tions, even when they are not sufficient to achieve the goal.
◊ Horizontal Logic of the Logical Framework Matrix
The horizontal logic of the matrix shows us that it is not enough to fulfill the activities
to obtain products, but, in addition, the assumptions of the activity level must be met to
have the necessary and sufficient conditions.
In the horizontal logic of the Logical Framework, the indicators that are formulated to
measure the achievement of the goal, the purpose, the components and activities must
have the capacity to effectively measure what is to be measured (validity). If the indi-
cators are not adequately expressed, a contradiction is produced between the goal or
objective, and the indicator that will be used to measure compliance with them.
To illustrate the elements discussed about the Logical Framework, a fictional project is
presented below.
115
INTERVENTION VERIFICATION
STRATEGY MEANS
The number of reports of

GOAL: The police maintain
assaults by juvenile gangs
their levels of effi-
Decrease levels of at Villa Esperanza police
Police statistics. ciency and effective-
juvenile delinquency station is reduced by 30%,
ness in their police
in Villa Esperanza. two years after the begin-
surveillance work.
ning of the project.
The participation of
young people in men- The socioeconomic
PURPOSE: tal health, training and Social statistics conditions of young
employment programs (Municipality). people do not change
Socially integrated increases by an average significantly.
young people of 25%.
improve their Parents maintain
indicators of health, their supportive atti-
Reduction of 20% in the
education and labor tude for young peo-
number of cases attended
participation. Hospital ple with problems.
that involve overdose or
statistics.
alcohol intoxication in
health centers.
COMPONENTS: 70% of young people

Survey con-
Young people know declare that they have
ducted on a
the implications of adopted self-care behav-
representative
certain behaviors for iors regarding drug and
sample. The number of drug
health. alcohol consumption.
traffickers in the
area does not grow
100 vulnerable young
Report from significantly.
Sports and recre- people participate annu-
activity
ational services are ally in programs regarding Sport continues to
supervisors.
offered. the use of free time. be one of the main
focuses of attention
Job training
At least 500 young people for young people.
programs are
from Villa Esperanza are Administrative
developed.
enrolled in the activities records of the
of completing studies and program.
job training.
In the clinic, 200 young

Accessible services
patients are attended
for young people are Clinic statistics
annually, starting in the
established.
first year of the project
Module 3:
ACTIVITY MATRIX
VERIFICATION
INTERVENTION STRATEGY INDICATORS ASSUMPTIONS
MEANS
Component 1: Young people know ◊ 40% of the young popu-
the implications of certain health lation receive courses in
◊ The young people
behaviors. sex education, drugs and
are interested
Activities alcoholism.
◊ Costs (budget). in attending the
◊ Provide sex education courses. ◊ Over a period of eight courses.
months, the main places ◊ Records of training
◊ Provide courses on drugs and institutions.
that young people use
alcoholism. ◊ Youth associations
frequently for studying and ◊ Campaign supervi-
◊ Conduct awareness campaign. recreation have posters and sion records in the neighbor-
accessories with prevention hood establish
messages.. sports programs
to use the facilities
set up.
Component 2: Sports and recre- ◊ Sports center budget
◊ Accounting records.
ational facilities are offered. executed.
◊ Municipality
◊ Build a sports center. ◊ An average of 100 vulnera-
registry.
ble young people per month
◊ Generation of recreation initia-
participating in workshops
tives and use of free time, for
and other activities is
example, art workshops.
maintained.
◊ Establish summer camps.
◊ At least 200 vulnerable
young people participate in ◊ Municipality
summer camps. registry.
◊ Training institution
Component 3: Job training pro- ◊ At least 200 vulnerable records.
grams are developed. young people have access ◊ Training institution
◊ Programs of recovery or com- to study completion records.
pletion of studies. programs.
◊ Municipal account-
◊ Job training programs for vul- ◊ At least 300 vulnerable ing records.
nerable youth. young people are trained
◊ Municipal account-
for work.
ing records.
Component 4: Accessible services

◊ Budget executed in youth
for young people are established.
clinic authorization and
◊ Open a clinic for young people. functioning.
◊ Hire specialized medical staff. ◊ Budget executed for hiring
◊ Provide pharmaceutical specialists and condom
equipment. distribution or drug rehabil-
itation program.
117
BEST PRACTICES IN INFORMATION SYSTEMS
While it is recognized that the concept of best practices arises in the field of governance
to highlight the good implementation of a public policy, program or social project and,
therefore, not directly applicable to information systems, we believe that it is possible
to establish criteria that information systems must comply with in order to be classified
as rigorous instruments that offer relevant and quality data. These criteria, as defined
in previous sections, are: validity, reliability, objectivity, standardization, periodicity and
comparability.
In light of these criteria, two examples of the region will be discussed next. The first case
is a measurement instrument that has been applied in Chile since 2003, the National
Urban Survey of Public Safety, which has been recognized by international statistical
experts, such as the Center of Excellence in Statistical Information on Government,
Crime, Victimization and Justice of the United Nations (CDEUNODC). The second exam-
ple is the Management Model by Result of the province of Pernambuco, Brazil, which
has been highlighted in the “Gobernarte” contest as a good practice in the use of criminal
information systems for the planning and management of policies at local level.
Module 3:
Urban National Survey of Citizen Security (ENUSC), Chile
The ENUSC was first in Chile in 2003, and it has been implemented annually since 2005,
keeping most of the instrument and the application methodology steady over time. From
its inception, ENUSC sought to collect information at national and regional level on the
experiences of criminal victimization suffered by households and individuals, as well as
to characterize crime and examine the reaction and perception of individuals on crime.
The survey is applied by the National Institute of Statistics and it is financed and admin-
istered by the Undersecretary of Crime Prevention of the Ministry of the Interior and
Public Security of Chile. The information is obtained from a sample of around 26,000
households which are representative of urban areas, nationally and regionally. The sur-
vey measures crime asking people older than 15 years-old, who are members of selected
households, about their victimization experiences that occurred in the 12 months prior
to the application of the tool. The tool records all “greater social connotation” offences,
including those not reported to the authorities. In addition, it enquires about economic
crimes, cybercrimes and other offences or incivilities that have occurred in the local
context.
This survey was quickly positioned as the main instrument to measure victimization in
Chile and, for this reason, has been used as a source of key information for the design of
citizen security policies (Lagos and Dammert, 2012). The validity, reliability and standard-
ization of the instrument, as well as its continuity in applications, has made it possible to
use its results over time and in international comparisons (INE, 2011).
Victimization and Insecurity in Chile, National Level, Period 2010-2016
86,8%
85,0%
Decrease People who
82,6% 2015-2016
79,9% perceive that crime
increased in the country
70,6% 71,2%
60,2%
44,5%
40,7% 40,5% 41,3% People who believe that
38,7% 39,7% Steady
35,0%
they will be victims of a
2015-2016
crime in the next 12
28,8%
26,4% 27,3% months
25,6% 24,3% 22,8% 23,5%
Steady
2015-2016 Household victimization
2010 2011 2012 2013 2014 2015 2016

The variation between 2015 and 2016 is statistically signiﬁcant
The variation between 2015 and 2016 is NOT statistically signiﬁcant
Source: Undersecretariat of Crime Prevention (2016)

For more information, please see:
http://www.seguridadpublica.gov.cl/media/2017/04/00_Total-Pa%C3%ADs_ENUSC-2016.pdf
119
Pernambuco Results Management Model
The model is based on four key pil-

lars that sustain its development
and execution. The first pillar is Planning
PDCA
the establishment of a planning
culture using the virtuous circle of
the PDCA. Namely, it is based on Indicators Goals and
objectives
planning (Plan), doing (Do), verify- and strategies
Governor
Balanced Logical Framework
ing the operation (Check) and acting Score Card Methodology
(Action). The second is the definition
of goals and objectives through the Monitoring
Logical Framework Methodology. and evaluation
Management
The third pillar is the establishment by results
of monitoring and evaluation tools
through the general idea of man-
agement by results. And, finally, the
fourth pillar is based on the construction of indicators that would allow the redefinition of
strategies, using the private administration tool called Balanced Score Card (López Ribeiro,
2014) (Graph N° 1).
Considering the high death rate by the state (53 cases per 100,000), and the aforemen-
tioned model, the Pact for Life Program (PPV) was designed in 2007. In the framework
of this program, the Ministry of Social Defense of Pernambuco adopted the indicator of
intentional lethal violent crimes (CVLI) as a criterion to monitor the reduction of violent
mortality (Pernambuco, 2007). In addition, through the management of crime analysis
and statistics, they developed a methodology of rigorous and efficient data collection
based on the crossing of data from various institutions: Civilian Police, Military Police,
the Institute of Forensic Medicine and the Institute of Criminalistics. The latter reflects
one of the main achievements of the program: inter-sectoral governance, which made it
possible to achieve the main goal of the project, a 12% reduction in lethal violent deaths.
In this way, the death rate in Pernambuco first matched the Brazilian national rate and,
later, the best international standards (López Ribeiro, 2014).
References:
Governo de Pernambuco (2007). Pact for life. State Plan of Public Safety. Recife: CEPE.
Lopes Ribeiro, L.M. (2014). Management Model by Results of the Pact for Life Program. In
Béliz, G., Basco, Ana Inés; Ribeiro, Ludmila, and others, GobernArte, Category safe govern-
ment. Inter-American Development Bank [IDB technical note; 2014, 684]: 5-21.
Module 3:
BIBLIOGRAPHY
Dammert, L., Salazar, L., Montt, C. and González, P. (2010). Crime and Insecurity. Indicators
for the Americas. Santiago de Chile: FLACSO-Chile/Inter-American Development Bank
(IDB).
Hernández Sampieri, R., Fernández Collado, C. and Baptista Lucio, P. (2007). Fundamentals
of Research Methodology. Madrid: McGraw Hill.
National Statistics Institute -INE- Chile (2011). Report on the National Urban Survey of
Citizen Security. Available at: http://historico.ine.cl/canales/chile_estadistico/
encuestas_seguridadciudadana/pdf/memoria_enusc.pdf
Lagos, M. and Dammert, L. (2012). Public Security. The Main Problem of Latin America. Lima:
Latinobarómetro Corporation.
Manzano, L. and Mellado, C. (2016). “Part three. Models and Tools for the Evaluation of Citizen
Security Policies or Programs.” Teaching Support Document, prepared within the frame-
work of the course Methodologies Applied to Social Research in Security and Justice.
Center for Studies in Public Safety, INAP, University of Chile.
Ortegón, E., Pacheco, J. and Prieto, A. (2005). Methodology of the Logical Framework for
the Planning, Monitoring and Evaluation of Projects and Programs. Santiago, Chile:
ECLAC.
Program for Citizen Coexistence PCC (2012). Community Master Plan for the Prevention
of Crime and Violence. Technology Park: Francisco I. Madero, Ciudad Juárez, Chihuahua.
Mexico. http://mexicosos.org/descargas/dossier/estudios/plan_maestro_juarez_
fco_i_madero.pdf
Trochim, M. and Donnelly, J. (2008). The Research Methods Knowledge Base. Mason:
Atomic Dog.
121
LEADERS FOR
CITIZEN SECURITY AND
JUSTICE MANAGEMENT

Asset-V1 IDBx+IDB17.3x+3T2019+Type@Asset+Block@M3 Evaluation and Information Systems

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Asset-V1 IDBx+IDB17.3x+3T2019+Type@Asset+Block@M3 Evaluation and Information Systems

Uploaded by

Copyright:

Available Formats

LEADERS FOR

CITIZEN SECURITY AND

Editors and contributors to the module

Course academic coordinators

Course instructional design

Design and layout

Graphic identity and cover

Violence and crime in Latin America have experienced strong

This module discusses the relevance of the use of information for

At the end of this module you will be able to:

◊ Reflect upon the usefulness and the use of information

◊ Identify types of relevant information sources in the

◊ Identify different sources of information available at

◊ Define concepts of diagnosis, monitoring and evaluation.

◊ Identify the different types of indicators to measure

◊ What are the purposes and contributions of information

◊ What are the most used sources of information in this

◊ What are the main attributes and limitations of these

◊ Why evaluate? What are the main functions and uses of

◊ What does it mean to evaluate within the public policies

◊ How are evaluation models classified? What are the main

In particular, to evaluate the results of specific programs or projects it is necessary

An effective security policy requires the use of an accumulation of modern methodologi-

3.1. USE OF INFORMATION SYSTEMS

Table 1. Statistical Information is Useful in Three Areas

◊ Organization of ◊ Decision making ◊ Understanding crim-

◊ Integration of different sources of information.

◊ It must have its financial sustainability guaranteed.

Table 2. Information Sources, Objects of Intervention and Key Indicators1

Objects of Source of Information Examples of Indicators1

Police records Detainees (presumed perpetrators)

Source: Own formulation.

3.2. INDICATORS ON CITIZEN SECURITY

“An indicator is a numerical relationship that is established between two or

◊ Homicide rate per 100,000 inhabitants.

◊ Suicide rate per 100,000 inhabitants.

◊ Death rate by traffic accident per 100,000 inhabitants.

◊ Death rate by firearm per 100,000 inhabitants.

◊ Rate of reports of sexual crimes per 100,000 inhabitants.

◊ Rate of reports of intrafamily/family violence per 100,000 inhabitants.

◊ Rate of reports of child and adolescent abuse per 100,000 inhabitants.

◊ Rate of theft per 100,000 inhabitants.

◊ Rate of robbery per 100,000 inhabitants.

◊ Rate of kidnapping per 100,000 inhabitants.

◊ Rate of robbery and theft for every 100,000 registered cars.

◊ Prevalence of criminal victimization in people over 18 years of age.

◊ Prevalence of intrafamily, family and domestic violence.

◊ Prevalence of sexual violence.

◊ Levels (percentages) of perception of insecurity in people over 18 years of age.

◊ Levels (percentages) of trust in public institutions, especially police and justice.

3.3. ADVANTAGES AND DISADVANTAGES OF

Figure 1. Information on the Occurrence, Detection and Prosecution

Estimated Crimes Victimization Polls

Registered Crimes Police Statistics

Conviction Judicial Power

Source: Dammert et al., 2010

◊ Public institutions act as guarantors of the rigor of the data.

◊ They collect information about a large number of variables.

◊ There is information available regarding different geographical levels (national,

◊ The records of the different organisms may not be comparable.

◊ Access to this type of information may be limited or partial.

◊ The data can be used for political purposes.

◊ The processing of this type of data tends to take time.

Ultimately, for purposes of comparability between different sources of information at