Professional Documents
Culture Documents
3 INFORMATION AND
EVALUATION SYSTEMS
Inter-American Development Bank (IDB)
Course: Leaders for Citizen Security and Justice Management
Module author
Liliana Manzano Chávez
Researcher and Professor, University of Chile
This course was developed by the IDB in collaboration with the Academic Consortium of the University of Chile, John
Jay College of Criminal Justice, and Santa Maria School of Law (FADISMA).
Inter-American
Development Bank
September 2018
1300 New York Ave NW, Washington, DC. 20577
http://www.iadb.org
Copyright © 2018 Inter-American Development Bank. This work is licensed under a Creative Commons IGO 3.0
Attribution-NonCommercial-ShareAlike (CC BY-NC-SA 3.0 IGO) (http://creativecommons.org/licenses/by-nc-nd/3.0/igo/
legalcode) and may be reproduced with attribution to the IDB and for any non-commercial purpose in its original or in any
derivative form, provided that the derivative work is licensed under the same terms as the original.
Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration
pursuant to the UNCITRAL rules. The use of the IDB’s name for any purpose other than for attribution, and the use of IDB’s
logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as a part
of this CC-IGO license.
Note that link provided above includes additional terms and conditions of the license.
The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Inter-
American Development Bank, its Board of Directors, or the countries they represent.
INDEX
1. Presentation........................................................................................................................5
2. Introduction .......................................................................................................................8
3. Information Systems on Citizen Security......................................................................10
3.1. Use of Information Systems..............................................................................................10
3.2. Indicators on Citizen Security..........................................................................................13
3.3. Advantages and Disadvantages of Information Sources........................................14
4. Sources of Information in Latin America and the Caribbean ..................................18
4.1. Criminal Acts or Offenses..................................................................................................18
4.2. Perpetrators............................................................................................................................26
4.3. Victims .....................................................................................................................................34
4.4. Violence ..................................................................................................................................40
4.5. Consequences of Crime and Violence...........................................................................50
5. Evaluation of Citizen Security Policies..........................................................................54
6. Methodologies and Indicators for the Evaluation of
Citizen Security Programs...............................................................................................61
6.1. Ex ante Evaluation ...............................................................................................................61
6.2. Design Evaluation and the Use of the Logical Framework Tool............................63
6.3. Processes Evaluation...........................................................................................................67
6.4. Results Evaluations..............................................................................................................70
6.5. Impact Evaluation.................................................................................................................72
7. Conclusion .........................................................................................................................79
8. Glossary...............................................................................................................................82
9. Bibliography........................................................................................................................87
Optional Reading....................................................................................................................96
1. PRESENTATION
LEARNING OBJECTIVES:
GUIDING QUESTIONS:
2. INTRODUCTION
Many recent studies show the growth that the phenomena of violence and crime have
experienced in Latin America in recent decades. As a result, security has been placed
among the priority issues of citizen demands and in the political agendas of the govern-
ments of the region. However, various analysts, both from an academic perspective as
well as from political spheres and public opinion, consider that the response from gov-
ernments and public institutions in the region to this problem has been insufficient, and
sometimes ineffective or with short-term impacts, pointing out important deficits in the
design, management and evaluation of security and justice policies (UNDP, 2013; CAF,
2014; Frühling, 2012; Dammert et al. 2010).
In order to perfect the policies that seek to solve this complex situation, it is necessary
to develop better quality information systems, both administrative registers and other
autonomous measuring instruments that record data on the concept or phenomenon
being studied, which allow making diagnoses according to the regional, national and
local reality, improve management and promote the evaluation of citizen security poli-
cies (Beato, 2005; Frühling, 2012, Dammert et al., 2008; Salazar, 2013).
In the first part of this module we will look at the main characteristics of the information
systems developed to collect data and provide useful information for the diagnosis of
the situation, as well as the systematization of experiences and the analysis of results
in the field of citizen security and justice in the region. These systems aim to guide the
design of policies, accompany their implementation and provide information for their
evaluation.
In the second part we will explore the main components of the different models or types
of evaluation, the methodologies and tools that are applied in each type of evaluation,
and the most recommended indicators in each case, particularly with regard to the anal-
ysis of programs and projects related to citizen security policies. These are introductory,
but above all practical, content that will allow the director or public manager in charge of
a specific program or project to propose guidelines for its evaluation, either at the design
stage, during its execution, or once it has ended.
Module 3:
10 Information and Evaluation Systems
3. INFORMATION SYSTEMS
ON CITIZEN SECURITY
Public security statistics systems, defined as “the set of statistical information on the
subject, which, for the process of production, management, analysis and dissemination,
requires the interaction and coordination of institutions and organizations that are
linked together” (ECLAC, 2014: 9), allow decisions to be made from a technical perspec-
tive, in interdependent spheres of public administration, and with respect to different
activities, as shown in the following diagram.
Research in Public
Administration Planning
Policies
Source: Own elaboration based on the United Nations (2003) and ECLAC (2014)
Information systems can therefore be useful at different stages of the public policy
process, from identification, planning and design, to implementation and evaluation.
However, statistical information records must meet criteria to ensure their quality. The
Office of the United Nations (2014) highlights the following:
Leaders for Citizen Security
and Justice Management
11
◊ The information must be timely and credible, in order to facilitate collection and
achieve legitimacy.
Security policies address complex social phenomena, such as crime and violence. To
facilitate the definition of these policies it is therefore necessary to study these phenom-
ena in depth, considering the different aspects involved in them. It is necessary to study,
for example, criminal acts, perpetrators and victims, violence, and the consequences of
these problems. For information systems, each of these aspects implies an object that
needs to be identified, measured and described. But these “objects of measurement” are,
in turn, the main “objects of intervention” with respect to which security and coexistence
policies are formulated.
These phenomena or measurement objects are usually analyzed from multiple sources
of information, which provide us with various indicators, as listed in Table 2.
1 The indicators presented in this table are only some examples of the indicators that can be used in each area or subject
of intervention of the security policies; they do not correspond to an exhaustive registry of all the possible indicators,
since these vary between countries and even within a country - between states/regions - according to the availability of
information of each institution.
Module 3:
12 Information and Evaluation Systems
What do we mean by indicator? How are they formulated and classified? These are
some of the topics that we review in this section.
The indicators, in terms of citizen security, can have a variety of sources and be con-
structed by different procedures, but, undoubtedly, the ones used most are those from
administrative records (police, judicial and penitentiary system), as well as victimiza-
tion surveys. The indicators allow us to understand the magnitude, distribution (ter-
ritorial and population) and evolution (temporal) of phenomena such as those already
mentioned.
Worth noting among the indicators of crime and violence from administrative records are:
Besides these, the following indicators can be obtained from surveys on the subjec:
Quantitative information for the assessment of the management of public agencies, such
as performance indicators and effectiveness of institutional work, satisfaction and pub-
lic trust, etc., should also be added to these indicators. Similarly, it is possible to measure
economic indicators, which allow us to evaluate the cost of the programs or projects that
are implemented. For example: public spending on security programs, cost effectiveness
and cost of crime, among others.
To find out more about the construction of indicators, consult the reading Construction
of Indicators (optional reading).
Among the sources of information available to analyze phenomena in the field of security,
such as those already mentioned, we can consider: victimization surveys and statistics
recorded by the police, the judicial system and the prison system, among others. These
sources allow us to analyze the occurrence of crime and the different stages related to
its detection and prosecution: from the experience of victimization, reporting, deten-
tion, to the possible conviction within the prison system.
As illustrated in the figure proposed by Dammert et al. (2010), when going from one stage
to another, the volume of information about crimes and the actors involved decreases,
while the complexity of recorded information increases.
Leaders for Citizen Security
and Justice Management
15
Each of the sources of information has advantages and disadvantages, which should be
considered when using this type of data. The advantages and disadvantages presented
by administrative information records are set out below.
Among the advantages or strengths of official records are the following (UNDP, 2013;
Maguire, 2012; Dammert et al., 2010).
◊ The fields and concepts used are usually standardized and vary little over time,
which allows the analysis of trends and comparisons in the area to be made.
◊ There is a systematic record of information. This enables the analysis of time series
or trends.
However, this type of source also has disadvantages or limitations that must be taken
into consideration for its proper use. Among the weaknesses, the following stand out.
(UNDP, 2013; Maguire, 2012; Dammert et al., 2010):
◊ Not all crimes committed are reported, meaning there is a dark (hidden) figure2 of
criminality.
◊ Not all the actors involved in the collection and analysis of this type of data have
the same criteria.
◊ The trust that exists in the police and in the Public Ministry determines the volume
of information collected.
2 Dark figures are those crimes that occur in a given area and time, but do not become known to the police because they
are not reported or detected in flagrante delicto.
Leaders for Citizen Security
and Justice Management
17
Source: Westfelt and Estrada (2005), cited in Frühling and Mohor (2009: 6).
Module 3:
18 Information and Evaluation Systems
The following are the main sources of information available in Latin America and the
Caribbean (LAC), which provide information on the following aspects: a) criminal acts,
b) perpetrators, c) victims, d) violence and e) consequences of crime and violence. For
each object of intervention, one or two of the most relevant information sources are
described, and their main characteristics, indicators, purposes, usefulness, advantages
and disadvantages are analyzed.
When talking about crime, the debate usually focuses on the criminal acts, those that
correspond to “an omission or intentional act in violation of criminal law, committed
without defense or justification and sanctioned by the state as a felony or misdemeanor”
(Tappan, 1947: 100). This definition considers criminal acts as those classified as felonies
in each country, i.e., that violate legal provisions defined and classified within each coun-
try or each state (in the case of federated countries).
To organize the available data and carry out the desired analyzes, the crimes must be
classified. At national level, a uniform classification of crimes based on legal codes facil-
itates the comparative analysis of data and the exchange of information between the
different institutions of the criminal justice system. At regional level, standardized defi-
nitions and classifications are necessary for the comparability of records and for analyz-
ing regional trends.
Information regarding criminal acts or felonies can be obtained through multiple sources
according to the specific object or indicator to be analyzed. The main sources used in the
countries of the region to analyze crimes are: police records, judicial system records and
victimization surveys. In this section, we will address the first two and the third will be
looked at later.
The administrative records from police institutions correspond to the most widely used
sources of information in the field of security policies, because they present multiple
advantages (Dammert et al., 2008). They have stable indicators over time, collected
under standardized systems, applied routinely and they respond to broad territorial
coverage.
From reports filed by the community, as well as their own procedures (controls, arrests,
investigations), the police register a large amount of information regarding allegedly
criminal acts. These registers usually include types of crimes, characteristics of the event
(time, place, use of weapons, etc.), actors involved (victims and potential perpetrators),
as well as offences and disorders reported by the community.
Nevertheless, most police institutions only publish aggregate statistics of reports and
arrests, and these records have a dark figure. Dark figures are those crimes that occur in
Module 3:
20 Information and Evaluation Systems
a given area and time, but that do not become known to the police because they are not
reported or detected in flagrante delicto. As an example, homicides and theft of property
show a higher level of reporting than other crimes and, therefore, a lower dark figure
(Dammert et al., 2008).
There are four factors that can affect, to a greater or lesser extent, the administrative
records of police sources: the list of reportable crimes, the rules for counting crimes, the
conduct of police reporting and the conduct of citizen reporting.
List of reportable crimes: defines what crimes are included when compiling
statistical registers. This list may be modified according to legal adjustments,
which may result in decriminalizing an offense or defining another one as a
crime.
Rules for counting crimes: indicate how to proceed to count offenses repeated
by the same offender, how many and which of the offenses should be recorded.
Citizen reporting conduct: whether citizens report the criminal acts of which
they witness, or victims depends on a set of variables. Included in these are
trust in police institutions, the belief that the perpetrator will be found, or the
property lost will be recovered, reports for insurance claims.
Usually, the information collected by the police is published according to two indicators:
reports and arrests. This information is presented, according to certain time intervals
(annual, quarterly) and certain territorial divisions (country, states, regions, districts,
etc.). The reports are usually restricted to crimes of greater volume and impact for the
community, including robbery, property theft, violent robbery, injury and homicide.
These crime indicators are usually expressed in terms of absolute frequencies or in rates
per 100,000 inhabitants. Using rates makes it possible to compare territorial units that
have different population sizes.
For example, in the Region of Maule (Chile) the amount or frequency (absolute) of police
cases for crimes of Greater Social3 Connotation for 2015 was ten times greater than the
3 According to the Undersecretary for Crime Prevention, interpersonal or violent crimes, such as robbery with violence,
robbery with intimidation, robbery with surprise, injuries, homicide and rape, are considered among the most socially
charged crimes. In addition, crimes against property are included: theft of a vehicle, theft of a car object or an object
from inside the vehicle, theft in an inhabited place, and other robberies using force and theft.
Module 3:
22 Information and Evaluation Systems
number of police cases for the same group of crimes in the Aysén Region. However, when con-
sidering the size of the regional population it should be noted that the rate of police cases per
100,000 inhabitants is practically the same for both regions. This means that the magnitude of
the criminal phenomenon is similar in both regions.
Table 4. Frequency and Rate of Police Cases for Crimes of Greater Social
Connotation in Chile, in 2015 and 2016.
Note: includes official data from Carabiniers of Chile [Chilean national police force] and Investigations Police of Chile
Source: Center for Crime Studies and Analysis (CEAD), Ministry of the Interior and Public Security of Chile.
Additionally, the analysis of criminal acts based on police statistics offers the possibility of geo-
referencing. Georeferencing techniques include the geographical location, based on coordinates,
of the crimes that occurred. It identifies places with a high incidence of crime and focuses police
resources, as well as crime prevention strategies. For more information, access the Geographic
Information System (GIS) reading (optional reading).
Leaders for Citizen Security
and Justice Management
23
A second source of information on criminal acts are the administrative records of the
judicial system, from the Public Prosecutor’s Office, the National Prosecutor’s Office or
the National Attorney General’s Office, according its name in each country. In addition,
in some countries information can also be obtained from the Public Defender’s Office,
the Ombudsman, courts and tribunals that may differ (labor courts, family courts, courts
of appeals, local courts, etc.), forensic institutions and the prison system.
Such information is usually more complete than that contained in police sources with the
full process of a case in the criminal justice system: ranging from admission, the study
of the characteristics of the specific act and those of the victim and the perpetrator,
right through to the prosecution, its ruling and types of rulings (absolution, sentences
and types of penalties) 4. Additionally, judicial records enable the study of the effective-
ness and efficiency of procedural management carried out by the justice system (Mohor,
2007). For example, the efficiency of judicial proceedings can be monitored through the
average duration of the cases.
Despite these attributes, judicial records also have limitations. One of the main draw-
backs is that the withdrawal by the victims can impact the recorded numbers. For exam-
ple, excessive bureaucracy and slow progress of the processes make the victims drop
the criminal charges. Additionally, in cases of violent crimes, for example, child abuse
or sexual crimes, the system does not have mechanisms to avoid repeating statements
and offers few protection mechanisms, which often also results in the withdrawal of the
charges (Mendes de Souza et al., 2014).
4 This type of information is available when the bodies in charge of the criminal process are adequately organized to pro-
duce this type of data.
Module 3:
24 Information and Evaluation Systems
On the other hand, in many countries of the region, judicial registries continue to respond
to administrative or institutional criteria that limit their systematization in an accessible
and comprehensible manner. For example, the volume of cases entered into the judicial
system is specified, but the types of crimes or other relevant characteristics of the cases
or the accused are not always stated, which would allow for better criminal and crimino-
logical analyzes. Worse still, in most cases, judicial statistics are simply not available to
the public (Dammert et al., 2010).
Access to public information has several advantages with the main ones being its func-
tioning as an instrument of democratic control of public institutions, and its impact on
the way that individuals relate to public administration. From an overall perspective,
this serves for purposes of transparency, accountability and public policy design. From
an individual perspective, it allows citizens to obtain and use data for various personal
interests, such as academic research (Síndic de Greuges de Catalunya, 2012).
As an example, it is worth highlighting the volume, quality and periodicity of the informa-
tion disseminated by the Public Ministry of Chile and the Public Ministry of Peru. These
institutions have information which includes the cases distributed geographically, types
of perpetrator, victim profiles, types of crimes, applied sentences and the process man-
agement, etc. This information is of high quality insofar as it is standardized information
at a national level, responds to the legal typology of each crime and is delivered peri-
odically. These standards allow the information to be used for multiple purposes, both
academically and for public policy.
Leaders for Citizen Security
and Justice Management
25
Number of rulings
The administrative records of judicial institutions are usually more complete than those
of police sources, since they have the full flow of a criminal case entered into the jus-
tice system. However, excessive bureaucracy and slowness of the processes, as well as
the scarce protection mechanisms, leads to the withdrawal by victims of crime that can
impact the recorded numbers.
4.2. PERPETRATORS
In addition to the criminal acts, the perpetrators are also an important object of inter-
vention for citizen security. Throughout history, much of the development of the crim-
inological discipline, the construction of theories and the performance of empirical
studies have sought to find out and understand the characteristics and motivations of
the law offenders or perpetrators.
In particular, the intended goal is to understand and describe the demographic profile
of the perpetrators, the type of crime committed and the circumstances under which
the event occurred, and especially the causes or motivations behind these events.
Independent of the theoretical perspective under which the criminal behavior is inter-
preted, at present most of the academics and designers of public policies coincide in the
fact that the causes and motivations to commit a crime are multiple and, therefore, it is
necessary to have diverse sources of information (and multiple indicators) that contrib-
ute to a more detailed characterization of the perpetrators.
Only through analyzing the demographic, social and psychosocial characteristics of the
condemned subject, as well as their criminal history (criminal career) and the elements
involved in the crime, is it possible to come closer to, in some measure, a greater under-
standing of the possible factors that led this person to commit a crime, continue his/her
criminal career or to desist from it, initiating a process of reintegration into society.
Official records from the police and judicial systems provide information about alleged
perpetrators or defendants, since when speaking of perpetrators, they must have been
convicted of a crime.
To this extent, prison systems, in their different areas or subsystems (charged or con-
victed incarcerated population, population condemned by type of conviction or by type
Leaders for Citizen Security
and Justice Management
27
of prison), are the only institutional source that allows for the analysis of the perpetra-
tor’s profile.
Below, we will discuss the type of information provided by the penitentiary systems.
The statistics of the penitentiary system allow for the characterization of the prison
population in global terms and with respect to specific dimensions, including the prison
population according to crimes which they are serving sentences for, types of sentences,
the system in which the sentence is served (closed or semi-open), the population with
precautionary measures, the population with prison privileges, sentence time, previ-
ous criminal history, and demographic characteristics (sex, age, nationality, occupation,
education).
The main indicator for diagnosing the phenomenon of crime and violence in the region,
which can be extracted from these sources, is the prison population per 100,000 inhab-
itants, which not only accounts for the levels of crime and violence in the country, but
also the rate of incarceration. However, other indicators, such as overpopulation of the
system or admission to serve time with new sentences are also relevant.
In Latin America, most countries make administrative records about their prison sys-
tems. However, access to this information is not always public. Another weakness that
records of the penitentiary systems present is that the registration criteria are not stan-
dardized in most cases, which prevents comparative analysis at regional level.
However, international organizations such as the Institute for Criminal Policy Research
(ICPR) contribute towards greater standardization, comparison and dissemination of
prison system indicators worldwide, representing a useful source of information for the
countries in our region.
Module 3:
28 Information and Evaluation Systems
The ICPR provides an online database with information about the world’s
prison systems. On its website, it is possible to find an updated report for each
of the countries in the region with indicators such as:
◊ Number of prisons.
Source: http://www.prisonstudies.org/
Given the deficiencies of the official prison records, and the difficulties that exist to
access this information and to learn about the profile of the perpetrators serving sen-
tences, there are several quantitative and qualitative studies on this population, which
allow access to more detailed and complex information about those who have committed
crimes. Among them are the census of the prison population and the self-report surveys.
The prison population census reports information on people held in penitentiary cen-
ters such as the prison population, distribution by sex and age, nationality, marital status,
fertility, educational level, peer group, health, alcohol and drug use, type of crime, ethnic
origin, disability, religion, employment, criminal behavior, living conditions in prisons and
inmate population by custodian-guardian, among other data.
Leaders for Citizen Security
and Justice Management
29
The National Institute of Statistics and Computing of Peru publishes some data
on the perpetrator profile on its website, which was obtained from the First
National Penitentiary Census:
◊ 132% overpopulation.
◊ 24.6% of the penitentiary population consumed drugs during the last year.
Information gathering programs in the prison system have been promoted in the United
States and some European countries. For example, about every six years the BJS (Bureau
of Justice Statistics) of the U.S. Government Department of Justice (DOJ) carries out the
survey on the inmates in detention - one for sentenced prisoners and another one for
prisoners on trial - with 10,000 inmates in each survey (www.bjs.gov). A survey about
the social climate in prisons is also carried out annually (Prison Social Climate Survey
-PSCS-) on inmates and, separately, on prison staff, in order to evaluate the management
of each prison unit.
In LAC, there is limited experience in applying this methodology. In addition, these have
not been institutionalized as a system for collecting information which is regularly
applied, mainly because the application of these instruments entails high financial costs,
but also because of the ethical implications presented by studies carried out with the
incarcerated population. In 2013, for the citizen security report of the United Nations
Development Program (UNDP), penitentiary surveys were managed in the province of
Buenos Aires (Argentina), the state of São Paulo (Brazil), the central regions of Chile, and
national surveys in Peru and El Salvador. Also, since 2002 there have been four surveys
on inmates in prisons in the Federal District (D.F.) and the State of Mexico (2002, 2005,
2009, and 2013), and a survey on inmates in federal prisons in Mexico (2013). (Bergman
et al., 2014).
Leaders for Citizen Security
and Justice Management
31
The sample considered people between 10 and 25 years old from households
in England and Wales, which is distinct from most surveys of this type in the
United Kingdom which are usually applied to people serving a sentence for one
or more crimes.
General Objective
Pre-requisites
To consider some central aspects that can determine the success or limitations
of a survey and especially a census. For example, if it is not possible to access a
penitentiary facility when a census is taken, valuable information is lost, and no
other information can be used in its place.
4.3. VICTIMS
Victimology is the branch of criminology that focuses on the study of victims. It charac-
terizes them, evaluates the causes and consequences of victimization, the interaction of
victims with the justice system, and studies the prevention of crime (Daigle, 2017).
Nowadays, the sources of information regarding victims in LAC come mainly from offi-
cial records (police, judicial and forensics) and victimization surveys.
Police records in LAC have always featured information concerning the victims of crimes.
However, for a long time, this information has been restricted to basic aspects, e.g., the
demographic profile, aspects related to the reporting of the crime without going into
depth.
Also, this information was rarely systematized and distributed because, to a large extent,
police and judicial work focused on the persecution of the offender and not on knowl-
edge about victims, their reality, or on the avoidance of going through their victimization
experience again (Pasara, 2015; Riego, 2014).
During the mid-2000s, after the beginning of the of Criminal Procedure Reform process
in most of the countries in the region, the role of the victim in the judicial process was
brought to attention. This led to the recognition of victims’ rights, the creation of victim
assistance units, and the systematization of statistics regarding them (Riego, 2014).
Leaders for Citizen Security
and Justice Management
35
Presently, several countries in the region have judicial and forensic records of crime vic-
tims. However, although in some cases this information is published on a regular basis,
its systematization and standardization are still limited, since in most case the reports
are, very technical and unintelligible. For example, monthly and annual newsletters
on performed procedures are issued at the National Institute of Forensic Sciences of
Guatemala 5, but these are based on medical definitions. The National Institute of Legal
Medicine and Forensic Sciences of Colombia, on the other hand, publishes reports con-
taining information about common criminal definitions which is summarized with easily
understandable graphs.
2016 2017
Manner of
death
Male Female Total Male Female Total
Transportation
3.572 841 4.413 3.395 780 4.175
accident
Source: Monthly statistical newsletters, Instituto Nacional de Medicina Legal y Ciencias Forenses
[National Institute of Legal Medicine and Forensic Sciences].
http://www.medicinalegal.gov.co/boletines-estadisticos-mensuales
Despite the fact that official records have information about the victims of crime, in
terms of volume, this information remains insufficient since most of the people who suf-
fer from crimes fail to report the facts to the authorities.
5 http://www.inacif.gob.gt/index.php?option=com_content&view=article&id=96&Itemid=2
Module 3:
36 Information and Evaluation Systems
Between the 1970s and the early 1980s, the first victimization surveys were imple-
mented as a mechanism to fill the information gaps created by the non-reporting of vio-
lence. Between mid and late 1990s, these surveys were implemented in Latin America.
According to the “Inventory of Victimization Surveys in Latin America and the Caribbean”
(CDEUNODC, 2013), out of 23 Latin American countries consulted, 17 countries and
two cities 6 implemented at least one victimization survey, but only ten countries apply
this type of surveys on a regular basis at national level.
With regard to methodologies, the inventory optimistically acknowledges that given the
similarity of recorded crimes and the methods applied, it is possible to conclude that
progress has been made in the region towards a greater standardization of these instru-
ments for generating comparable data.
However, in order to extend the production and use of this type of instrument a num-
ber of obstacles present in some of the countries of the region will have to be overcome
(Aromaa, 2012; Lagos and Dammert, 2012):
6 Bolivia, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Guyana, Honduras, Jamaica, Mexico,
Panama, Peru, Dominican Republic, Uruguay, and Venezuela. Cities of Buenos Aires and Quito.
Leaders for Citizen Security
and Justice Management
37
◊ Distrust of system stakeholders: these stakeholders usually see these new sources
as a threat.
Victimization surveys, as well as other sources of information, have strengths and weak-
nesses. The main ones are:
Advantages Disadvantages
◊ They allow for the recording of ◊ They are not very sensitive at measuring
a greater number of crimes than violent crimes (domestic violence, sexual
official records. crimes), especially when these are commit-
ted by acquaintances.
◊ They are necessary to capture
crimes suffered by individuals ◊ They are not sensitive at measuring crimes
and members of the household and offences that affect groups/communities.
and are applicable to the entire
population. ◊ They are an inadequate measure of orga-
nized crime (human trafficking, drug traffick-
◊ They are adequate instruments ing and other crimes).
for measuring crimes on the
public streets and committed by ◊ They involve methodological biases derived
strangers. from sampling, the instrument, field work,
response rate, inaccurate counting and cod-
◊ These instruments capture a wide ing, among other factors.
range of information including:
victimization, characterization of ◊ They have margins of error, so in less fre-
the victim, fear of crime and trust quent crimes or in subsampled groups, less
in police institutions. accurate results are obtained.
Source: Own elaboration from UNDP (2013); Aromaa (2012); Maguire (2012).
Module 3:
38 Information and Evaluation Systems
As highlighted in the above-mentioned report, so far only two countries in the region
have managed to overcome these obstacles and institutionalize their victimization
surveys at the national level: Chile and Mexico. Chile’s National Urban Safety Survey
(ENUSC, according to its Spanish acronym) is the only one that has implemented the
instrument on a yearly basis in the region. From 2003 to 2016, 12 versions have been
applied. Only in 2004, as an exception, the survey was not conducted. For more infor-
mation regarding Chile’s experience, you can read “Buenas prácticas en los sistemas de
información” [Good Practices in Information Systems] (optional reading).
In Mexico, the National Survey of Victimization and Perception of Public Safety (ENVIPE,
according to its Spanish acronym) has been implemented on a yearly basis from 2011 to
the present day. The previous version of this instrument, the National Insecurity Survey,
was implemented seven times between 2002 and 2010. (CDEUNODC, 2013).
The following graph shows the countries which carry out national or regional surveys
on citizen security or victimization, highlighting the year in which the implementation
process began.
Leaders for Citizen Security
and Justice Management
39
Source: Own preparation based on information provided by the Center of Excellence for Statistical Information on Government Public
Safety, Victimization and Justice (CDEUNODC); the Hemispheric Security Observatory of OAS, and various official sites (electronic links
in the bibliographical references are at the end of the reading).
Module 3:
40 Information and Evaluation Systems
For the region, the challenges of implementing periodically the national victimization
surveys, ensuring they maintain consistency of methodology over time which allows
them to obtain quality and representative information from each country still remain.
Along with these efforts, work should be done towards the construction of standardized
instruments that address crime categories and similar issues. This will make the com-
parability of data between countries in the region easier, identifying similar patterns of
behavior, levels of victimization, particular uses of violence, among other aspects. For
more information, consult the reading [Criteria of quality or technical rigor] (optional
reading).
Additionally, it is worth highlighting the periodic deployment at the regional level of two
public opinion surveys: Latinobarómetro and Proyecto de Opinión Pública de América
Latina [Latin American Public Opinion Project] of Vanderbilt University (LAPOP). These
surveys facilitate the comparative analysis of topics such as victimization, fear percep-
tion and trust in the criminal justice system institutions, although because of the reduced
size of their national samples, they are not useful for an analysis at national/local level.
For more information about these surveys, consult the reading [Examples of public opin-
ion surveys] (optional reading).
4.4. VIOLENCE
In Latin America, between the 1990s and the 2000s, studies on violence focused on
the analysis of homicide or lethal violence, with high rates of this crime observed in the
region, particularly in countries that had faced (or were still facing) internal armed con-
flicts—e.g., Nicaragua, El Salvador, Guatemala and Colombia—but also in countries with
high levels of criminal violence, such as Venezuela. (Fajnzylber et al., 1998; Arriagada and
Godoy, 1999). In these analyzes, an epidemiological approach was adopted using con-
cepts from public health, which considered this phenomenon as an epidemic plaguing
the region (Concha-Eastman and Guerrero, 1999; Londoño and Guerrero, 1999).
Leaders for Citizen Security
and Justice Management
41
In studies of that period, but also in more recent ones, the most commonly-used indi-
cators were frequencies and rates of complaints and apprehensions for crimes such as
homicides, injuries, injuries with the use of weapons, crimes associated with trafficking,
and carrying weapons.
Most of these studies have used data from police and judicial records, despite all the
problems they faced, such as those mentioned previously (low levels of reporting, lack of
standardization in indicators, lack of comparability between sources, lack of periodicity
and distribution of information). Only on a few occasions was it attempted to supple-
ment such information with ad hoc survey data, given the lack of periodic victimization
surveys. This has changed in the last decade with the expansion of information sources
(UNDP, 2013, CAF, 2014, Vilalta, Castillo and Torres, 2016).
Below, we will explore three of the main indicators for measuring violence in LAC: the
homicide rate, the intrafamily and gender violence rate, and the femicide rate.
The homicide rate has been the main indicator used to analyze both the magnitude and
characteristics of this phenomenon within a country and to compare the reality among
countries in the region (UNDP, 2013). It is expected that on the basis of these analyzes,
the potential causes of lethal violence will be established, and public policy strategies
will be proposed at regional and local levels (CAF, 2014).
In this context, reports issued by international organizations, such as the “World Reports
on Homicide”, published by the United Nations Office on Drugs and Crime (UNODC),
which compare homicide rates worldwide should be mentioned. Nevertheless, the focus
is mainly on Latin America’s situation. In the most recent report (2013), data from offi-
cial, police, judicial or forensic sources are presented. The difference between sources
of information is relevant to the extent that each institution has different systems of
records and there is no similarity in the measurement criteria, so the comparison of
homicide rates between countries might not be accurate.
Module 3:
42 Information and Evaluation Systems
Figure 2. Map of Homicide Rates in the World (data for 2012 or last
available year)
The UNODC report of 2013 reported a worldwide homicide rate of 6.2 cases per
100,000 inhabitants. Considering this world average, the previous map shows that sev-
eral countries in the region have a homicide rate of over 30 cases per 100,000 inhabi-
tants—Venezuela, Colombia, Guatemala and El Salvador—, while countries such as Chile,
Peru and Uruguay have a rate that does not exceed 5 cases per 100,000 inhabitants.
According to the report by UNDP (2013), over the last decade the deadly violence in
the region increased by 12%, and in 11 of the 18 countries in the region discussed in the
report, the deadly violence can be described as an “epidemic”, according to the crite-
ria of the World Health Organization (WHO), with a rate higher than 10 homicides per
100,000 inhabitants.
Another source of information that offers updated and compared data on homicides
worldwide, also expressed in georeferenced maps, is the Homicide Observatory of the
Brazilian Igarapé Institute. The most recent information reported by the site indicates
that in 2017 of the ten countries with the highest world ranking in terms of homicide
rates, eight are in the region.
Leaders for Citizen Security
and Justice Management
43
Source: Map downloaded in April 2018, from the Homicide Observatory site of the Brazilian Igarapé Institute
https://homicide.igarape.org.br/
Despite the extensive use of the homicide indicator, it should be noted that it is not the
only nor the best way to represent the real magnitude of a problem so complex as vio-
lence, especially in our region, where there are still many deficits in terms of quality of
the information provided by official sources. In the diagnosis made by Ribeiro, Borges
and Cano (2015) it was found that homicide figures are not always comparable between
countries, and not even within the same country. This is due to differences in the defini-
tions of a homicide and the unit of analysis, under-reporting and a coverage gap in police
institutions, deficiencies in the systematization of the data, in the handling of residual
categories and lost cases, among other factors.
In order to develop this diagnosis and seek solutions, the “Conference on the Quality
of Homicide Data in Latin America and the Caribbean” was held in the city of Bogota,
Colombia, in September 2015. The conference brought together members of criminal
Module 3:
44 Information and Evaluation Systems
justice, statistical and academic institutes with civil society from 12 countries in the
region. The main result was the drafting of a Protocol on Homicide Data Quality. The
document defined criteria and thresholds which were grouped into eight thematic areas:
a) registration unit, b) definition of a homicide, c) minimum fields to record, d) lost data
and indeterminate cases, e) convergence between sources, f) mechanisms of verification
and validation of data, g) disclosure and transparency, and h) periodicity and opportunity.
For more information regarding the Protocol, you can access the available video
(http://conferenciahomicidiosbogota2015.org)(optional resource).
As addressed previously, with the objective of improving knowledge about the victims
of crime, in the middle of the 2000’s the first surveys of victimization were implemented
in some countries of the region. The data generated through these instruments allows
us to learn in greater volume the amount of unreported crimes. However, as we have
seen, several authors have claimed that these standardized questionnaires are rela-
tively insensitive when it comes to measuring crimes committed by relatives or someone
known by the victim, or that occur in the context of an ongoing process, and not just at a
specific moment sometime, as happens in cases of domestic violence and sexual crimes
(Maguire, 2012; Hoyle, 2012).
Recognizing these shortcomings, countries in the region have recently developed and
implemented data collection instruments that seek to measure, various manifestations
of domestic violence and/or violence against women in a more sensitive way. These
instruments seek, on one hand, to provide a more complete figure for the occurrence of
7 This issue will be addressed in more detail in the “Prevention of Youth and Violence Against Women” module.
Leaders for Citizen Security
and Justice Management
45
these crimes and, on the other, to describe the socio-demographic profile of the house-
holds/women who suffer these acts, to investigate the nature of the violence (physical,
emotional, psychological and sexual), the characteristics of these acts (periodicity, repe-
tition), and the link between victim and aggressor.
Unfortunately, only a few countries in the region have implemented this type of instru-
ment (Argentina, Chile, Ecuador, Peru, Mexico, and Uruguay), and in most of these cases
they are single studies and not regular ones. Below, two examples of this type are high-
lighted; the first one is of intrafamily violence and the second one is of gender violence.
Module 3:
46 Information and Evaluation Systems
The ENDIREH is a survey resulting from the joint work of the National
Institute of Women (INMUJERES) and the National Institute of Statistics
and Geography (INEGI) in Mexico. It has been implemented on four occa-
sions (2003, 2006, 2011, and 2016) and its purpose is to scale and charac-
terize the dynamics of couple relationships in households, as well as the
experiences of women with different types of violence, in school, work, and
the community.
In 2016, the ENDIREH was applied to 142,363 women over 15 years old
in private households, using a probabilistic, three-phase, stratified and con-
glomerate sampling. The survey has urban national representativeness for
the three populations under study: married women or in union, women once
in union (now divorced, separated or widowed), and women never having
been in union (unmarried).
◊ Obstetric violence.
The survey considers specific forms of violence, such as having been kicked
or hit with a fist, being attacked with a weapon, being a victim of sexual
harassment, offensive comments, pinching, pulling of hair, slapping, touch-
ing, and other incidents of violence.
Leaders for Citizen Security
and Justice Management
47
Source: http://www.beta.inegi.org.mx/proyectos/enchogares/especiales/endireh/2016/
Module 3:
48 Information and Evaluation Systems
Based on the data gathered by the survey, it can be observed that 68.8% of
Uruguayan women, aged 15 or older, have experienced a situation of gender
violence throughout their lives. This prevalence is even greater in the case of
large cities such as Montevideo (76.9%). The National Council expects this
information will promote the design of more effective social policies in the
field of the prevention of gender-based violence and assistance to victims.
For more information, consult the Results Report of the First National Prevalence Survey on Violence,
based on Gender and Generations, p. 23
http://www.mysu.org.uy/wp-content/uploads/2015/08/PENPVBBG.pdf#page=9&zoom=auto,-107,713
Leaders for Citizen Security
and Justice Management
49
According to the study of Matthias Nowak (2012), 14 out of the 25 countries with the
highest rates of feminicides (also known as femicides) in the world are Latin American
countries. This manifestation of violence has therefore been considered a key indicator
for analyzing the magnitude and forms in which this phenomenon is expressed in the
countries of the region.
Although in most of the legislation of the region the concept of “femicide” is understood
as the murder of a woman in the context of a couple’s relationship, this definition differs
depending on the regulatory framework of each country. For example, while in Chile only
feminicide is recognized when it occurs in the context of marriage or formal cohabitation
(courtship is excluded), in Peru this crime is recognized in cases of homicide that occurred
in the context of intrafamily violence. According to Nowak (2012), in order to confront
the phenomenon with more effective public policies, it is necessary to first establish a
consensual definition at the regional level and, second, to make progress in addressing
the lack of registers and/or the poor quality of the existing data within each country.
Although only 15 countries in the region had laws on “feminicide” by 2014, the Gender
Equality Observatory in Latin America and the Caribbean (http://oig.cepal.org/es)
gathered information from the administrative records of the police, judicial and health
authorities of all countries in the region, concluding that at least 1,678 women were
murdered for gender reasons in that period.
Although the Observatory’s effort to account for these cases is remarkable, it is clear
that additional efforts are needed to put the problem at the top of the priorities to be
addressed by security policies, for which it is not only necessary to improve legislation
and public policies, but also to improve the mechanisms for its standardized, periodic,
comparable and public access measurement.
Module 3:
50 Information and Evaluation Systems
Jamaica (2015) 9
Trinidad and Tabago (2015) 5
México a
Santa Lucia (2015)
Barbados (2014)
1
2
countries of the region.
2 289
In 2014, Mexico was the
St. Vincent and the Granadines (2013) 1
Guatemala Granada (2013) 1
217 Suriname (2014) 7
Honduras
Dominica (2014) 1
country with the highest
531
Dominican Republic
188
number of femicides,
Puerto Rico
44
with 2,389, followed by
El Salvador
183
Panamá
26 Honduras, El Salvador,
Nicaragua
36 Costa Rica
the Dominican Republic
14
Colombia
Venezuela (Rep.Bol.de) and Guatemala.
74
145
Brasil (Sin información oficial) However, as the Central
COUNTRIES WITH THE
Ecuador
97
American countries have
HIGHEST RATE OF
FEMINICIDES IN THE REGION
a smaller population,
Perú
90 they have higher rates
HONDURAS than those of Mexico.
EL SALVADOR Bolivia (Est.Plur. de)b On the contrary, coun-
96
tries such as Uruguay
Paraguay
DOMINICAN REPUBLIC 32 NUMBER OF
GUATEMALA Uruguay FEMINICIDES (24 cases) and Costa
24
Chile
40 2014 Rica (14 cases) have the
lowest number of femi-
Argentina Feminicides
225 Female deaths apparently
with presumption by homicide
cide cases in the region.
Source: ECLAC, 2014
https://www.cepal.org/sites/default/files/infographic/files/3_feminicidio_final_20_oct_2016_correcta.pdf
* Mexico’s number is not comparable with the other countries in the region.
* Brazil’s number corresponds to 2015
We have already analyzed the subjects of investigation that refer to committing crimes
or violent acts and the actors involved in them. These subjects deal with the objec-
tive aspects of citizen security, which correspond to the levels of violence and crime.
However, as we have seen, there is also another dimension to the problem of citizen
security: subjective security. This considers “both the emotions and the representations
and behaviors of the public towards insecurity” (Otamendi, 2016: 62).
Leaders for Citizen Security
and Justice Management
51
Violence and crime have many consequences for people, ranging from the effects on
their behaviors and freedoms, how they relate to others (known and unknown people),
how they use public spaces, even the way in which state institutions are evaluated. Also,
beyond the effects on people and households, violence and crime have effects on the
economy of a country and even on democratic governance.
The epidemic situation of violence and crime in Latin America leads to the consequent
perception that public institutions are not effective which results in low confidence in
the institutions, particularly in the police and justice. As a consequence of this situation
of low confidence in the police, there is a deterioration in the relations between these
institutions and the community, making dialogue between both parties difficult, reduc-
ing the rates of reported cases and making police work more difficult.
The paradox between objective and subjective security in Latin America is that indica-
tors for both are not directly correlated. As an example, homicide rates for each country
are not directly correlated with the perception of security in those countries.
There are multiple definitions of fear of crime, but we can understand it as the emotional
response to the possibility of being a victim of crimes or violent acts. This is one of the
subjective aspects of insecurity. Fear restricts activities in public spaces and social activ-
ities, can cause psychological problems, changes in commercial activities and corruption.
This concept has been the subject of study since the seventies, thanks to the implemen-
tation of victimization surveys.
In Latin America, based on the information provided by the victimization surveys, it has
been observed that victimization is not perfectly correlated with the fear of crime. In
other words, inasmuch as victimization decreases, the fear of crime does not necessar-
ily diminish. This is clearly illustrated in the results of the Americas Barometer 2013, a
public opinion survey that includes questions about victimization and fear, as indicated
above.
Module 3:
52 Information and Evaluation Systems
35 Guatemala
25 Honduras
Intentions to migrate 11,4
Perception of insecurity 23,2
20
Victimization per crime
18,9
15 Costa Rica
2004 2006 2008 2010 2012 Intentions to migrate 10,1
Perception of insecurity 29,7
Intentions to migrate Perception of insecurity Victimization per crime Victimization per crime 17,5
Source: © Barómetro de las Américas por LAPOP
Panama
0 10 20 30 40 50
This graph shows the evolution in time of This graph shows the levels of victimization
both the victimization and the perception and perception of insecurity in six coun-
of insecurity, indicating that between tries of Central America. This allows us to
2004 and 2008 the decrease in the per- observe that those countries with higher
ception of insecurity was not associated levels of perception of insecurity are not
with a decrease in victimization levels. necessarily those with the highest victimiza-
tion rates.
Source: LAPOP (2014: 2-3). http://www.vanderbilt.edu/lapop/insights/IO901es.pdf
Knowledge about the costs of crime allows for the implementation of public security pol-
icies with measurable results based on their economic value. The costs of crime can be
classified into two types: those detectable for measuring and being monetarily calculated,
Leaders for Citizen Security
and Justice Management
53
and those that cannot be estimated this way. Isolated studies analyze this problem in Latin
America. As an example, UNDP (2013) analyzed the costs of crime in two ways, it first
calculates the accounting costs by estimating the monetary expenses directly associated
with insecurity, among which are the losses and damages associated with the crime and
the costs of containment, attention and prevention. Second, it estimates the years of life
lost due to homicides and their impact on Gross Domestic Product (GDP).
In Uruguay, the Economic Research Center carried out a study of this type in 2013, which
estimates the costs of crime prevention and the justice system, the monetary value of
stolen goods, health costs and the costs of deprivation of liberty. Based on this analysis,
the study concludes that the cost of crime in Uruguay rises to 3.96% of GDP, with homi-
cides and domestic violence representing the highest costs.
Sources: Cafferata (2011), Ospina and Giménez (2009), Brand and Price (2000), systematized by UNDP (2013: 102).
http://www.undp.org/content/dam/rblac/img/IDH/IDH-AL%20Informe%20completo.pdf
Module 3:
54 Information and Evaluation Systems
5. EVALUATION OF CITIZEN
SECURITY POLICIES
As acknowledged by authors such as Frühling (2012) and Sherman (2012), to ensure bet-
ter design and implementation of citizen security programs, particularly in the field of
crime prevention, it is necessary to have evidence on the effectiveness of interventions.
“To evaluate is to order and measure the reality, to organize it into categories
or variables that are then measured, either quantitatively or qualitatively. In
this sense, to evaluate is also to investigate a dynamic exercise that goes from
the theoretical-deductive (generation of categories or variables from existing
knowledge) to the empirical-inductive (quantification and qualification of cate-
gories or variables from data collected)” (Tocornal et al., 2005: 3).
Leaders for Citizen Security
and Justice Management
55
Those who make decisions require quality information to ensure that their actions are
adequate and that, based on these, the expected objective will be achieved. This require-
ment is solved through the evaluation of social policies or programs, which will enable
planning, designing and implementing policies and programs that reach, the problem of
intervention effectively and efficiently. With evaluation strategies it is possible to imple-
ment programs of greater coherence based on evidence, which are sustainable in the
long term and more effective.
◊ What is the cost of it? How are the costs related to the effectiveness of
the program and its benefits?
Although the aims of the evaluation can be multiple, the following table highlights the
main purposes from the point of view of their usefulness, feedback or improvement in
the programs or projects.
Module 3:
56 Information and Evaluation Systems
Function Description
To choose between different options or possible solu-
tions, allocating resources to those that are more effi-
Decisions on public policy cient and effective.
regarding the implementation
To establish the position of the actors in the debate that
of a program will revolve around in the program announce a possible
vote in the National Congress, in the Communal Council
or other collegiate body.
During implementation, the evaluation monitors the
processes involved in the programs and guides efforts
to improve them.
Rectification or modifications
Given that the evaluations provide their findings in a
within the program (learning from
documented form and often identify the causes of the
the implementers) problems that are affecting their performance, the
results of the evaluation studies are used by the people
who formulate the public policy to implement rectifica-
tions or reforms in the programs.
An evaluation at the end of a program will give an
account of the degree of compliance with the objec-
Decisions on the continuity,
tives, the effectiveness of it, and possible reasons for
termination or replicability of a
its success and/or failure. These elements will help to
program
decide if the program should be maintained, modified or
eliminated, and if it can be replicated in other contexts.
The information provided by the evaluation of pro-
grams will allow the authorities, citizens and represen-
Legitimization and accountability tatives to learn about how the interventions have been
towards the community implemented, if they reach their target population, and
the impact, results, benefits and effectiveness achieved
by them.
The contents presented below are structured based on one of the possible classifica-
tions of the evaluation typologies, which is related to the time of the project in which it
is implemented, added to the main purposes of these evaluations. In this way, the text
is organized into the ex ante evaluation (design, diagnosis), and the intermediate (pro-
cesses) and ex post (results and impact) evaluations.
Leaders for Citizen Security
and Justice Management
57
Project Evaluation
Definition Considerations
stage type
Source: Own elaboration based on Tocornal et al. (2005) and Olavarría (2007)
For example, the program “Improving citizen security in Panama: towards the social con-
struction of a culture of peace”, with the general objective of strengthening the installed
capacities for the management of public policies on citizen security (Arbulú, 2013) has
several types of evaluations. Between 2012 and 2013 the MDG Achievement Fund
carried out a comprehensive evaluation of the program aiming to establish the extent
to which the activities had been adequately executed, the products delivered, and the
expected results obtained. To accomplish this, three types of evaluation were devel-
oped: design, processes and results. The information-gathering techniques developed
for the evaluation were: documentary analysis, bilateral interviews, focus groups and
semi-structured interviews.
Leaders for Citizen Security
and Justice Management
59
In the design evaluation, the relevance of the program is established. In other words, it
determines the extent to which the program is consistent with the defined objectives.
The aforesaid report identified the following weaknesses of the program in terms of its
design:
◊ Programming problems.
◊ The stakeholders involved are coordinated, with their roles defined appropriately.
The evaluation of results establishes the extent to which the objectives have been
achieved (effectiveness) and the probability of maintaining the benefits of the program
in the long term (sustainability). The results of the evaluation in this area revealed the
following:
◊ Seminars, training activities, manuals and guides are undertaken to improve the
Module 3:
60 Information and Evaluation Systems
◊ Prevention, training and institutional capacity building activities are carried out on
issues related to children and youth.
Finally, the evaluation process identifies a set of good practices of the program, includ-
ing: a) the key role of intersectoral coordination, b) empowerment of local governments,
c) promotion of community participation, d) association of the program with private sec-
tor institutions. (For more information, consult Arbulú, 2013)
Leaders for Citizen Security
and Justice Management
61
For the purposes of this course, we have defined four key evaluation processes in the
implementation of citizen security policies or programs which we will explore below: ex
ante evaluation (diagnosis, design evaluation), process evaluation or monitoring, evalua-
tion of results and impact evaluation.
As we have seen, there are two basic types of ex ante evaluation: the security diagnosis
and the design evaluation. The economic evaluation is also a type of ex ante evaluation,
but it will not be explored in this module since it is outside the scope of the course.
Every project or social program starts with the following questions: What is the social
problem? Is a social condition really a problem? What are the characteristics of this
problem? What is the seriousness of the problem? All these questions must be resolved
with an appropriate diagnosis of the issue that will be intervened, one of the main tools
for the design and implementation of effective public policies. Therefore, the diagnostic
phase should be considered as a foundational part of the intervention process.
and quantitative analysis tools must be used. The in-depth knowledge of the problem
allows us to define priorities in the intervention strategies, their viability, the available
means and the actors involved.
There are several topics that we can analyze during the diagnostic evaluation, as summa-
rized in the table below:
For example, in Chile, the Communal Plan for Public Security Program of the Under sec-
retariat of Crime Prevention (SPD, according to its Spanish acronym) of the Ministry
of the Interior and Public Security aims to promote planning and management capaci-
ties in the field of security in the municipalities. Its purpose is to identify and prioritize
the security problems that most afflict the inhabitants of each community and, in this
way, establish a plan to meet these needs; the community teams have to develop citizen
security diagnoses. In accordance with the technical guidelines provided by the SPD, the
diagnoses have the following main objectives:
Leaders for Citizen Security
and Justice Management
63
◊ Prioritize and organize those problems into a hierarchy in order to define the
objectives of the Communal Public Security Plan.
◊ To understand and describe the main situations or phenomena that generate per-
ception of insecurity among citizens.
Another type of ex ante evaluation is the design evaluation, which consists in the assess-
ment of the formulation of the intervention project. According to Olavarría (2007),
before implementing the project, the design evaluation analyzes if it can effectively
address the social phenomenon which it is addressing.
This type of analysis takes into account the coherence between the problem identified
from the diagnosis, the population to be intervened with, the objectives, the activities
and their organization, and the planning of the processes. Additionally, the viability of
the intervention can be estimated on the basis of a design evaluation.
One of the main design evaluation tools is Logical Framework. The Logical Framework
approach consists in the application of an analysis matrix that makes it possible to for-
mulate a project, as well as identify and review its constituent elements and coherence.
Module 3:
64 Information and Evaluation Systems
From this analysis, identification and prioritization of problems, formulation and evalu-
ation are facilitated ex ante while operational planning, monitoring and evaluation, and
evaluation are facilitated ex post. The aim of the Logical Framework matrix is to structure
the planning process, clearly explaining the following elements:
INTERVENTION MEANS OF
INDICATORS ASSUMPTIONS
STRATEGY VERIFICATION
Activity indicators
Activities
Budget
The analysis of the information provided in the logical framework matrix is based on a
double logic: horizontal and vertical. A project or program must meet both criteria to
confirm that its design is consistent.
◊ Horizontal logic of the Logical Framework: This evaluates whether the proposed
indicators adequately measure the results obtained at purpose, components and
activities levels.
Leaders for Citizen Security
and Justice Management
65
For more information regarding the logical framework, consult [The Logical Framework
Matrix for Formulating and Evaluating Projects] (optional reading).
Based on this analysis, public policy makers and decision makers can evaluate the devel-
opment of their activity, the structure of its components, the dynamics it generates,
among other issues relevant to achieving the objective of the intervention. The review of
the processes involved in the implementation of a citizen security program allows public
policy makers to:
Process evaluations can be carried out using qualitative and quantitative methods, the
main components of which are: a) comparison of the operation of the project in relation
to the design, b) description and analysis of activities, components and processes and c)
recommendations to improve the intervention (Manzano and Mellado, 2016). From this
and other information, process indicators to monitor the activities, their beneficiaries
and intermediate products of the project or program can be constructed.
Module 3:
68 Information and Evaluation Systems
Process indicators are aimed at analyzing the specific activities in order to meet the
objectives of the security plan or program.
Process indicators:
◊ Number of benefits.
◊ Number of derivations.
The Office of the High Commissioner of the United Nations for Human Rights
in Mexico (OHCHR) and the Ministry of Public Security of Mexico City (SSPDF
according to its Spanish acronym) (2014), propose a set of indicators on citizen
security and human rights. The document states that indicators “enable the evalu-
ation of the way in which a state complies with its obligations” (OHCHR and SSPDF,
2014: 23), with those processes based on budgetary allocations, coverage of cer-
tain population groups, complaints regarding reparation, the proportion of human
resources destined for citizen security tasks, etc. In particular, on the subject of
human rights, the OHCHR and the SSPDF (2014) propose the following process
indicators:
◊ Number of complaints received by the SSPDF about violations of human rights.
◊ Number of cases which were considered by the Office of the Internal
Controller and the General Directorate of Police Inspection.
◊ Percentage of SSPDF personnel trained in the field of human rights.
◊ Number of human rights training courses given by the SSPDF.
◊ Number of activities and events related to human rights at the SSPDF.
◊ Rate of participation of SSPDF staff in events related to human rights.
◊ Number of SSPDF staff.
◊ Total budget of SSPDF.
◊ Budget allocated to different items.
The Center for Research for Development A.C. (CIDAC, 2016) designed a methodology for
the evaluation of the first responder police, which contains elements of process evaluation.
The methodological proposal is constructed in order to evaluate the performance of the
police as the first authority that takes cognizance of the criminal acts in the accusatory crim-
inal system. The proposed evaluation methodology considers the analysis of the following
sources of information:
◊ Police records: knowledge of criminal act, assistance to people, protection of the place
of crime, inspection of people, evidence processing, detention, interviews.
◊ Surveys on police offices: quality and adequacy of equipment and work tools, police
training, inter-institutional coordination, police performance.
◊ Surveys on public ministries: quality and adequacy of equipment and work tools, train-
ing of police, inter-institutional coordination, police performance.
◊ Evaluation guide on compliance with detention regulations.
Module 3:
70 Information and Evaluation Systems
The evaluation of results can be defined as “an evaluation of the effects of the immediate
products of the project. The products are closely related to the objectives of the project
since they are precisely those that correspond to the means to achieve the objectives.”
(Tocornal et al., 2005).
The results evaluations are ex post assessments, i.e., they are applied once the imple-
mentation phase of the project or program has been completed. This is the most widely
used type of evaluation, insofar as it provides information regarding the results, effec-
tiveness, efficiency and benefits of an intervention, generating learning that can be
applied to the design of new programs.
To perform the required analyzes and to determine whether a program has achieved the
planned objective, as well as to examine the relationship between costs and benefits,
quantitative methodologies are traditionally used. These are based on indicators, from
which the expected and obtained results can be compared numerically.
For the comparison between the achieved results and the objectives established in the
design of the program, different sources of information should be used depending on the
objectives, coverage and activities defined in the design of the program. The collection
of information must at least include information on: a) the activities performed, b) use
of resources, c) achievement indicators, d) users’ impressions, community and program
staff (Olavarría, 2007).
An impact evaluation aims to identify the desired and undesired effects on the imple-
mentation of a project or program, to verify whether these changes correspond to the
objectives set out in the original program design and, in particular, to determine if the
observed effects are attributable to the intervention. The analysis of this information
allows us to learn lessons from the intervention, which will help to redirect and refine
new implementations of the program.
It is important to highlight the difference between the evaluation of results and the
impact evaluation. On one hand, the evaluation of results provides information on
changes in the variables that measure the results of an intervention, which are not nec-
essarily due to the program. On the other, the impact evaluation assesses the extent to
which the results are attributable to the intervention.
For example, in a program whose general objective is to reduce the participation of ado-
lescents (between 14 and 18 years old) in violent crimes in a certain territory, the main
lines of intervention have been defined: reintegration of adolescents into the school sys-
tem, training of leaders in mediating conflicts and treatment and reintegration in cases of
abuse or dependence on drugs or alcohol. The results evaluation of this program would
analyze:
◊ The performance of the activities and services established for each line of
intervention.
◊ The achievement of the program regarding: number of young people who re-en-
tered the school system, number of leaders trained as mediators and number of
young people who paid for an abusive drug use habit.
The impact evaluation of the same program would only seek to determine compliance
with the general objective of the program within the target group of the intervention
but trying to establish unequivocally that the changes perceived in the target group at
the end of the program are directly attributable to it. To this effect, the evaluation must:
Leaders for Citizen Security
and Justice Management
73
◊ Measure the total number of adolescents from 14 to 18 years old, and the propor-
tion of those who have committed crimes within the territory to be intervened.
◊ Randomly assign the adolescents into two groups, one as a control group and the
other as an experimental group. In each group, the personal and family charac-
teristics of young people should be measured, as well as variables related to their
schooling, work-life balance or conflict resolution skills, and substance use.
◊ During the program, a detailed follow-up should be made of each group, recording
the changes, particularly with respect to the key variables.
◊ At the end of the program, both groups should be measured again in relation to the
variable of participation in violent crimes, and to the other key variables.
The questions that guide the impact evaluation are: To what extent did the project affect
the beneficiaries? Were any of the improvements the direct result of the project or
would it have occurred anyway? Also, the results of the impact evaluation are useful to
establish the extent to which program costs were justified (Baker, 2000).
Once the groups are established, the researchers manipulate one or more independent
variables to observe how these affect a dependent variable. Examples of dependent vari-
ables are the impact indicators which include: rate of complaints, perception of insecu-
rity, homicide rate, victimization due to family violence. The independent variables are
those that explain the behavior of the dependent variables, for example: age, schooling,
place of residence, criminal record, etc.
In an evaluation with experimental design, the groups are randomly selected from a
sample of eligible individuals or intervention units eligible for the benefits of the program.
Module 3:
74 Information and Evaluation Systems
It was found that the program decreased the rate of pregnancies in the student
population, mainly in eighth, eleventh and twelfth grade students.
Indicators that measure the specific progress of the agreed targets and the changes
resulting from the intervention must be constructed to evaluate the impact of the inter-
ventions. In this sense, the impact indicators seek to measure the effect of a policy or
program on an issue.
Module 3:
76 Information and Evaluation Systems
For example, if the objective were to reduce crime in a given territory, one possible indi-
cator is the variation in complaints over the last year (Mohor, 2007). Since they compare
the current situation with the initial situation, baseline information and a measurement
to the evaluation date are required prior to the intervention of the program.
In Latin America, impact evaluations are scarce and lack the necessary rigor to obtain
relevant information about what works and what does not (Frühling, 2012). However,
in recent years efforts have been made to evaluate the impact of numerous public secu-
rity programs that are underway in the region. This is the case of the Quadrant Plan of
Carabineros of Chile; the Quadrant Plan of the Colombian National Police; the evalua-
tion of the Fica Vivo (Stay Alive) program in Brazil; the Comprehensive Citizen Security
Management Program in Montevideo, Uruguay; the community-based strategy of
the United States Agency for International Development (USAID) for the prevention
of crime and violence in Central America: El Salvador, Guatemala, Honduras, Panama
(Frühling, 2012).
Leaders for Citizen Security
and Justice Management
77
Generation of Evidence
For almost a decade, IDB has been developing various strategies in order to
strengthen the evaluation of security programs and to promote the designing
of programs based on evidence. Therefore, as an example, it has supported the
efforts of academics to systematize programs evaluated in areas such as: fam-
ily and school (Hein, 2010); intrafamily violence and child maltreatment (Larraín,
2010); situational prevention (Rau and Prosser, 2010); prevention of violence
through local interventions: multi-agencies, drugs and gangs (Oviedo, 2010);
police (Frühling, 2010); and the criminal justice system (Olavarría y Pantoja,
2010)8. Another program worth highlighting is the “Regional System for Impact
Evaluation of Public Security Policies for Latin America”, which began in 2012
aiming to create a regional network that systematically promotes the perfor-
mance of impact evaluations on citizen security programs with the highest qual-
ity standards in the field in Latin America. The national governments of Mexico, El
Salvador, Honduras, Costa Rica, Uruguay and Chile, the Mayor’s Office of Bogota
(Colombia) and the government of the Province of Cordoba (Argentina) partici-
pate in this program, with the support of the National University of Villa María
and the Center for Studies in Citizen Security of the University of Chile.
Source: Frühling, 2012
As we have seen throughout this unit, evaluations of security programs in the region are
scarce and are infrequently disseminated. Nevertheless, it is evident that over the last
decade evaluations have increased both in volume and seen improvement in the appli-
cation of better technical standards. In this process, academic centers aimed at promot-
ing public policies in these matters, as well as international cooperation agencies, have
played a key role.
In this context, the United Nations Evaluation Group (UNEG) launched a document
named: “Standards and Evaluation Standards” in 2016. This is an updated version of the
guide developed in 2005, “Evaluation Standards in the United Nations System”, which
was intended to strengthen evaluation practices and become a key reference for evalu-
ators around the world.
Leaders for Citizen Security
and Justice Management
79
7. CONCLUSION
Various studies and sources of information highlight the fact that the phenomena of vio-
lence and crime in Latin America have experienced a strong growth in recent decades. In
particular, the high levels and persistence of lethal violence (homicide rate) in the region
have been highlighted, in contrast with the downward trends observed in other parts of
the world.
At the same time, several studies reflect on the fact that the governments and public
institutions of the region have so far not been able to provide effective and far-reaching
solutions to these problems, which is largely due to shortcomings in the design, manage-
ment and evaluation of citizen security and coexistence policies.
In order to enhance public policies in this area, it is claimed that better quality informa-
tion systems must be developed, both administrative records and victimization or secu-
rity surveys. The existence of better information systems would certainly contribute to
the realization of security diagnoses, according to the regional, national and local reality,
as well as improving the management and evaluation of citizen security policies.
Statistical systems of public security, for the process of production, management, anal-
ysis and dissemination of their results, require coordination between institutions and
agencies that are linked together. The information provided by these systems allows
decisions to be made from a technical perspective in interdependent spheres of public
administration.
Considering the complexity of the problems of crime and violence that afflict the region,
such as the ones mentioned above, the evaluation of policies or programs in this area
presents obstacles and challenges greater than those observed in other areas of social
policies.
The multicausality of the phenomena (violence, crime, fear, among others), the diver-
sity of public and private agencies that execute programs in this area and, finally, the
Module 3:
80 Information and Evaluation Systems
As it was seen in the module, the different evaluation models that are implemented fulfill
different purposes and have different characteristics according to the time of its appli-
cation and its objectives.
In this sense, the development of ex ante evaluations for diagnosis facilitate the identi-
fication, characterization and prioritization of problems, as well as the targeting needs
of certain territories/groups. The evaluation of design, on the other hand, analyzes the
coherence between the identified problem, the population which will be intervened with
and the proposed program, its objectives, activities and, in general, the planning of the
processes. The design evaluation also establishes whether the technical and financial
conditions exist for the intervention to be implemented (or its viability). The evaluation
of processes gives an account of the functioning of the programs, the execution of activ-
ities and the budgets involved, enabling policy makers to correct those activities that
mark a departure from the original design and do not show results in the short term.
On the other hand, once the implementation of the program has been completed, the
evaluation of results offers information regarding the correct execution of the activi-
ties, the attainment of products and/or achievements, and the fulfillment of objectives
according to the initially proposed goals. Finally, the evaluation of impact reveals if there
were changes in the target group with respect to the problem situation, and if they are
attributable to the executed program. Both the evaluation of results and the impact
allow us to make decisions about the continuity, term or replicability of a program.
In LAC, the implementation of evaluative strategies is still incipient because they face
multiple obstacles, from political and organizational to technical problems derived from
the lack of access to data/indicators on the subject. As a result of these difficulties, most
of the citizen security programs that have been implemented in the last two to three
decades in the region have had no evaluation which would allow the stakeholders to
establish, in an empirical, rigorous and objective manner, whether such intervention
strategies and actions had an effect on the reduction of the “situation-problem”. Due
to this lack of evidence, most of the programs that are designed are not coherent and
Leaders for Citizen Security
and Justice Management
81
do not offer pertinent answers to the security problems that afflict the region, both at
national and local levels.
In spite of this, it is evident that over the last decade these evaluations have increased
both in volume and better technical standards have been applied. This process has been
possible, on the one hand, thanks to the greater emphasis given by national and local
governments towards better management of citizen security and crime prevention poli-
cies, and, on the other hand, due to the improvement of information systems. In this pro-
cess of improving the standards of the region’s evaluations, academic centers aimed at
promoting public policies in these matters, as well as international cooperation agencies,
have played a key role.
Undoubtedly, there is still a long way to go and this requires that managers and public
managers make greater efforts to develop integrated evaluation practices from the out-
set in the formulation of security and crime prevention policies. Only through a more
in-depth and systematic knowledge about which interventions work and which ones do
not, will it be possible to make further progress in the development of public policies
for efficient and effective citizen security, but above all based on a constant process of
learning and feedback.
Module 3:
82 Information and Evaluation Systems
8. GLOSSARY
Efficacy: refers to the degree of compliance with the objectives of a project, not neces-
sarily measured by the resources allocated for this, but through the percentage of cov-
erage of the target population.
Evidence-Based Programs: those programs that are designed based on the empirical
results of other similar experiences (what works and what does not), that is, through the
systematic collection of observations that answer the question of whether a program or
public policy has achieved the expected results.
Leaders for Citizen Security
and Justice Management
83
Statistics: refers to the collection, interpretation and analysis of numerical data that are
presented in an orderly and systematic manner. The purpose of statistics is to describe
numerical information and make inferences regarding the occurrence of certain phe-
nomena, enabling the analysis of high volumes of information.
9. BIBLIOGRAPHY
Aguilar, M. and Ander-Egg, E. (2001). Diagnóstico Social. Conceptos y Metodología [Social
Diagnosis. Concepts and Methodology]. Argentina: Grupo Editorial Lumen.
Análisis y Desarrollo Social Consultores. (2010). Guía de evaluación de programas y proyec-
tos sociales [Evaluation of programs and projects guide] . Madrid: Plataforma de ONG
Acción Social [Platform of Social Action NGO].
Araya, J. (2009). Documento de Apoyo Docente: Diagnóstico y Gestión Pública en Políticas de
Seguridad [Educational support document: diagnosis and public management of secu-
rity policies] . Santiago de Chile: Centro de Estudios en Seguridad Ciudadana.
Arbulú, A. (2013). Evaluación final del programa Mejorando la Seguridad Ciudadana en
Panamá: Hacia la Construcción Social de una Cultura de Paz. Fondo para el Logro
de los Objetivos de Desarrollo del Milenio. Naciones Unidas [Final evaluation of
the Improving Citizen Security in Panama program: Towards the social construc-
tion of a culture of peace. Funds
Aromaa, K. (2012). Victimization Survey - What are they Good for? Viktimologija: Teorija,
Praksa, Aktivizam, Jun 2012: 85-94. Serbia.
Arriagada, I. and Godoy, L. (1999). Seguridad ciudadana y violencia en América Latina:
Diagnóstico y políticas en los años noventa [Citizen security and violence in Latin
America: Diagnosis and policies in the 90s]. Serie políticas sociales. Santiago de
Chile. CEPAL. Baker, J. (2000). Evaluación de Impacto de los Proyectos de Desarrollo
en la Pobreza [Evaluation of the impact of poverty development projects] . Washington
D. C.: World Bank.
Beato, C. (2005). Producción y Uso de Informaciones y Diagnósticos en Seguridad Urbana
[Production and use of information and diagnosis of urban security]. Washington D.C.:
World Bank.
Bergman, M., Flom, H., Vilalta, C. and Masello, D. (2014). Informe sobre Encuestas
Penitenciarias en América Latina. Herramientas conceptuales y metodológicas para su
diseño, ejecución, evaluación [Report on Prison surveys in Latin America. Conceptual
and methodological tools for thier design, implementation, evaluation]. Washington
D.C.: Inter-American Development Bank (IDB).
Berk-Seligson S., D. Orcés, G. Pizzolitto, M. Seligson and C. Wilson (2014). [Impact
Evaluation of USAID’s Community-Based Crime and Violence Prevention Approach in
Central America: Regional Report for El Salvador, Guatemala, Honduras and Panama].
Module 3:
88 Information and Evaluation Systems
Latin American Public Opinion Project (LAPOP) and USAID. Vanderbilt University.
Brookman, F. y Robinson, A. (2012). Violent Crime. In Maguire, M., Morgan, R. and Reiner,
R., The Oxford Handbook of Criminology. Oxford: Oxford University Press: 563-594.
Center of Excellence in Statistical Information on Government, Seguridad Pública,
Victimización y Justicia [Public Security, Victimization and Justice] (CDEUNODC).
(2012). Available at https://cdeunodc.wordpress.com/2012/12/18/encuestas-de-
victimizacion-un-recorrido-breve-a-lo-largo-de-la-historia-lo-bueno-y-lo-malo/
Center of Excellence in Statistical Information on Government [Public Security,
Victimization and Justice]a (CDEUNODC). (2013). Inventario de Encuestas de
Victimización en América Latina y el Caribe [Inventory of victimization surveys in
Latin America and the Caribbean]. UNODC.
Cohen, E. and Franco, R. (2000). Evaluación de Proyectos Sociales [Evaluation of social proj-
ects]. Mexico: Siglo XXI Editores.
Comisión Económica para América Latina y el Caribe (CEPAL) [Economic Commission
for Latin America and the Caribbean (ECLAC)]. (2014). Documento conceptual de
los sistemas estadísticos de seguridad pública y justicia en los países de América
Latina y el Caribe [Concept paper on Statistical Systems Citizen Security and
Justice in Latin America and the Caribbean]. Statistical Conference of the Americas.
Mexico: United Nations, ECLAC and Center of Excellence in Statistical Information
on Government, Public Security,, Victimization and Justice.
Concha-Eastman, A. and Guerrero, R. (1999). Vigilancia epidemiológica para la pre-
vención y el control de la violencia en las ciudades [Epidemiological surveillance
for the prevention and control of violence in cities]. Panameña de Salud Pública/
Panama Journal of Public Health (4/5): p.322. Washington D.C.: Organización
Panamericana de la Salud [Pan American Health Organization].
Daigle, L. (2017). Victimology: A text/reader. London: Sage Publications.
Dammert, L., Ruz, F. and Salazar, F. (2008). ¿Políticas de seguridad a ciegas? Desafíos para
la construcción de sistemas de información en América Latina [Blind security policies?
Challenges for the construction of information systems in Latin America]. Santiago de
Chile: FLACSO-CHILE.
Dammert, L., Salazar, L., Montt, C. and González, P. (2010). Crimen e Inseguridad. Indicadores
para las Américas [Crime and Insecurity. Indicators for the Americas]. Santiago de Chile:
FLACSO-Chile/Banco Inter-American Development Bank (IDB).
Gibbons M. and M. Rossi (2015). Evaluación de Impacto de un Programa de Inclusión
Social y Prevención de la violencia Estudiantil [Impact Evaluation of a Social
Leaders for Citizen Security
and Justice Management
89
Inclusion and Student Violence Prevention]. IDB Working paper No. IDB-WP-624.
Washington DC.
Estrategias de Acompañamiento y Servicios Educativos S de RL de CV (EASE) (2017).
-Report entitled Acompañamiento y Evaluación de la intervención de Cauce Ciudadano-
Cure Violence en Ecatepec, Estado de México. Teoría del cambio, modelo de marco lógico
e indicadores [Accompaniment and evaluation of the Cauce Ciudadano-Cure Violence
intervention in Ecatepec, State of Mexico.Theory of change, logical framework model
and indicators]. Washington D.C.: Inter-American Development Bank (IDB).
Fajnzylber, P., Lederman, D. and Loayza, N. (1998). Determinants of crime rates in Latin
America and the world. Washington D. C.: World Bank.
Frühling, H. (2003). Police Reform and the Process of Democratization. In Frühling,
H., Tulchin, J. and Golding, H., Crime and Violence in Latin America: Citizen Security,
Democracy and the State. Washington D. C.: Woodrow Wilson Center Press: 15-44.
Frühling, H. and Mohor, A. (2009). Medición de análisis de la criminalidad en los países
de América Latina. Lecciones aprendidas [Measurement and analysis of criminal-
ity in Latin America. Lessons learned]. Round table on criminality. Cali, Colombia:
Proyecto Sistema Regional de Indicadores Estandarizados de Convivencia y
Seguridad Ciudadana [Regional System of Standardized Indicators in Peaceful
Coexistence and Citizen Security Project].
Frühling, H. (2012). Discussion paper: La Eficacia de las Políticas Públicas de Seguridad
Ciudadana en América Latina y el Caribe: Cómo Medirla y Cómo Mejorarla [Efficacy
of public policies in citizen security in Latin America and the Caribbean: how to
measure it and how to improve it]. Diálogo de Política Regional [Dialogue on Regional
Policy]. VENUE: Inter-American Development Bank (IDB).
Hales, J., Nevill, C., Pudney, S. and Tipping, S. (2009). Research report 19: Longitudinal anal-
ysis of the offending, crime and justice survey 2003-06. London: Home Office. United
Kingdom.
Hernández Sampieri, R., Fernández Collado, C. and Baptista Lucio, P. (2007). Fundamentos
de Metodología de la Investigación [Fundamentals of the Methodology of
Research]. Madrid: McGraw Hill.
Hiskey, J., Malone, M. and Orcés, D. (2014). Violencia y migración en Centroamérica
[Violence and Migration in Central America]. Vanderbilt University: Barómetro de las
Américas [The Americas Barometer].
Hoyle, C. (2012). Victims, Victimisation and Restorative Justice. In M. Maguire, R. Morgan,
and R. Reiner (eds.), The Oxford Handbook of Criminology. Oxford University Press.
Module 3:
90 Information and Evaluation Systems
Public Ministry of Chile. (2017). Boletin estadístico anual. Enero - diciembre 2016 [Annual
statistics bulletin. January - December 2016]. Santiago de Chile: Public Ministry of
Chile.
United Nations Development Program (UNDP). (2013). a Citizen Security with a Human
Face: Evidence and Proposals for Latin America. New York: UNDP.
Ribeiro, E., Borges, D. and Cano, I. (2015). Calidad de los datos de homicidio en América
Latina [Quality of homicide data in Latin America]. [Document prepared by the
“Conference on the quality of homicide data in Latin America and the Caribbean”].
Bogotá: Open Society Foundations.
Riego, C. (2014). La expansión de las facultades de la víctima en la Reforma Procesal
Penal y con posterioridad a ella. [The expansion of the victim’s faculties in the penal
process reform and subsequent to it] Política Criminal, Vol. 9,18: 668-690.
Salazar, F. (2013). Estado del arte de definición de indicadores en América Latina y el Caribe
[State-of-the-Art in the Use of Indicators in Latin America and the Caribbean]. Quito:
FLACSO-Ecuador.
Sherman, L. (2012). Development and Evaluation of Citizen Security Programs in Latin
America. Protocol for Crime Prevention Based on the Evidence. Washington D.C.: Inter-
American Development Bank (IDB)
Síndic de Greuges de Catalunya. (2012). Informe sobre el derecho de acceso a la información
pública [Report on the right of access to public information]. 1a edición. Barcelona:
Síndic de Greuges de Catalunya
Straits, B. y Singleton, R. (2011). Social Research: Approaches and Fundamentals. Oxford:
Oxford University Press.
Tappan, P. (1947). Who is the criminal? American Sociological Review, Vol. 12, No. 1:
96-102.
Tocornal, X., Viano, C. and Zuloaga, L. (2005). Los Qué, Cómo y Cuándo de la Evaluación:
Recomendaciones para el Diseño de Evaluaciones de Proyectos de Prevención
Comunitaria del Delito [The What, How and When of Evaluation: Recommendations
for evaluation design for Community Prevention of Crime Projects]. Boletín
+Comunidad +Prevención. Santiago, Chile: CESC.
Vilalta, C., Castillo, J. and Torres, J. (2016). Delitos violentos en ciudades de América
Latina [Violent Crime in Latin American cities] . Washington D. C.: Inter-American
Development Bank (IDB).
Module 3:
94 Information and Evaluation Systems
http://www.cdeunodc.inegi.org.mx/articulos/doc/1_inv_
encuesvic.pdf
Regional references: spe-
cialized centers or security http://www.cdeunodc.inegi.org.mx/index.php/mapa/
observatories
http://www.oas.org/dsp/espanol/cpo_observatorio_enlaces_
victimizacion.asp#AR
Argentina (2017)
http://www.jus.gob.ar/areas-tematicas/estadisticas-de-polit-
Buenos Aires and other cities ica-criminal/estudios-de-victimizacion.aspx
(since 1995)
http://www.oas.org/dsp/espanol/cpo_observatorio_enlaces_
Barbados (since 2002)
victimizacion.asp#AR
Bolivia, large cities (2011) http://www.onsc.gob.bo/
Brazil (2010-2012)
www.inec.go.cr
http://www.ecuadorencifras.gob.ec/encuesta-de-victim-
izacion-y-percepcion-de-inseguridad-2011/
Ecuador - Quito (2003, 2004 http://www.oas.org/dsp/documents/victimization_surveys/
and 2008, 2011) ecuador/ecuador%20-%20encuestas%20de%20victimazi-
Ecuador (2008 and 2011) cion.pdf
http://omsc.quito.gob.ec/index.php/12-contenido-ejem-
plo/40-encuesta-de-victimizacion.html
Leaders for Citizen Security
and Justice Management
95
http://www.oas.org/dsp/documents/victimization_surveys/
El Salvador (2004 and 2009)
el_salvador/victimizaci%C3%B3n_el_Salvador_2009.pdf
http://www.cdeunodc.inegi.org.mx/index.php/mapa/
Guatemala (since 2004) www.poderjudicial.gob.ni/genero/ovgn/informe_final_
febrero_2015.pdf
Guyana (LAPOP, 2014) http://www.cdeunodc.inegi.org.mx/index.php/mapa/
https://presencia.unah.edu.hn/seguridad/articulo/iud-
Honduras (2014 and 2016) pas-aplicara-encuesta-de-percepcion-de-inseguridad-y-vic-
timizacion-en-honduras
Jamaica (2006, 2009 and
http://statinja.gov.jm/projects.aspx
2012-2013)
http://www.inegi.org.mx/est/contenidos/proyectos/encues-
tas/hogares/regulares/envipe/http://www.cdeunodc.inegi.
Mexico org.mx/index.php/mapa/
(2002-2010/2011-2017)
http://www.beta.inegi.org.mx/proyectos/enchogares/
regulares/envipe/2017/
Nicaragua (since 2007) https://www.ieepp.org/media/files/publicacion-2-509.pdf
http://www.pa.undp.org/content/panama/en/home/library/
Panama (since 2010) democratic_governance/viii_informe_seguidad_ciudad-
ana_2017.html
http://www.cdeunodc.inegi.org.mx/index.php/mapa/
Paraguay (2010)
http://www.dgeec.gov.py/
Peru (National Survey of http://www.inei.gob.pe/media/MenuRecursivo/publica-
Budget Programs, since 2010) ciones_digitales/Est/Lib1442/libro.pdf
OPTIONAL READING
Two key steps must be followed to build indicators: conceptualization and operational-
ization (or transforming the concept into an indicator).
Conceptualization
Define theoretically the concept
that will be measured
Operationalization
Transform the abstract concept
into objective categories
CONCEPTUALIZATION
The indicator must have a solid theoretical basis, which supports the decisions that are
taken in its construction and the theoretical construct it intends to represent. In this first
stage, it is necessary to clearly define what the indicator is expected to measure, the con-
cept that will be measured, what the limits of the concept are, and what the indicator will
emphasize, if anything at all. The process of constructing the indicator consists of moving
from the more abstract to the more concrete so that at the end of the conceptual discus-
sion, the result will be a clear and concise theoretical definition to be disaggregated and
operational in subsequent phases.
OPERATIONALIZATION
The process of operationalization of the theoretical definition has two components: the
definition of dimensions and, based on this, the definition of indicators .
The vast majority of phenomena under study—from the fields of citizen security and
criminal justice to many other social sciences—are clearly multidimensional. For this
reason, it is necessary to extract from the conceptual development in the previous phase
some specific dimensions that explain the phenomenon under study in its complexity.
B. Definition of Indicators
At this stage, it is necessary to select the indicators, which are numerical measurements
that capture the meaning of the defined dimensions. In many cases, more than one indica-
tor will be required to give an account of the concept, and instead a cluster of indicators
from the same or different sources of information may be needed. It is recommended
that a list is prepared of all the indicators that could provide the most complete informa-
tion about the dimensions of the concept for each dimension, and then to evaluate the
relevance of including the indicator: in terms of its strengths and weaknesses; the infor-
mation source; the quality of the indicator; and the replicability of the data (for example,
if the same information will be available in two or three years).
GIS can play an important role in decision-making in public policies and in the reduction
of crime by providing information for the diagnosis, monitoring and evaluating the inter-
vention object.
In recent decades the development of technologies has facilitated the use of georef-
erencing techniques. The use of digital maps and databases standardized in the same
format allows for the display of a crime phenomenon in its temporal-spatial context in
greater depth, making it possible to access small units of analysis. This methodology can
be used on different scales to observe trends at a global, regional, national or local level.
Module 3:
100 Information and Evaluation Systems
Components of a GIS:
The georeferencing of crimes is a technique widely used by the police, and one of the
main sources of information to perform this type of analysis are police administrative
records since they have information on the location of reports and arrests made by the
police. From this and other sources of information, the main analyses that are performed
observe points or frequencies, areas of concentration and the analysis of hotspots.
◊ Analysis of points or frequencies: the criminal facts registered at a point are posi-
tioned on a map. As the frequency of criminal acts increases in the same place, the
size or color of this point is modified.
◊ Hotspots: These facilitate the identification of the areas with the highest density
of criminal activity occurrence, making it possible to observe the displacement of
these facts. This methodology is used by the police for the organization of police
activities, identifying those areas where their impact can be greater. While sev-
eral studies support the results of this strategy in Latin America, very few police
departments deploy it.
Leaders for Citizen Security
and Justice Management
101
For the intervention of problem areas associated with security and justice at local level,
it is essential to know the criminal dynamics, the spatial distribution of criminal or vio-
lent acts and their causes. With an adequate diagnosis, local governments can design
public policies that respond effectively to reality, focusing their interventions on those
territories of greater complexity.
In Mexico, the implementation of the Program for Citizen Coexistence began in 2012
as part of the Merida initiative—an agreement between the governments of Mexico
and the United States to reduce violence in border states. In this framework, a group
of industrial sites in Ciudad Juarez were selected for the program. Before designing the
Master Plan for Prevention, a local diagnosis was developed, identifying the major prob-
lems that required intervention. These diagnostics integrated information from official
records, qualitative data obtained from the discourse of the community, demographic
data and georeferenced information. This made it possible to make a diagnosis that cov-
ered the local situation in terms of socio-demographic characteristics, education, access
to health, housing conditions, equipment, income, population distribution and crime
context. The mapping of criminal information allows for the construction of a local diag-
nosis that takes into account the spatiality of the phenomenon. By way of example, the
diagnosis made for the city of Juarez presented georeferenced information on the teen-
agers’ place of residence who had violated the law in the period 2007-2010.
Based on this information, the diagnosis could conclude that there was a concentration
of adolescents who broke the law in some areas of the city. The diagnosis suggested that
this distribution was associated with areas with few educational services on offer, as
well as few recreational areas. This is illustrated more clearly in the chart below, which
uses the hotspots methodology to georeference the location of young people who have
broken the law.
Module 3:
102 Information and Evaluation Systems
The use of measuring instruments on violence and crime, such as victimization surveys,
only makes sense if they have coherent and quality information (Dammert et al., 2010).
This implies that this information meets certain criteria of technical rigor, such as valid-
ity, reliability, objectivity, standardization and comparability. These criteria are based on
the definitions proposed by Trochim and Donelly (2008) and Hernandez Sampieri et al.
(2007), although they can be found in other methodological manuals.
Validity: refers to the measuring capacity of an instrument; that is to say, a valid instru-
ment is one that, through the variables to be investigated collects information about the
phenomenon under study and nothing else. An instrument can have content, criterion
and construct validity.
Reliability: refers to the consistency between different measurements taken by the same
instrument, which is achieved through a coherent instrument that is stable over time.
This implies that the error induced by a particular measurement—either by chance, by
the observer’s application of the instrument or by distortions of the instrument itself—
are the least possible. Different strategies, including the test-retest of the instrument,
the parallel forms method or internal consistency tests are used to measure reliability.
See Figure 3 for an illustration of validity and reliability, in which the red point is the
concept to be measured. First of all, there is an instrument that is reliable since the mea-
surements are consistent; however, it is not valid, as the measurement does not repre-
sent the measured concept properly. In second place, there is an instrument that is not
reliable due to the dispersion that the observations show, although it is valid because it
refers to the concept measured. In third place, we can see an instrument that does not
Leaders for Citizen Security
and Justice Management
105
Objectivity: an instrument is objective to the extent that the observation of the phe-
nomenon is impartial, without it being committed to obtain certain results or distort
the information collected. As long as an instrument is valid and reliable, its objectivity
improves. However, the design and administration of an instrument by expert and inde-
pendent bodies undoubtedly ensures higher levels of objectivity.
Standardization: refers to the establishment of definitions and categories that are pre-
cise, exclusive, mutually coherent and that remain consistent in time. This facilitates the
comparability of the data over time (analysis of trends), independent of any changes that
may occur in the teams applying the instruments.
Periodicity: Maintaining a measuring instrument over time, but also applying it in fixed
and continuous time divisions—for example, semi-annual, annual, biannual—is essential
to ensure the quality of the information being collected and enable the trend analysis
of the criminal phenomenon. Furthermore, maintaining periodicity in the dissemination
of data promotes the legitimacy of the instruments. Therefore, while administrative
records are usually collected routinely and disseminated regularly, in terms of victimiza-
tion surveys this remains a challenge for most of the countries in the region.
Module 3:
106 Information and Evaluation Systems
◊ Approval of the government, evaluation of the policy, the national economic situation
and the media.
◊ Victimization for delinquency and domestic violence, fear of crime, and corruption.
The Logical Framework is a tool designed to facilitate the process of identification, for-
mulation or design, execution and evaluation of projects, with the objective of structur-
ing the planning process so that the essential information of the project is communicated
clearly (Ortegon et al., 2005; Manzano and Mellado, 2016).
The Logical Framework is used during all phases of the project cycle, enabling the follow-
ing actions:
Module 3:
108 Information and Evaluation Systems
Ex-post evaluation and analysis Carried out in order to determine its specific contribu-
of the social impact of a project tion in the context which the project was designed for.
◊ Means of verification.
The rows of the matrix present information about the objectives, indicators, verification
means and assumptions at four different moments in the life of the project:
INTERVENTION VERIFICATION
INDICATORS ASSUMPTIONS
STRATEGY MEANS
Activities Budget
Module 3:
110 Information and Evaluation Systems
GOAL
◊ Every project responds to a problem or obstacle that has been detected. The goal
expresses the solution to the diagnosed problem.
◊ The goal, therefore, is a higher-level objective which the project attempts to contribute to.
◊ This is the description of how the project contributes, to the long-term solution of the
identified problem, which does not mean that the project itself will be sufficient to
achieve the goal and that there may be other projects and/or programs that also con-
tribute to its achievement.
◊ The indicators at the goal level measure the overall impact that the project will have.
Therefore, they are specific in terms of quantity, quality and time.
◊ The assumptions indicate the events, conditions or important and necessary decisions
for the sustainability (continuity in time) of the benefits generated by the project.
PURPOSE
◊ This is the general objective of the project and constitutes the concrete and direct
effect or impact, which will be achieved as a result of the use of the components pro-
duced by the project during its execution.
◊ It is a hypothesis about the desired impact or benefit which can and should be measured.
◊ The achievement of the purpose will contribute to achieving the goal, i.e. the specific
contribution to the solution of the problem diagnosed.
◊ It is expressed as a result, for example: “It improves the quality of teaching”, and not, “It
will improve”.
◊ It is what we expect to happen when the beneficiaries of the project use the products
we deliver them.
◊ It must be taken into account that for the purposes of the Logical Framework each proj-
ect is required to have only one purpose or general objective: two purposes are two
projects.
◊ The indicators at purpose level describe the impact achieved at the end of the project
and include goals that reflect the situation on the completion of the project. Each indi-
cator specifies quantity, quality and time of the results to be achieved.
Leaders for Citizen Security
and Justice Management
111
COMPONENTS
◊ These are the specific results of the project, i.e. the products (goods and/or services)
produced or delivered by the program to fulfill its purpose.
◊ A component is a good and/or service aimed at the end beneficiary or, in some cases, at
intermediate beneficiaries: it is not a stage in the production process of the component.
◊ Each component must be necessary to achieve the purpose and therefore it is reason-
able to assume that if all the components are produced as planned, the purpose will be
fulfilled: the set of components allows for the achievement of the purpose.
◊ The components must be written clearly and as results, actions, finished work, or final
products; for example: alarms installed, workshops held, training carried out, etc.
◊ Its production mode should also be noted (enforcement mechanisms or ways to pro-
vide products and/or services), specifying who is responsible for the execution of the
components.
ACTIVITIES
◊ These are the detailed tasks that must be performed to achieve each of the compo-
nents of the project, and they involve costs.
◊ To determine them, a list of activities is prepared in chronological order for each com-
ponent and the time and resources required to carry it out are estimated. This detailed
list of activities will be the basis for the preparation of the Execution Plan, Calendar,
Gantt Chart or Operational Plan of the Project.
◊ In some cases, the activities indicator is the budget for each component, which will be
produced by the project.
◊ The means of verification indicate where an evaluator can get information to verify if
the budget was spent as planned and is normally the accounting record of the execut-
ing unit.
Module 3:
112 Information and Evaluation Systems
INDICATORS
MEANS OF VERIFICATION
◊ These correspond to the sources of information that will be used to obtain the val-
ues of the indicators. They must be identified in the matrix for each of the indicators
presented.
◊ This requires that the project designers identify sources of information or arrange for
information to be collected, possibly as a project activity, with its corresponding cost.
◊ Not all information has to be statistical. The production of components can be verified
through a visual inspection of the specialist and can include published material, sur-
veys, information records, statistical reports, etc.
ASSUMPTIONS
◊ They are the external factors that are beyond the control of the person responsible for
the project, but which affect its success or failure. In other words, the assumptions are
related to the risks involved in a project, be they environmental, financial, institutional,
social, political, climatological or otherwise.
◊ The risk is expressed as an assumption that has to be fulfilled to advance to the next
level in the hierarchy of objectives. The reasoning is as follows: if we carry out the indi-
cated activities and certain assumptions are met, then we will produce the indicated
components. At the same time, if we produce the indicated components and other
assumptions are met, we will then achieve the purpose of the project. Finally, if we
achieve the purpose of the project, and further assumptions still keep showing up, then
we will contribute to the achievement of the purpose.
◊ The logical framework therefore requires that the risks be identified at each stage:
activity, component, purpose and goal. However, the objective is not to record every
eventuality that can be conceived, but to identify possibilities with a reasonable degree
of probability.
◊ The assumptions column plays an important role, both in planning and in the imple-
mentation of the project. In the first case, it serves to identify the risks that can be
avoided by incorporating additional components in the project itself, while, during exe-
cution, the assumptions indicate the factors that must be anticipated, influenced, and/
or confronted with adequate emergency plans.
◊ Assumptions can also be expressed in quantitative terms, including the minimum val-
ues of certain external factors that guarantee the normal development of the project.
Module 3:
114 Information and Evaluation Systems
A Logical Framework matrix has a double logic: horizontal and vertical. Only if both are
met, will the design of the matrix be consistent.
This shows the cause-effect relations between objectives at different levels: in this way,
to achieve the goal it is necessary for the purpose to be fulfilled; to accomplish the pur-
pose it is necessary to produce the results (components or products); and to comply with
the products it is necessary for the activities to be carried out (to perform the activities
it is necessary to have the inputs).
The purpose, together with the assumptions at this level, describe the necessary condi-
tions, even when they are not sufficient to achieve the goal.
The horizontal logic of the matrix shows us that it is not enough to fulfill the activities
to obtain products, but, in addition, the assumptions of the activity level must be met to
have the necessary and sufficient conditions.
In the horizontal logic of the Logical Framework, the indicators that are formulated to
measure the achievement of the goal, the purpose, the components and activities must
have the capacity to effectively measure what is to be measured (validity). If the indi-
cators are not adequately expressed, a contradiction is produced between the goal or
objective, and the indicator that will be used to measure compliance with them.
To illustrate the elements discussed about the Logical Framework, a fictional project is
presented below.
Leaders for Citizen Security
and Justice Management
115
INTERVENTION VERIFICATION
INDICATORS ASSUMPTIONS
STRATEGY MEANS
The participation of
young people in men- The socioeconomic
PURPOSE: tal health, training and Social statistics conditions of young
employment programs (Municipality). people do not change
Socially integrated increases by an average significantly.
young people of 25%.
improve their Parents maintain
indicators of health, their supportive atti-
Reduction of 20% in the
education and labor tude for young peo-
number of cases attended
participation. Hospital ple with problems.
that involve overdose or
statistics.
alcohol intoxication in
health centers.
ACTIVITY MATRIX
VERIFICATION
INTERVENTION STRATEGY INDICATORS ASSUMPTIONS
MEANS
Component 1: Young people know ◊ 40% of the young popu-
the implications of certain health lation receive courses in
◊ The young people
behaviors. sex education, drugs and
are interested
Activities alcoholism.
◊ Costs (budget). in attending the
◊ Provide sex education courses. ◊ Over a period of eight courses.
months, the main places ◊ Records of training
◊ Provide courses on drugs and institutions.
that young people use
alcoholism. ◊ Youth associations
frequently for studying and ◊ Campaign supervi-
◊ Conduct awareness campaign. recreation have posters and sion records in the neighbor-
accessories with prevention hood establish
messages.. sports programs
to use the facilities
set up.
Component 2: Sports and recre- ◊ Sports center budget
◊ Accounting records.
ational facilities are offered. executed.
◊ Municipality
◊ Build a sports center. ◊ An average of 100 vulnera-
registry.
ble young people per month
◊ Generation of recreation initia-
participating in workshops
tives and use of free time, for
and other activities is
example, art workshops.
maintained.
◊ Establish summer camps.
◊ At least 200 vulnerable
young people participate in ◊ Municipality
summer camps. registry.
◊ Training institution
Component 3: Job training pro- ◊ At least 200 vulnerable records.
grams are developed. young people have access ◊ Training institution
◊ Programs of recovery or com- to study completion records.
pletion of studies. programs.
◊ Municipal account-
◊ Job training programs for vul- ◊ At least 300 vulnerable ing records.
nerable youth. young people are trained
◊ Municipal account-
for work.
ing records.
While it is recognized that the concept of best practices arises in the field of governance
to highlight the good implementation of a public policy, program or social project and,
therefore, not directly applicable to information systems, we believe that it is possible
to establish criteria that information systems must comply with in order to be classified
as rigorous instruments that offer relevant and quality data. These criteria, as defined
in previous sections, are: validity, reliability, objectivity, standardization, periodicity and
comparability.
In light of these criteria, two examples of the region will be discussed next. The first case
is a measurement instrument that has been applied in Chile since 2003, the National
Urban Survey of Public Safety, which has been recognized by international statistical
experts, such as the Center of Excellence in Statistical Information on Government,
Crime, Victimization and Justice of the United Nations (CDEUNODC). The second exam-
ple is the Management Model by Result of the province of Pernambuco, Brazil, which
has been highlighted in the “Gobernarte” contest as a good practice in the use of criminal
information systems for the planning and management of policies at local level.
Module 3:
118 Information and Evaluation Systems
The ENUSC was first in Chile in 2003, and it has been implemented annually since 2005,
keeping most of the instrument and the application methodology steady over time. From
its inception, ENUSC sought to collect information at national and regional level on the
experiences of criminal victimization suffered by households and individuals, as well as
to characterize crime and examine the reaction and perception of individuals on crime.
The survey is applied by the National Institute of Statistics and it is financed and admin-
istered by the Undersecretary of Crime Prevention of the Ministry of the Interior and
Public Security of Chile. The information is obtained from a sample of around 26,000
households which are representative of urban areas, nationally and regionally. The sur-
vey measures crime asking people older than 15 years-old, who are members of selected
households, about their victimization experiences that occurred in the 12 months prior
to the application of the tool. The tool records all “greater social connotation” offences,
including those not reported to the authorities. In addition, it enquires about economic
crimes, cybercrimes and other offences or incivilities that have occurred in the local
context.
This survey was quickly positioned as the main instrument to measure victimization in
Chile and, for this reason, has been used as a source of key information for the design of
citizen security policies (Lagos and Dammert, 2012). The validity, reliability and standard-
ization of the instrument, as well as its continuity in applications, has made it possible to
use its results over time and in international comparisons (INE, 2011).
86,8%
85,0%
Decrease People who
82,6% 2015-2016
79,9% perceive that crime
increased in the country
70,6% 71,2%
60,2%
44,5%
40,7% 40,5% 41,3% People who believe that
38,7% 39,7% Steady
35,0%
they will be victims of a
2015-2016
crime in the next 12
28,8%
26,4% 27,3% months
25,6% 24,3% 22,8% 23,5%
Steady
2015-2016 Household victimization
Considering the high death rate by the state (53 cases per 100,000), and the aforemen-
tioned model, the Pact for Life Program (PPV) was designed in 2007. In the framework
of this program, the Ministry of Social Defense of Pernambuco adopted the indicator of
intentional lethal violent crimes (CVLI) as a criterion to monitor the reduction of violent
mortality (Pernambuco, 2007). In addition, through the management of crime analysis
and statistics, they developed a methodology of rigorous and efficient data collection
based on the crossing of data from various institutions: Civilian Police, Military Police,
the Institute of Forensic Medicine and the Institute of Criminalistics. The latter reflects
one of the main achievements of the program: inter-sectoral governance, which made it
possible to achieve the main goal of the project, a 12% reduction in lethal violent deaths.
In this way, the death rate in Pernambuco first matched the Brazilian national rate and,
later, the best international standards (López Ribeiro, 2014).
References:
Governo de Pernambuco (2007). Pact for life. State Plan of Public Safety. Recife: CEPE.
Lopes Ribeiro, L.M. (2014). Management Model by Results of the Pact for Life Program. In
Béliz, G., Basco, Ana Inés; Ribeiro, Ludmila, and others, GobernArte, Category safe govern-
ment. Inter-American Development Bank [IDB technical note; 2014, 684]: 5-21.
Module 3:
120 Information and Evaluation Systems
BIBLIOGRAPHY
Dammert, L., Salazar, L., Montt, C. and González, P. (2010). Crime and Insecurity. Indicators
for the Americas. Santiago de Chile: FLACSO-Chile/Inter-American Development Bank
(IDB).
Hernández Sampieri, R., Fernández Collado, C. and Baptista Lucio, P. (2007). Fundamentals
of Research Methodology. Madrid: McGraw Hill.
National Statistics Institute -INE- Chile (2011). Report on the National Urban Survey of
Citizen Security. Available at: http://historico.ine.cl/canales/chile_estadistico/
encuestas_seguridadciudadana/pdf/memoria_enusc.pdf
Lagos, M. and Dammert, L. (2012). Public Security. The Main Problem of Latin America. Lima:
Latinobarómetro Corporation.
Manzano, L. and Mellado, C. (2016). “Part three. Models and Tools for the Evaluation of Citizen
Security Policies or Programs.” Teaching Support Document, prepared within the frame-
work of the course Methodologies Applied to Social Research in Security and Justice.
Center for Studies in Public Safety, INAP, University of Chile.
Ortegón, E., Pacheco, J. and Prieto, A. (2005). Methodology of the Logical Framework for
the Planning, Monitoring and Evaluation of Projects and Programs. Santiago, Chile:
ECLAC.
Program for Citizen Coexistence PCC (2012). Community Master Plan for the Prevention
of Crime and Violence. Technology Park: Francisco I. Madero, Ciudad Juárez, Chihuahua.
Mexico. http://mexicosos.org/descargas/dossier/estudios/plan_maestro_juarez_
fco_i_madero.pdf
Trochim, M. and Donnelly, J. (2008). The Research Methods Knowledge Base. Mason:
Atomic Dog.
Leaders for Citizen Security
and Justice Management
121
LEADERS FOR
CITIZEN SECURITY AND
JUSTICE MANAGEMENT