“Learning from Existing

Evaluation Practices on the

Impacts and Effects of
Intellectual Property on
Development”
Geneva
6th/7th October
2011

Evaluation Section Internal Audit and Oversight Division

(IAOD)World Intellectual Property Organization (WIPO)
BREAKING the CULTURE of ‘FAKES and
COUNTERFEITS’:
INFORMATION and EDUCATION PROGRAMS not
CAUSING RISE of INTELLECTUAL PROPERTY
AWARENESS in the PHILIPPINES?

Romeo B. Santos
WorkLand M&E Institute, Inc.
 COUNTRY BACKGROUND
BACKGRO
Introduction _____PHILIPPINES [PHL]____
PHL

POPULATION: 92 million
GDP per capita: 3,724
AREA: 299, 764 sq. m.
MAJOR IPR LAWS NAME YEAR
Republic Act 165 PATENT LAW 1947
Republic Act 166 TRADEMARK LAW 1947
INTELLECTUAL PROPERTY
Republic Act 8293 1998
CODE
IPR GLOBAL
TREATIES / COVERAGE YEAR
ORGANIZATIONS
BERNE CONVENTION LITERARY AND ARTISTIC 1951
PARIS CONVENTION INDUSTRIAL PROPERTY 1965
WIPO INTELLECTUAL PROPERTY 1968
PERFORMERS, PRODUCERS
AND PHONOGRAMS AND
ROME CONVENTION 1984
BROADCASTING
ORGANIZATIONS
TRIPS AGREEMENT TRADE 1995
PCT PATENTS 2001
WIPO COPYRIGHT TREATY COPYRIGHT 2002
MAJOR LAWS, GLOBAL TREATIES AND PERFORMANCES AND
ORGANIZATIONS [PHL is MEMBER] WPPT 2002
PHONOGRAMS

http://www.infoplease.com/atlas/country/philippines.html
 COUNTRY BACKGROUND
BACKGRO
Introduction _____PHILIPPINES [PHL]____
PHL
STATE OF IPR IN THE
COUNTRY
http://ph.news.yahoo.com/anti-software-piracy-drive-nets-p4-3m-may-022218863.html

Source: Business Software A lliance and the

Presidential Inter Agency Committee-
Intellectual Property Rights Executive
Summary for 1999-2001

http://ph.news.yahoo.com/philippine-mayors-next-target-pirates-manila-district-111003534.html

USTR PRIORITY WATCH List until 2005

Introduction
STATE OF IPR IN PHL

Has attained substantial

PROGRESS in the LEGAL &
INSTITUTIONAL ENVIRONMENTS
IMPROVING
BUT in general:

DECLINING

STALLING
Software Piracy rate in the Asia-Pacific Region
1996-2003 [in percent]

SOURCE: Annual BSA Piracy Study, January 2003: First Annual BS and IDC
 Introduction

3 CORNERSTONES of IPR
TREATIES’
IMPLEMENTATION in the
Philippines. APPROACH
used by the Philippines in its
fight against PIRACY.
[PROGRAM
INTERVENTION
APPROACH]
The Total IP Protection Model, Philippines
Source: THE IP COALITION REPORT I: Copyright in the PHILIPPINES 2004
Key Lessons
APPROACH TO EVALUATION
--Analyzed the PHILIPPINE GOVERNMENT IPR PROTECTION
INTERVENTION APPROACH used as basis of the
EVALUATION APPROACH

INTERVENTION APPROACH EVALUATION

APPROACH

A B C
3 CORNERSTONES of the PHILIPPINE IPR
TREARTIES IMPLEMENTATION
A: PUBLIC INFORMATION and EDUCATION
B: LEGAL and POLICY
C: ENFORCEMENT and ADJUDICATIOIN

[ PROBLEM = EVALUATION CONTEXT ]

Key Lessons
APPROACH TO EVALUATION

INTERVENTION APPROACH EVALUATION

APPROACH

THEORY OF CHANGE:
Recreating the LOGIC OF
CHANGE, the CAUSAL LINKS IMPACTS

between the PHILIPPINE IPR OUTCOMES

PROGRAM INTERVENTION
EFFECTS
and desired RESULTS.
A B C

EVALUATION APPROACH = EVALUATION DESIGN

Key Lessons
APPROACH TO EVALUATION

THIS REPORT

FOCUS
OF THIS
REPORT

The EVALUATION PROJECT

Key Lessons
METHODOLOGY
EVALUATION ANALYSIS KEY FINDINGS
CONTEXT TRIANGULATE RECOMMENDATIONS
DESIGN MATRIX LESSONS LEARNED
EVAL Qs; RESEARCH
DESIGN
Re-createTHEORY
OF CHANGE

GETTING the DATA

 DOCUMENTS REVIEW
 SURVEY
 STRUCTURED INTERVIEWS [KRPI]
 CS, FGD
 TRANSECTS
Key Lessons
METHODOLOGY THEORY OF CHANGE

RESULTS: RESULTS:
INPUTS ACTIVITIES OUTPUTS
OUTCOMES IMPACTS
IPR Hold People
education, /Sectors Raised people’s
trainings & Increased IPR
training and trained on IPR awareness measured
educational responsible practices
publicity by their
programs Advocacy & among individuals,
events understanding of the
educ. sectors and communities
programs held importance of IPR
Budgets Enact
legislations Laws enacted

Personnel Establish Policies Increased peoples’

institutions implemented IPR supportive Sustained anti
Systems& activities measured -infringement culture
Enforce laws Laws enforced
Procedures by reduced violations among the public
Operate IPR P, U, D, TM, and IPR system
Other system ®, © granted applications/awards
resources
Key Lessons
METHODOLOGY KEY EVALUATION QUESTIONS/DESIGN MATRIX

Measures &
QUESTION* SUB-QUESTION TYPE Target Baseline
Indicators
1. To what extent What PROGRAM
has the PHL IPR component [PIEP,
Number of
PRP, EAP]? What
Treaties’ Descriptive respondents per NA None
audience [schools,
Implementation audience type
professionals,
Program SMES, others]?
[PROGRAM]
What extent among
increased the people in general? Number of
awareness on IPR Descriptive NA None
respondents
among Filipinos?
Design One shot Survey/ Interviews/ Transects
Data Sources Program records
Sample ‘Inventory’ of all IPR orgs in PHL
Data Collection Instrument Record Retrieval
Data Analysis Frequency count
Comments Graphic

* Only Selected KEY EVALUATION QUESTIONS are presented.

Key Lessons
METHODOLOGY KEY EVALUATION QUESTIONS/DESIGN MATRIX

Measures &
QUESTION SUB-QUESTION TYPE Target Baseline
Indicators
2. To what extent do Do the program
PROGRAM components align with Check of
WIPO & other alignment,
components Descriptive/
conventions? conformity NA None
harmonize with Norm with WIPO,
objectives of WIPO & etc.
other conventions?
3. What specific Are the strategies
strategies have consistent with
international Check of
generated the highest
practices? consistency
responses and Descriptive/ with WIPO, NA None
actions among Norm etc.
recipients & the
public in supporting
IPR?
Design: Record & document review; KRPI
Key Lessons
METHODOLOGY KEY EVALUATION QUESTIONS/DESIGN MATRIX

Measures &
QUESTION SUB-QUESTION TYPE Target Baseline
Indicators

4. How effectively have How effectively has

the PHL IPR systems been the coordination
Number of
mechanism among
utilized and how have cases;
IPR entities Explanatory Yes NA
these contributed to the Attributes of
contributed to the
results? Instance
achievement of
results?
5. What are the underlying Which categories of
factors that explain stakeholders were
found to be most Number of
barriers to knowledge and
committed to Descriptive/ cases;
appreciation of IPR among Yes NA
understanding and Explanatory Attributes of
the public? applying IPR Instance
responsible
practices?
Design: Record & document review; FGD, KRPI, CS, Transects
Key Lessons
METHODOLOGY KEY EVALUATION QUESTIONS/DESIGN MATRIX

Measures &
QUESTION SUB-QUESTION TYPE Target Baseline
Indicators

6. Is there What is the number of

Number of
evidence of reported infringement
violations,
incidents in PIEP recipients
behavioural C&E suits, NA None
and the public, before and
change among resolved
after the program?
Filipinos as a cases
result of the PHL
Due to PIEP, do recipients
IPR PROGRAM? & the public believe that the Number of
[This Sample Q program has made a C&E respondent
focuses on PIEP difference in their attitude cases
component of toward IPR?
PROGRAM]
Compared to before the
PIEP, is there evidence of Number of
increased activities [P, U, cases of
C&E NA None
D, ™, © ® applications, applications,
processing] among awards
recipients& the public?
Design: Record & document review; FGD, KRPI, Transects
Key Lessons
METHODOLOGY KEY EVALUATION QUESTIONS/DESIGN MATRIX

Measures
Targe
QUESTION SUB-QUESTION TYPE & Baseline
t
Indicators

7. What are the key What are the comparative Number of

lessons learned and advantages or strengths of Descriptive/ cases;
the PROGRAM? NA None
good practices that Explanatory Attributes
of Instance
can add up to the
knowledge base of What are the constraints and Number of
PHL IPR Program? complex instances worthy of Descriptive/ cases; NA None
deeper consideration of Explanatory Attributes
[ for policy change or
PROGRAM proponents? of Instance
program design]
To what extent do the Number of
PROGRAM proponents Descriptive/ cases;
willing to continue or scale NA None
Explanatory Attributes
up the program activities? of Instance

Design: Record & document review; FGD, CS, KRPI

Key Lessons
Evaluation Findings and Results
The Philippines’ Program for IPR protection appears
UNDERPERFORMING.

The Information and Educational component of the IPR

thrust is likely NOT causing substantial and sustained
RISE of AWARENESS among public.

Rise in Awareness is likely affected more by

ENFORCEMENT and not by INFORMATION &
EDUCATIONAL PROGRAMS, [particularly when raids
and arrests are given wide media coverage].
Key Lessons AWARENESS: 3 Key
Thrusts
Sector
Unit
RATING
Factor
RATING
Overall
RATING
Public Information and a. General L
Education Program public L
L
(PIEP) b. Industry

It appears that the Philippine IPR

Policy and Regulation
Program (PRP)
a. General
public
L

L-M
M M
b. Industry
protection program is
Enforcement and
Adjudication Program
a. General
public
M-H
M-H

UNDERPERFORMING.
H
(EAP) b. Industry
Rating System: H = High; M = Medium; L = Low
Table 1: Awareness Performance Rating of 3 IPR Protection Thrusts
EVALUATION Unit Factor Overall
CRITERIA
‘Information and Educational’
Perspectives

Organization
a. Capacity
RATING
S
RATING RATING

component, likely, NOT causing

b. Linkage S
a. Quality U
Program b. Content S U
substantial and sustained rise in c. Reach
a. Efficiency
U
U
U

Awareness among public.

Implementation
b. Effectivenes
s
U

U
U

c. Continuity
Rating System: VS = Very Satisfactory; S =
Satisfactory; U = Unsatisfactory; US =
Unsatisfactory
Table 2: Overall Performance of the Philippine IPR Protection Program
Key Lessons
Evaluation Findings and Results

There seems no strong harmonization among the

three key components of the present IPR thrust.

There is no mechanism and structure for determining

performance in the present IPR Implementation
Program.
The strong transnational nature of commerce involved
in piracy makes it difficult to control patronage despite
raised awareness on IPR value. Lopsided price gap
between ‘genuine and fake’ influences infringement.
Key Lessons
Conclusions and Recommendations
Legal and Institutional environments are already in place but
socio-economic realities and poorly designed programs are the
likely reasons for the slow progress in establishing a strong
culture for IPR.

Strong law enforcement, coupled with sound information and

educational program, with emphasis on Media participation,
may deliver better outcomes.

Strong transnational cooperation and market adjustments

[such as pricing mechanism realignment] may support
effective implementation of IPR thrust.
There shall be a built-in M&E framework within the IPR
Programs, premised on results-oriented implementation
objectives.
Key Lessons
LESSONS LEARNED
The weak empirical tradition and research culture in the
Philippines are constraints affecting availability and sound
collection of data that will feed evaluation. [NO existing
BASELINE [need to re establish], NO TARGET!!!]

IPR policies and programs designed based on traditional project

cycle management principles are weak. The use of Results-
Based principles can provide strong M&E mechanism built
within the system.

The media is a strong tool for IPR treaties implementation;

programs shall be designed with inclusion of media as a main
element of the program design.

Evaluation initiated by private entity without strong endorsement

from a government authority is prone to more difficulties.
Evaluation Experience
MY Evaluation experience in IP [START UP]
IP issues evaluated
NIL
The START UP, still LOW M&E
-- many CAPACITY
agencies in without
totally the
PHILIPPINES does
M&ENOT PROVIDE much
STATE OF M&E
OPPORTUNITIES for EVALUATION of IPR POLICIES in
theIN THE The awareness
country. START UPvalue of M&E is
on the
COUNTRY BEGINNING--TO
some beginning to see value of
TAKE HOLD!
M&E due to requirements from
It appears no comprehensive EVALUATION
partner, donor, of IP
and funding
Treaties’ implementation, other than the few ‘studies’,
organizations
START UP has been done.
LOW
LOW -- a number have been doing
monitoring, traditional or hybrid
NIL M&E
Evaluation Experience
My Other Evaluation Experiences

Assessment of construction industries [Philippines & Guam] to

determine market environment for entry of Canadian green technology
[2010]

Meta-evaluation of M&Es done on ODA-funded urban development

projects in the Philippines, Indonesia and Sri Lanka [2009]

Design /formulation of Theory of Change & M&E framework, EVAW

Program, Afghanistan [2009]

Assessment of effects of ESD in TVET Education [2009]

Assessment of socio-urban contexts for development planning of a fast

urbanizing locality [2002]
Evaluation Experience
My OPINION on main evaluation needs
NEED to ADOPT results-oriented
implementation of IPR treaties in the
Philippines. A sound M&E framework shall be
integrated with the IPR Implementation
Program.
Government Agencies tasked to manage IP
Protection in the country shall be restructured
to incorporate an M&E body mandated to carry
out systematic evaluation functions.
 Adoption of MfDR in PHL is sluggish, BUT
as it moves toward its IMPLEMENTATION, the M&E
capacity of IPR RELATED INSTITUTIONS must be
2008 SurveyThis
built-up. on will enable sound DETERMINATION of
Monitoring the Paris
Declaration PERFORMANCE
to see of IPR TREATIES
progress onIMPLEMENTATION
aid in the country.
effectiveness
 5% -countries with
largely developed
results-oriented
frameworks
 56% -taken action
since 2005.

(MfDR Policy Brief, OECD, 2009)

26
 SUGGESTION to PHL for strengthening IPR
Key Lessons treaties’ implementation*
PHL needs a mechanism for M&E in IPR protection. It
has to build one.

Somewhere here, PHL DIRECTOR

needs an M&E SYSTEM GENERAL
in place -an M&E Office
mandated to determine DEPUTY DIRECTOR DEPUTY DIRECTOR
performance and provide GENERAL GENERAL
feedback to the Operation, IP Policies & Management &
International Relations Support Services
stakeholders,
administration & policy
makers.
BUREAU OF BUREAU OF BUREAU OF BUREAU OF FINANCIAL MANAGEMENT
PATENTS TRADEMARKS LEGAL AFFAIRS DOCUMENTS & IT MANAGEMENT & INRORMATION
ADMINISTRATION SERVICE

DIRECTOR IV DIRECTOR IV DIRECTOR IV DIRECTOR IV

DIRECTOR IV DIRECTOR IV
Office for OPERATIONS &
POLICIES is the ideal office to
DIRECTOR III DIRECTOR III DIRECTOR III reorganize
DIRECTOR III& institutionalize an
DIRECTOR III DIRECTOR III
M&E department.

* [TYPICAL PHL AGENCY STRUCTURE FOR IP PROTECTION ]

THANK YOU