Learning Notes 5

Alternative Evaluation Designs

- Counterfactual
o Program of policy that is compared to the situation that would exist without the program
o Impacts are measured as differences in outcome
o E.g. health status or earnings
- Internal Validity
o Measured difference can be appropriately attributed to the program being evaluated
o Depends on the comparison between the program and the situation without the program is made
- Evaluation design
o Specific scheme used for making comparisons in order to measure impacts

Employment and Training (E&T) Programs

- Viewed as investments in human capital (i.e., attempts to improve skills and abilities)
- Major economic rationale for funding: E&T programs help correct market or institutional failures that cause
underinvestment in human capital
- Help correct imperfect information among participants about human capital investment opportunities by
guiding them into activities that yield the highest pay-off for them
- Widely accepted value = it is better to receive income from employment than from transfer programs
 - Advantages:
o Readily understandable to policy makers
o By displaying benefits and costs from the perspectives of both participants and non-participants, it
suggests some of distributional implications of programs being evaluated
o Measures each cost-benefit component listed can be obtained from data collected during
evaluation
o Operationally feasible
- Difficult to find practical alternatives
- Two groups relevant in assessing E&T programs:
o Participants or clients served by evaluated program
o Non participants, including taxpayers who pay for the program

Conceptual Issues in Conducting CBAs of E&T Programs

- The Participant Perspective
o Two alternative measures
 Standard E&T framework estimates participant net benefits as net changes in incomes of
program clients. However, the conceptually appropriate measure of participant net
benefits is net changes in surplus of program participants, not net changes in incomes.
 Two measure diverge depends on precise mechanism
o Non-pecuniary benefits from employment
 Suggests that there are benefits from being employed
 Scarring effects
 Results from deterioration of human capital during periods non-employment and
employer reactions to gas in employment, are avoided by maintaining
employment
o Job-Required Expenditures
 Traditionally used in CBA of E&T programs count increases in job-related expenditures
(e.g. child care and transportation) result from program participation as social cost of
program – this approach can sometimes result in double-counting
- Non-participants’ Perspective
o Intangible Benefits Received by Non-participants
 Measuring benefits and costs of E&T programs to its participants is changing surplus that
should be estimated rather than changes in their money income
 Effects on participant surplus of E&T-induced reductions in participant leisure time should
be take into account
o Changes in Taxpayer Excess Burden
 Taxes result in losses of economic efficiency, or deadweight loss. Particular E&T program
could cause taxes to increase or decrease, depending whether program have positive or
negative net expenditures by the government
o Benefits from In-Program Output
 It is infeasible to determine what taxpayers would be willing to pay for an output. So, an
alternative approach is used that determines what labor resources required to produce the
output would have cost if purchased on the open market. Researchers will used
appropriate wage rate but this estimate can cause overestimation or underestimation than
the true value.
o Possible Reduction in Crime
  If E&T program increases, it may also decrease criminal activities
o Costs from Public-Sector Labor Displacement
 If the displaced workers have a similar risk of joblessness as E&T participants, then
increases in output produced by E&T participants assigned to public-sector agencies may
be entirely offset by losses in the output formerly produced by those workers who are
displaced. This need not be the case, however, if the E&T participants are exceptional in
terms of lack of skills or if they face exceptional barriers to labor market access.
 Thus, to the extent that E&T participants are unskilled relative to those they replaced, the
labor market consequences will differ from a simple one-for-one replacement.
o Costs from Private-Sector Labor Displacement
 CBA usually assumes that full employment is maintained. If it is, then it should be
relatively easy for displaced non-participants to find alternative job opportunities (although
this may take some time), and E&T programs will raise the feasible employment level.
Prediction Parameters
- The discounting time horizon
o Total period over which benefit and cost streams are either observed or predicted
- The decay rate
o Necessary in extrapolating E&T effects that persist beyond the period over which post-program
data on outcomes are collected in order to take account of the possibility that size of effects may
change over time