CHAPTER ONE; INTRODUCTION

1.1. The project concept

Project planning and analysis has a long history in financial and business analysis.
Project planning has always been used as a means of checking the profitability of a
particular investment by private firms. Recent experiences show that project analysis
has attracted the attention of development economists. Projects are now assessed from
the economy‘s viewpoint instead of only from the firm‘s perspective. The selection
criteria have also included economic criteria on top of financial criteria.

Perhaps the most difficult problem confronting administrators in developing countries

is implementing development programs. Much of the failures can be traced to poor
project preparation. Especially from development viewpoint, for most development
activities careful preparation in advance of expenditure is, if not absolutely essential,
at least the best available means to ensure efficient, economic use of capital funds and
to increase the chances of implementation on schedule. Unless projects are carefully
prepared in substantial details, inefficient or even wasteful expenditure is almost sure
to result – a tragic loss in nations short of capital.

1.2. What is project

Project can be defined as an investment activity in which financial resources are

expended to create capital assets that produce benefits over an extended period of
time. A project is a complex set of activities where resources are used in expectation
of return and which lends itself to planning, financing and implementing as a unit.

The basic characteristics of capital expenditure (also referred to as a capital

investment or capital project or just project), is that it typically involve current outlay
(or current and future outlays) of funds in the expectation of a stream of benefits
extending far into the future. Capital investment decisions often represent the most
important decisions taken by the firm or other decision maker. Capital investment
decisions have far reaching impact into the future. They are also characterized by
irreversibility. Thus, a wrong capital investment decision often cannot be reversed
without incurring substantial loss. They also involve substantial outlay of capital.

1
 1.3. Common Features of project
 Project involves the investment of scarce resources in expectation of future
benefits
 Project is an activity that capable of being planned, financed and implemented
as a unit
 Have a definite set of objectives and specific start and end dates
 Have organizational or geography boundaries
 It is an activity around which conceptual boundaries can be ascribed
 A project can be seen as a activity which is likely to have a partially or wholly
independent administration

1.4. The linkage between projects and programs

It is necessary to distinguish between projects and programs because there is

sometimes a tendency to use them interchangeably. While a project refers to an
investment activity where resources are used to create capital assets, which produce
benefits over time and has a beginning and an end with specific objectives, a
program is an ongoing development effort or plan which may not necessarily be
time bounded.

Examples could be a road development program, a health improvement program, a

nutritional improvement program, a rural electrification program, etc

. For instance, a health program may include a water project as well as a construction
of health centers both aimed at improving the health of a given community, which
previously lacked easy access to these essential facilities. Projects, which are not
linked with others to form a program, are sometimes referred to as ―stand alone‖
projects.

Projects in such context are the concrete manifestations of the development plans in a
specific place and time. One can think of projects as subunits and bricks of programs,
which constitute the national plan (usually the direction is from plans to projects). We
have to note that projects could be either public or private. It is the smallest
operational element prepared and implemented as a separate entity in a national plan
or program. A program most of the time consists more than one project.

From the above discussion it can be seen that the major difference between a project
and a program is not so much in objectives stated but lies more in scope, the details
and accuracy. A project is designed with a high degree of precision and details as
2
regards its objectives, features, calculation of returns and implementation plan. A
program by contrast is general, lacks details and precision and aims at a broader
goal often related to a sectoral policy of a country or departmental policy of an
organization.

characteristics Program Project

Scope/Objectives Wide/diverse Narrower/limited

Location Diffused/wide Specific

Life time No-time bound Time bound

Resources Larger budget Limited budget

The basic Characteristics of Projects (SMART characteristics)

Perhaps the distinction between projects and programs would be clear if we see the
basic characteristics of projects. Projects in general need to be SMART.

S – Specific

A project needs to be specific in its objective. A project is designed to meet a specific

objective as opposed to a program, which is broad. A project has also specific
activities. Projects have well defined sequence of investment and production activities
and a specific group of benefits. A project is also designed to benefit a specific group
of people.

3
M - Measurable

Projects are designed in such a way that investment and production activities and
benefits expected should be identified and if possible be valued (expressed in
monetary terms) in financial, economic and if possible social terms. Though it is
sometimes difficult to value especially secondary costs and benefits of a project,
attempt should be made to measure them. Measure costs and benefits must lend
themselves for valuation and general projects are thought to be measurable.

A – Area bounded

As projects have specific and identifiable group of beneficiaries, so also have to have
boundaries. In designing a project, its area of operation must clearly be identified and
delineated. Though some secondary costs and benefits may go beyond the boundary,
its major area of operation must be identified. Hence projects are said to be area
bounded.

R – Real

Planning of a project and its analysis must be made based on real information. Planner
must make sure whether the project fits with real social, economic political, technical,
etc situations. This requires detail analysis of different aspects of a project.

T – Time bounded

A project has a clear starting and ending point. The overall life of the project must be
determined. Moreover, investment and production activities have their own time
sequence. Every cost and benefit streams must be identified, quantified and valued
and be presented year-by-year.

Perhaps the distinction between projects and programs would be clear if we see the
basic characteristics of projects. Projects in general need to be SMART.

S – Specific

A project needs to be specific in its objective. A project is designed to meet a specific

objective as opposed to a program, which is broad. A project has also specific
activities. Projects have well defined sequence of investment and production activities

4
and a specific group of benefits. A project is also designed to benefit a specific group
of people.

M - Measurable

Projects are designed in such a way that investment and production activities and
benefits expected should be identified and if possible be valued (expressed in
monetary terms) in financial, economic and if possible social terms. Though it is
sometimes difficult to value especially secondary costs and benefits of a project,
attempt should be made to measure them. Measure costs and benefits must lend
themselves for valuation and general projects are thought to be measurable.

A – Area bounded

As projects have specific and identifiable group of beneficiaries, so also have to have
boundaries. In designing a project, its area of operation must clearly be identified and
delineated. Though some secondary costs and benefits may go beyond the boundary,
its major area of operation must be identified. Hence projects are said to be area
bounded.

R – Real

Planning of a project and its analysis must be made based on real information. Planner
must make sure whether the project fits with real social, economic political, technical,
etc situations. This requires detail analysis of different aspects of a project.

T – Time bounded

A project has a clear starting and ending point. The overall life of the project must be
determined. Moreover, investment and production activities have their own time
sequence. Every cost and benefit streams must be identified, quantified and valued
and be presented year-by-year.

5
 1.5. Projects and Capital Expenditures Decisions

Almost all projects involve a capital expenditure decisions. Capital expenditure

decisions often represent the most important decisions taken by a firm. Their
importance stems from three interrelated reasons;

1. Long-term effects

The consequence of capital expenditure decisions extend far in to the future. The
scope of the current manufacturing/agricultural agro-processing activities of a firm is
governed largely by capital expenditure in the past. Likewise, the current capital
expenditure decisions provide framework for future activities.

2. Irreversibility

Most of the time wrong capital investment decisions cannot reversed without
incurring a substantial loss

3. Substantial outlay

Capital expenditures usually involve substantial outlay in the form of cash or other
assets or obligation to pay in the future /called cash equivalent/

1.6. Origin of Projects

Projects could originated from various sources

a. Some may be ‗‘resource based‘‘ and stem from the opportunity to make
profitable use of available resources.
b. Other may be ‗‘market based‘‘ arising from identified demand in home or
overseas markets.
c. Some may be‘‘ need based‘‘ where the purposes is to try to make available to
the target community , in a defined area, minimal amount of certain basic
material requirements and services.

6
 1.7. Classification of Projects

Projects can be classified based on several criteria, such as ownership, source of

finance, and forces behind the projects.

1.7.1. Based on Ownership

 Private sector: projects mostly undertaken by business enterprises.
 Public sector: projects undertaken by national and local government
body
 NGOs: development projects are most often undertaken by non-
government and not for profit organization
1.7.2. Based on Sources of Finance
 Government treasury: projects may be entirely financed by
government budget as per its priority. For instance, agricultural
development projects;
 Government treasury and external sources: most projects are
financed by the joint participation of the government and donor groups.
For example a road project may be financed 50% by the government and
50% by a foreign donors
 External Sources of Finance: projects may be financed totally by
parties other than the government but established for the wellbeing of
citizens and the ownership may be for the government or the public
1.7.3. Based on the Forces Behind:
 Demand Driven/Need Driven: based on identified unsatisfied
demand projects can be created or on unsatisfied basic needs like
food, water, and shelter.
 Donor Driven: the force behind the financing organization. Donors
will have their own say and influence the types of the projects to be
established.
 Political Driven: projects may be established in response to some
political situation such as for example because of National Elections,
Project by Religious Organizations etc.

1.8. Project Analysis/Reasons for Project Analysis

All countries, but particularly the developing countries, are faced with the basic
economic problem of allocating resources such as labor at all levels of skill,
management and administrative capacity, capital, land and administrative and other
natural resources and foreign exchange, to many different uses such as current
production of consumer goods and public services or investment on infrastructure,
industry, agriculture, education and other sectors. These different uses of resources,
7
however, are not the final aim of the allocative process; rather they are the means by
which an economy can marshal its resources in the pursuit of more fundamental
objectives such as the removal of poverty, the promotion of growth and the reduction
of inequality in income. Pursuit of one objective (better income distribution) however,
may involve a sacrifice in other objective (rapids growth).

A choice therefore has to be made among competing uses of resources based on the
extent to which they help the country achieve its fundamental objectives. If a
country consistently chooses allocations of resources that achieve most in terms of
these objectives, it ensures that its limited resources are put to their best possible
use.

Project analysis is a method of presenting this choice between competing uses of

resources in a convenient and comprehensible fashion. In essence, project analysis
assesses the benefits and costs of a project and reduces them to a common
denominator.

1.9. The project format: advantages and limitations

1.9.1. Advantages

To enable analysis of projects a project format is conventionally used. This format

provides an analytical framework for a proposed investment in which the cost and
benefit accounts are prepared year by year in the form of a project cost and benefit
stream. The project format, which is an analytical tool, has some advantages.

1. Framework to analyze information from many sources

 It establishes the framework for analyzing information from a wide range of
sources. Information from a wide range of sources feed into the framework
(framework to analyze information from many sources).
2. Frame work to involve many specialists
 Since a good plan depends on accuracy of information, the framework enables
various specialists to judge the accuracy of the information provided and the
appropriateness of the assumptions (framework to involve many specialists).
3. The format also gives an idea of costs year after year, so that those
responsible for providing the necessary resources can do their own
planning (cost year by year).
If costs and benefits of the project are determined in financial terms, it would
be easy for financing institutions to act accordingly.

8
4. The project analysis tells us something about effects of a proposed investment
on the participants in the project, whether they are farmers, small firms,
government enterprises, or the society as a whole (Estimate effects on
participants).
5. Project format would give the chance to assess the financial impact of a
project on each participants of the project. It would also enable the analyst to
identify ‗gainers‘ and ‗losers‘ in the project area.
6. The administrative and organizational problems likely to be encountered
during the implementation of the project are also detailed in the project format
(judgment about administrative and organizational problems).

This enables the planners to make arrangements for strengthening the project
management if this appears weak.

7. Criteria for monitoring progress of implementation

At the same time managers planners and stake holders are given better criteria for
monitoring the progress of implementation as the objectives, targets and work
plans are set out at the onset of implementation (criteria for monitoring progress
of implementation).

8. Encourage systematic examination of alternatives;

The project format also enables us to systematically examine different alternatives

(encourage systematic examination of alternatives). Alternative formulating of
the same project or formulation of other projects, etc. are important. The project
format facilitates systematic and objective examination of a range of alternatives.
For instance, the effects of a proposed project on national income and other
objectives can be compared with the effects of projects in other sectors or other
projects in the same sector, or alternative formulations and designs of the same
project can also be compared. For a nation, the project format gives a chance to
assess and compare different alternative projects and designed so as to choose and
prioritize those projects that would meet the most of the objectives. Similarly for a
private owner the project format enables to assess not only different alternative
projects but also different design of the same project. This would enable him to
choose a project and a design that would maximize his objective(s). Thus, it is an
essential tool for allocating capital resources from the viewpoint of a nation and/or
private owner.

9
 9. Advantage of the project format will be to help contain the data problem.

Once a project has been initiated local information on which to base the analysis
can be efficiently gathered, field trials can be conducted and judgment can be
made about the institutional and cultural factors that might influence the choice of
project design and its implementation. Information gathered in the process of
project preparation and analysis could alleviate the data problem of developing
countries.

1.9.2. Limitations of the project format

1. Depends on the quality of data

Although the project format has so many advantages the result of project analysis
must be interpreted with caution. The first limitation is about the quality of the
data used. The quality of project analysis depends on the quality of the data used
and of the forecast of costs and benefits (depends on the quality of data used).
Unrealistic assumptions about market shares, future prices, yield potentials,
relevance of inflation, the quality of project management, etc., can make garbage
out of the project analysis. Of course the reliability of the results of project
analysis depends upon the extent to which the data, assumptions, and forecasts
diverge from the reality. Whatever efforts could be made there is always some
errors associated with these issues.

2. Limited usefulness in judging risk

The technique of project analysis provides limited support in judging the risk and
uncertainty surrounding the project (limited usefulness in judging risk). Project
planning is a forward looking. The realization of the expected net benefits of the
project depends on the extent that actual future circumstances deviate from the
expected future circumstances. Because future circumstances will change, project
analysts must judge the risks and uncertainty surrounding the project. But the
question is how are these risks and uncertainty being taken in to account in the
analysis and choice of projects..

Of course there are such techniques as sensitivity analysis, Monte Carlo simulation
analysis, decision tree analysis, etc. that are used to incorporate the risk element in the
analysis and choice of projects. Nevertheless, these techniques never can diminish or

10
avoid the risk problem. In summary, even though these techniques are useful and
essential, they are not a panacea of problems related to risks and uncertainty.

3. Partial Analysis

The other limitation of project format is that project analysis is a species of what
economists call ‗partial analysis‘. As a species of development planning models,
project analysis treat each project independent of the whole economy and usually
lacks consistency and overall feasibility. The apparent interconnection of a project
with the other projects and with the whole economy cannot be assessed. In this
respect, the project-by-project planning approach is most often used in those
economies where statistical data for an aggregate or complete main-sector model are
lacking. Therefore, it is advisable not to translate directly the net-benefits of projects
to the overall economy.

4. Difficult to compare widely different projects

The greater the difference among alternative projects the more difficult it would be to
use formal analytical techniques to compare them (difficult to compare widely
differing projects). Financial costs and benefits of a project can be used for
comparison of alternative projects that are similar in their nature. Such comparison
can be easily made between different alternatives of the same project. Alternatives can
also safely be compared if the benefits and costs of alternative projects can be valued
well. But objective comparison can hardly be made if costs and benefits of one project
are estimated reasonably well while not possible for the other (for instance between
irrigation and health projects). In such instances, the allocation of resources between
different projects must be made more subjectively and as a part of overall
development plan.

5. Limitation of prices as indicator of values

Another limitation of the project format is the underlying conceptual problem about
the valuation based on the price system (Limitations of prices as indicators of
value). The relative value of goods and services depends on the relative weights that
individuals participating in the system attach to the satisfaction they can obtain with
their income. Moreover, although project analysis must also address ‗externalities‘ or
side-effects, it is mostly difficult to value these effects objectively. One can, at best,
value for instance external costs of water or air pollutions or health hazards using

11
 proxy measures which in itself involve subjective judgment. In addition, it is mostly
address such broader objectives as national integrity, national sovereignty, regional
integration, etc. Of course the analyst could make his own justification but the
ultimate decision about such broader objectives will be left to political leaders.
However the problem is the way they weigh various tradeoffs may not lead to the
same conclusions a project analyst would reach.

CHAPTER TWO: THE PROJECT CYCLE

A project cycle is a sequence of events, which a project follows. These events, stages
or phases can be divided into several equally valid ways, depending on the executing
agency or parties involved. Some of these stages may overlap. Capital expenditure
decision is a complex decision process, which may be divided into six broad phases:

1. Identification

2. Pre-feasibility Study

3. Feasibility (technical, financial, economic)

4. Selection and project design

5. Implementation

6. Ex-post evaluation

Identification

Pre-feasibility study
Ex-post evaluation

Feasibility study
Implementation

Appraisal 12
 2.1. Identification:

The first stage in the project cycle is to find potential projects. Identification of
promising investment opportunities requires imagination, sensitivity to
environmental changes, and a realistic assessment of what the firm can do. This
phase may take two forms.

i. Private project:

If the project is largely a private venture in a widely market economy context the
initiating entity will define the concept, expectation and objectives of the project.

ii. Government project:

On the other hand the project idea can also emanate form government agencies in the
context of government development plans. In the latter case sectoral information (i.e.
the direct and indirect demands of sectors) is an important source of identification.

In market economy context anticipated demand for the projects output is important. In
addition assessment of appropriate technology, scale of the project, timing of the
project etc. are important. All types of specialists‘ input are required at this stage.

The planning phase of a firm‘s capital investment is concerned with the articulation of
its broad investment strategy and the generation and preliminary screening of project
proposal. The investment strategy of the firm delineates the broad areas or types of
investment the firm plans to undertake. This provides the framework, which shapes,
guides, and circumscribes the identification of individual project opportunities.

In general there are four major sources from which ideas or suggestions for project
may come:

 Project ideas from technical specialists

 Project ideas from local leaders

 Project ideas from entrepreneurs

 Project ideas from government policy and plans

13
Note that sometimes at identification stage there could be a number of alternatives
that could be examined. Some of these projects may appear for reasons nothing to do
with the national plan. In such circumstances its advantageous to understand the
‗political history‘ of the project.

The identification of project ideas is based on several aspects of development.

 Need - a need assessment survey may show the need for intervention

 Market demand - domestic or overseas

 Resource availability - opportunity to make available resources more

profitable

 Technology - to make use of available technology

 Natural calamity - intervention against natural calamity such as flood or

drought

 Political considerations

Possible alternative project must be adequately assessed.

Project preparation and analysis phase

Once project ideas have been identified the process of project preparation and analysis
starts. Project preparation must cover the full range of technical, institutional,
economic, and financial conditions necessary to achieve the project‘s objective.
Critical element of project preparation is identifying and comparing technical and
institutional alternatives for achieving the project‘s objectives. Different alternatives
may be available and therefore, resource endowment (labor or capital) would have to
be considered in the preparation of projects. Preparation thus require feasibility
studies that identify and prepare preliminary designs of technical and institutional
alternatives, compare their costs and benefits, and investigate in more details the more
promising alternatives until the most satisfactory solution is finally worked out. It
involves generally two steps:

 Pre-feasibility studies
Project preparation and analysis phase
 Feasibility studies
14
 2.3. Pre-feasibility Study

The identification process will give the background information for defining the basic
concept of the project, which leads to the feasibility study stage. Once a project
proposal is identified, it needs to be examined. To begin with, a preliminary project
analysis is done. A prelude to the full blown feasibility study, this exercise is meant to
assess (i) whether the project is prima facie /at first glance/ worthwhile to justify a
feasibility study and (ii) what aspects of the project are critical to its variability and
hence warrant an in-depth investigation. At the pre-feasibility study stage the analyst
obtains approximate valuation of the major components of the projects costs and
benefits. Some of the main components examined during the pre-feasibility study
include:

 Availability of adequate market

 Project growth potential

 Investment costs, operational cost and distribution costs

 Demand and supply factors; and

 Social and environmental considerations

That is, the study should cover

 Project or corporate strategies and scope of the project

 Market analysis and marketing concept
 Raw material and factory supplies
 Location, site and environment
 Engineering and technology
 Organization and overheads
 Human resources, in particular managerial (entrepreneur), labor costs and
training requirements and costs
 Project implementation schedule and budgeting

Using this preliminary data supplied by the various discipline specialists a preliminary
financial and economic analysis will be conducted. If the project appear viable form
this preliminary assessment the analysis will be carried to the feasibly stage.

15
 2.3.Feasibility Study

The major difference between the pre-feasibility and feasibility studies is the amount
of work required in order to determine whether a project is likely to be viable or not.
If the preliminary screening suggests that the project is prima facie worthwhile, a
detailed an analysis of the marketing, technical, financial, economic, and ecological
aspects will is undertaken. The focus of this phase of capital budgeting is on
gathering, preparing, and summarizing relevant information about various project
proposals, which are being considered for inclusion in the capital investment. Based
on the information developed in this analysis, the stream of costs and benefits
associated with the project can be defined. At this stage a team of specialists
(Scientists, engineers, economists, sociologists) will need to work together. At this
stage more accurate data need to be obtained and if the project is viable it should
proceed to the project design stage.

The final product of this stage is a feasibility report. The feasibility report should
contain the following elements:

 Technical analysis

 Market analysis

 Organizational analysis

 Financial analysis

 Economic analysis

 Social analysis, and

 Environmental analysis

2.4. Appraisal

The feasibility study would enable the project analyst to select the most likely project
out of several alternative projects. Selection follows, and often overlaps, analysis. It
addresses the question - is the project worthwhile? Wide ranges of appraisal criteria

16
have been developed to judge the worthwhile of a project. They are divided into two
broad categories, viz., non-discounting criteria and discounting criteria.

To apply the various appraisal criteria suitable cut off values (hurdle rate, target rate,
and cost of capital) have to be specified. The level of risk pursued influences these.
Despite a wide range of tools and techniques for risk analysis (sensitivity analysis,
scenario analysis Monte carol simulation, decision tree analysis, portfolio theory,
capital asset pricing model, and so on), risk analysis remains the most intractable part
of the project evaluation exercise. This exercise also involves the undertaking of
detailed engineering design; manpower and administration requirement as well as
marketing procedures should be finalized.

2.5. Implementation

After the project design is prepared negotiations with the funding organization starts
and once source of finance is secured implementation follows. Implementation is the
most important part of the project cycle. The better and more realistic the project plan
is the more likely it is that the plan can be carried out and the expected benefits
realized. At the project implementation phase tenders are let and contracts signed.
Project implementation must be flexible since circumstances change frequently.
Technical changes are almost inevitable as the project progresses; price changes may
necessitate adjustments to input and output; political environment may change.
Project analysts generally divide the implementation phase into three time periods.

 the investment phase, where the major investments are made. This may extend
from three to five years.

 the development phase, which may also extend from three to five years.

 the project life

The implementation phase for an industrial project consists of several stages:

1. Project and engineering designs,

17
  Site probing and protecting, preparation of blueprints and plant
designs, plant engineering , selection of specific machines and
equipment
2. Negotiations and contracting,
 With suppliers of technology and contractors, construction of building
and civil work
3. Construction
 Site preparation, construction of building and civil works, erection and
installation of machinery and equipment
4. Training, and
 Training of engineers, technicians and workers
5. Plant commissioning.
 It links the preceding construction phase, and the following operational
(production) phase
6. Production and marketing
 Promotional campaign before production. This may include securing of
supplies and setting up administration of the firm

Translating an investment proposal into a concrete project is a complex, time

consuming and risk fraught task. Delays in implementation, which are common, can
lead to substantial cost overrun.

2.6. Ex-post evaluation:

The final phase of the project is the evaluation phase. Many usually neglect this stage.
The project analyst looks carefully at the successes and failures in the project
experience to learn how better to plan for the future. In this stage it is important to
examine the project plan and what really happened. Performance review should be
done periodically to compare actual performance with projected performance.

A feedback device is useful in several ways:

(i) it throws light on how realistic were the assumptions underlying the
project;

(ii) it provides a documented log of experience that is highly valuable in future

decision making;

(iii) it suggests corrective action to be taken in the light of actual performance;

(iv) it helps in uncovering judgment biases;

18
 (iv) it induces a desired caution among project sponsors.

Weakness and strengths should carefully be noted so as to serve as important lessons

for future project analysis undertaking. Evaluation is not limited only to completed
projects. Ongoing projects could also be evaluated to rectify problems when the
project is in trouble. The project management, the sponsoring agency, or other bodies
may do the evaluation.

19
 CHAPTER THREE:

FINANCIAL FEASIBILTY STUDY

4.1. Introduction

Financial analysis seeks to ascertain whether the proposed project will be financially
viable in the sense of being able to meet the burden of servicing debt and whether the
proposed project will satisfy the return expectations of those who provide the equity
capital. Here the project analyst is concerned with the financial effects of the proposed
project on each of its various participants (firms, farmers/workers, government etc.).
By examining the financial implications of the project for these parties, the analysts
need to identify the projects financial efficiency, incentive impact to the participants
in the project, creditworthiness and liquidity (say, could the firm have enough
working capital?).

On the other hand, the financial evaluation aims at assessing the financial and
commercial feasibility of a project from the point of the investors and financer. The
enterprise‘s performance within a business environment is analyzed taking all
expenses for project inputs as cash outflows, and the income from operations as cash
inflows. All inputs and outputs are valued at market conditions. This means that the
analyst and decision makers measure the net gain or benefits and losses generated by
the project only in financial terms.

The financial analysis establishes the magnitude of costs of investment, production

and overheads and magnitude of benefits. This analysis will be the basis for
evaluating the project profitability. Project profitability depends on a comparison of
costs versus revenues using realistic market prices of materials, labor and outputs.

4.1.1. Objectives of Financial Analysis

Assessment of financial impact

The most important objective of financial analysis is to assess the financial effects the
project will have on participants (farmer, firms, government, etc). This assessment is
based on the comparison of each participant‘s current and future financial status with
the project against the projection of his future financial performance as the project is
implemented.

Judgment of efficient resource Use

20
For management especially, overall return is important because managers must work
within the market price framework they face. Investment analysis & financial ratio
analysis provide the tool for this review.

Assessment of Incentives

The financial analysis is of critical importance in assessing the incentives for different
participants of the project.

Will participants have an incremental income large enough to compensate them for
the additional effort and risk they will incur?

Will private sector firms earn a sufficient return on their equity investment &
borrowed resources to justify making the investment the project requires?

For semi-public enterprises, will the return be sufficient for the enterprises to maintain
a self-financing capability and to meet the financial objectives set out by the society?

Provision of sound financial plan

The financial plan provides a basis for determining. The amount and timing of
investment, debt repayment capacity, and also helps to coordinate financial
contributions.

Assessment of financial management competence especially for large projects,

financial analysis will enable the analyst to judge the complexity of the financial
management & the capability of managers so that he can judge what changes in
organization and management may be necessary.

Means of Finance: Shares capital Terms of loan Debenture capital Differed credit
Incentive sources Miscellaneous Share capital

There are two types of share capital-equity capital and preference capital.

i.Equity capital:-represent the contribution made by the owners of the business, the
equity shareholders, who enjoy the reward and bear the risks of ownership. Equity
capital being risk capital carries no fixed rate of dividend.

ii.Preference capital represents the contribution made by preference shareholders and

the dividend paid on it is generally fixed.

21
Terms loans

Provided by financial institutions and commercial banks, terms loans represent

secured borrowings which are a very important source (and sometimes, the major
resource) for financing new projects as well as for the expansion, modernization, and
renovation scheme of existing firms. There are two broad types of term loans
available ;

Domestic currency loans: given for financing land, building, civil works, indigenous
plant and machinery ,etc.

Foreign currency terms of loans: provided for meeting the foreign currency
expenditures towards the import of equipment and technical know-how.

Debenture capital:- akin/similar/ to promissory notes, debentures are instruments for

raising debt capital. There are two broad types of debentures;

Non-convertible debentures are straight debt instruments. Typical they carry a fixed
rate of interest and have a maturity period of 5 to 9 years.

Convertible debentures, as the name implies, are debentures which are convertible,
wholly or partly, into equity shares. The conversion period and price are announced in
advance.

Deferred credit: - many time the suppliers of the plant and machinery offer a deferred
credit facility under which payment for the purchase of the plant and machinery can
be made over a period of time.

Incentive source:- the government and its agencies may provide financial support as
an incentive to certain types of promoters or for setting up industrial units in certain
locations.

These incentives my take the form of seed capital assistance (provided at a normal
rate of interest to enable the promoter to meet his contribution to the project) or

Capital subsidy (to attract industries to certain locations) or

Tax deferment or exemption (particularly from sale tax) for a certain period.

Miscellaneous source:- a small portion of the project finance may come from
miscellaneous sources like
22
Unsecured loans are typically provided by the producers to bridge the gap between the
promoters‘ contribution (as required by the financial institutions) and the equity
capital the promoters can be subscribe it.

Public deposits represent a form of borrowing from the public at large

Leasing and hire purchase finance represent a form of borrowing different from the
conventional terms loans and debenture capital

Costs and Benefits of Projects

We undertake economic analysis of projects to compare costs with benefits and

determine which among alternatives project have an acceptable economic return and
we do the same for financial analysis. The costs and benefits of a project therefore
must be identified. Furthermore, once costs and benefits are known they must be
priced and their economic value determined.

Objectives, costs and benefits

In identifying costs and benefits of a project, objectives play important role. In project
analysis, the objectives of the project provide the standard against which cost and
benefits are defined. Simply put, a cost is anything that reduces an objective, and a
benefit is anything that contributes to an objective. The problem with such simplicity,
however, is that each participant in the project has many objectives.

For example - A private business farm investment can have objectives such as:

- Maximizing net income (profit), Increasing market share, Improving customer

satisfaction, Reducing risk, etc.

A society or a nation as a whole may want to achieve the following objectives as:

- Increasing national income (growth objective)

Ensuring equitable distribution between persons, regions, generations, etc.

(distributional objective),Improving balance of payments,Improving regional
integrity,Reducing inflation,Reducingunemployment,Maintaining environment, etc

In financial analysis, which is conducted from the viewpoints of the private project-
operator, we will evaluate the project in terms of its contribution to the net income

23
(profit) of the private owner (which is usually considered to be the fundamental
objective of the private business firm).

The project that will generate the highest profit for the owner will be given priority to
other alternative projects.

In contrast to this, in economics analysis, which is conducted from the standpoint of

the society as a whole, we will evaluate a project in its contribution to the national
income - the value of all final goods and services produced during a particular period,
usually a year.

Thus, project that contributes the highest to the national income and also that makes a
significant contribution to other social objectives will be selected.

E.g. If two agricultural projects contribute equal income to national income, we will
choose the one that favor equitable distribution or the one that creates the most jobs,
etc.

Categories of Costs and Benefits

Benefits of the Project

The financial evaluation aims at assessing the financial and commercial feasibility of
a project from the point of the investors and financer. The enterprise‘s performance
within a business environment is analyzed taking all expenses for project inputs as
cash outflows, and the income from operations as cash inflows.

The financial analysis establishes the magnitude of costs of investment, production

and overheads and magnitude of benefits. This analysis will be the basis for
evaluating the project profitability. Project profitability depends on a comparison of
costs versus revenues using realistic market prices of materials, labor and outputs.

Determining Relevant Cash Flows

Elements of cash inflows of the project under consideration can be described as

follows. Project cash inflows and outflows are expected to appear from the following
sources.

Sales of the products or services:- there are the principal sources of income for the
project for which it has been established. It represents the dominance source of cash
flows and is termed as cash flows from operations.
24
Sales of by products:- some projects may have byproducts that are salable and serve
as source of cash inflows. For example in sugar factory, the molasses is a salable
byproduct.

Recovery of net working capital:- by the end of the planning horizon of the project the
initial working capital of the project is expected to be recovered and represents
another inflow of cash;

Other miscellaneous sources:- a project may have inflow of cash from other minor
sources such as investment of idle cash temporarily or sale of old assets

Tangible benefits of projects

Increased production:- increased physical production is the most common benefit of

projects. Whether the increased output is marketed or consumed at home, it represents
the benefit of a project.

Quality improvement:- to account as a benefit in both financial and economic analysis

this must be reflected in the market price of the good.

Change in time of sale:- In some projects, especially in agriculture, benefits will arise
from improved marketing facilities that allow the product to be sold at a time when
prices are more favorable. (Marketing function that adds time utility). The benefits of
these projects arise out of the change in ―temporal value‖.

Change in location of sale:- Such projects as investment on transport facilities to carry

products from the local area where price are low to distant market where prices are
higher. The benefits of such projects arise from the change in ―location value‖.

Change in product form (grading & processing):- projects involving agricultural

processing industries expect benefits to arise from a change in the form of the
agricultural products.

Cost reduction (through mechanization):- The classical example of a benefit arising

from cost reduction in projects is the gained by investment in agricultural machinery
to reduce labor costs. In other industries also use of improved technologies that
substitute labor could be an incremental benefit from the reduction in cost of labor as
compared to the 'without' condition.

25
Losses avoided:- The ‗with and‘ without‘ project analysis tends to point out such costs
avoided by the project. Similarly risks avoided or reduced can be considered as
benefits; sometimes such benefits are reflected by output increment through loss
reduction.

Since all these benefits are real increase in value of commodities or reduction in costs,
they will be considered in both analyses.

Cost of Farm Investment/Project:

Conceptually, the cost of project/investment/ represents the total of all items of outlay
associated with a farm investment/ or project which are supported by long-term funds.
Hence the farm investment has the following major categories of costs/ cash outflows.
The project, therefore, will have the following major categories of each outflow.

Initial investment cost

These are defined as the sum of fixed assets/ fixed investment costs plus pre-
production expenditures/ and net working capital. Expenditures for fixed assets
constitute the resources required for constructing and equipping an investment project.

Net working requirement corresponds to the resources needed to operate the project
outlay or partially. Their breakdown is stated as follows;

Investment cost=fixed capital+ net working capital

Fixed capital = fixed investment+ pre-production capital costs

Hence;

Investment costs=fixed investment + pre-production capital costs + net-working

capital

Fixed capital costs

Fixed capital costs are costs of fixed investments such as the costs of the following:

26
Land acquisition and site preparation costs, Basic cost of land including conveyance
and other allied charges, Premium payable on leasehold and conveyance and other
allied charges, Cost of leveling and issue expenses, Pre-operative expenses, Margin
money for working capital, Initial cash losses, Building and civil work costs, Plan,
machineries and equipment costs,Pre –production Capital Costs

Pre-production costs incurred prior to commercial production. These costs need to be

capitalized; Preparatory study costs

Expenditures for pre-investment studies such as opportunity study, pre-feasibility and

feasibility study and support and financial studio; Preproduction manpower cost

Consultant fees for preparing studies, engineering and supervision of erection and
construction

Trail run, start up and commissioning expenditures

Other expenses preproduction costs such as preproduction marketing know-how and

patent fees, training costs etc.

Net working capital requirement: is computed as current assets less current less
current liabilities in operating the project during the planning horizon.

Net working capital represents the liquid assets which will be tied up in the project.
The reason why we treat net working capital as an investment cost is without them,
the project will not function.

So it is part of the investment costs. It forms essential part of the initial capital outlays
required for an investment project, because it is required to finance the operation of
the plant. Any change in current assets or liabilities has an impact on the financial
requirements.

Production Costs- production costs include the following three main categories of
costs:

Material costs (direct)- as were discussed previously, various types of materials and
parts are consumed in running the project.
27
Labor costs(direct)-represent the costs incurred in relation to the human resource of
the organization

Utilities consist of power, water, and fuel.

Factory overhead costs-represent indirect materials and parts, indirect labor and other
overhead costs such as depreciation of facilities and equipment, etc.

Other Costs- included in this category are costs incurred in the process of producing
and selling goods and services but other than production department‘s costs. These
represent operating and other expenses of the company.

Administration costs

Sales and distribution costs (marketing costs)

Special issues in the determination of relevant cash flows are what are listed below:

Identify sunk costs; money already spent or committed is irrelevant to the decision.
We are concerned with only future cash flows.

Identify opportunity costs: existing resources used should be included at their

economic or opportunity cost, which is the cost incurred as a result of diverting the
resource from its next best available use to the project under review.

For example, if a project occupies premises which could otherwise be rented at Br

1000 per annum, then that Br 1000 could be regarded as a comparative cash outflows
I(in fact it is a cash inflow forgone).

Interest Payments:- since the analysis is based on discounting, it would be double

counting to include the interest payments on the financial resources used to fund the
project in the cash flows. Interest arises because money has a time value and it is
precisely this time value which discounting and compounding is designed to account.

Dividend Payments:-represent appropriation of the benefits of project acceptance

rather than a relevant element in its appraisal. Incremental dividend cash flows should
thus be ignored.

Taxation payment:-these are a cash outflow when they are paid;

28
Scrap or terminal proceeds:- where any equipment used in a project is scraped, the
proceeds are a cash inflow.

Non- cash outlay expenses:- expenses such as depreciation and amortization (pay-
back) are non-cash outlay expenses which are not relevant to determine cash flows of
a project.

Pricing Project Costs and Benefits

Once costs and benefits have been identified, if they are to be compared, they must be
valued. Since the only practical way to compare differing goods and services directly
is to give each a money value, we must find the proper prices for the costs and
benefits in our analysis.

Finding Market Prices

Project analysis characteristically are built first by

Identifying the technical inputs and output for a proposed investment,

Then by valuing the inputs and outputs at market prices to construct the financial
accounts, and

Finally by adjusting the financial prices so they better reflect economic values.

Thus, the first step in valuing costs and benefits is finding the market prices for the
inputs and outputs. The project will have to consult many sources such as merchants,
consumers, experts, published statistical bulletins, etc.

Point of first sale and farm-gate price

Point of sale means market price which include all selling costs, taxes, and profit
margin of retailer

Farm- gate price means price at project location.

In project analysis, a good rule for determining a market price for agricultural
commodities produced in the project is to seek the price at the ―point of first sale‖.

The increased value added of the product as it goes to higher markets in the channel
arises as a payment for marketing services. Thus, if the project includes such
marketing services in its design, we can take these higher prices.
29
Even in this case, the analyst must make the project as small as possible and try to
analyze the marketing service component independently of the production component.

If the product is sold only in central markets, no local market, then the analyst must
find out the value of marketing service to arrive at price at project site.

Prices for some products like agricultural products generally are subjected to
substantial seasonal fluctuation.

If this is the case as it may often is some decision must be made about the price in the
seasonal cycle at which to choose the price to be used for the analysis.

A good starting point is the farm-gate price at the peak of the harvest season. This is
probably close to the lowest price in the cycle. The reasoning is that the rise in price is
due to marketing services.

Predicting Future Prices

Since project analysis is about judging future returns from future investment, we have
to judge what the future prices of inputs and outputs may be. The best starting point is
;

To see the trend of these prices over the past few years.

Having this data, the project analyst can forecast the price with certain degree of
precision.

However, even then judgment is important to arrive at what price we have to use to
value inputs and outputs of the project. Moreover, we have to keep in mind that, as
projects involve distant future, the prediction power of the model will decline as we
go far from the present.

Change in prices

Change in prices could be general change in price or change in relative prices of

goods.

Change in relative price

If relative price of inputs or outputs are variable over time, i.e.,

30
Where Px-price of commodity and Py- price of commodity.

These changes in relative price of items imply a change in marginal productivity of

inputs in production or a change in marginal satisfaction (MU) in consumption.

Thus, changes in relative prices have a real effect on the project objective and must be
reflected in project accounts in the years when such changes are expected. This can be
judged from past trend.

For instance, the price of agricultural products to price of inputs (manufactured) may
rise over time. This would have a real effect on the net benefit of the firm.

Inflation: an increase in general prices of goods

Inflation is common for every country although the magnitude may vary between
countries. However, the approach most often taken is to work the project analysis in
constant price.

Elements of Farm Investment Analysis

Estimates of sales and production

Typically, the starting point for profitability projection is the forecast of sales revenue.
In estimating sales revenues, the following considerations should be borne in mind.

It is not advisable to assume a high capacity utilization level in the first year of
operation. Even if the technology is simple and the company may not face technical
problems in achieving a high rate of capacity utilization in the first year itself, there
are likely to be other constraints like raw material shortage, limited power, marketing
problems, etc.

31
It is sensible to assume that capacity utilization would be somewhat low in the first
year and rise there after gradually to reach the maximum level in the third or forth
years of operation.

A reasonable assumption with respect to capacity utilization is as follows: 40-50

percent of the installed capacity in the first year, 50-80 percent in the second year, and
80-90 percent from the third year onwards.

Estimates of the material cost

While estimating the material cost, the following points should be borne in mind.

The requirements of various material inputs per unit of output may be established on
the basis of one or more of the following:

Theoretical consumption norms

Experience of industry/ farm projects

Performance guarantees, and

Specification of machinery supplies

The total requirement of various material inputs can be obtained by multiplying the
requirements per unit of output/ per square meter, etc./ with expected output during
the year.

The prices of material inputs are defined in CIF (cost, insurance, and freight0 terms.

The present costs of various material inputs is considered. In other words, the factor of
inflation is ignored. It may be recalled that the factor of inflation is ignored in
estimating the sales revenues too.

If seasonal fluctuation in prices is regular, the same must be considered in estimating

the costs of material inputs.

Working Capital Requirement and Its Financing

In estimating the working capital requirement and planning for its financing, the
following points to be born in mind:
32
The working capital requirement consists of the following;

-Raw materials and components (indigenous as well as imported)

-Stocks of goods- in- process (also referred to as work- in process)

-Stock of finished goods

-Debtors

-Operating expenses, and

-Consumable stores

-The principal sources of working capital finance are:

-Working capital advances provided by the commercial banks

-Trade credit

-Accruals and provisions ,and

-Long- term sources of financing

There are limits to obtaining working capital advances from commercial banks. They
are two forms:

The aggregate permissible bank finance is specified as per the norms of lending
banks, followed by the lending bank

Against each current asset a certain amount of margin money has to be provided by
the firm

The margin requirement varies with the type of current asset. While there is not fixed
formula for determining the margin amount, the ranges within which margin
requirement for various current assets lie as follows;

Current assets margin

33
6. Financial Analysis of Investment:

Balance Sheet

The cash flow statement shows the movement of cash into and out of the firm/ farm
project/ and its net impact on the cash balances within the firm/farm project. The
Balance sheet, showing the balance in various Asset and Liabilities accounts, reflects
the financial conditions of the firm/ farm projects) at a given point of time. The format
of a balance sheet as prescribed by APC Flower Farm Project is given in Exhibit 4.3.

Total asset =outstanding debt+ owners‘ equity (or, Asset=Capital +liability)

To illustrate how the projected cash flow statement is prepared let us considered a
simple example. The balance sheet of APC Flower Farm Enterprise at the end of year
n (the year which is just over) is as follows.

Exhibit 4.3. Format of Balance Sheet Prescribed by APC Flower Farm Enterprise
(million birr)

Assets Liabilities

Liabilities

34
The liabilities side of the balance sheet shows the sources of finance employed by the
business. A word about its components shown on the right hand side of Exhibit 4.3. is
in order.

Share capital consists of paid-up equity and preference capital

Reserves and surplus represent the accumulated retained earnings. They are shown in
different accounts like the capital reserve, the investment allowance reserve, and the
general reserve.

Secured loans represent the borrowing of the firm /farm project/ against which
security has been provided. The important components of secured loans are
debentures, term loans from financial institutions, and loans from commercial banks.

Unsecured loans represent borrowings against which no-specific security has been
provided. The important constituents are: fixed deposits from public and unsecured
loans from promoters.

Current liabilities are obligations which mature in the near future, usually a year. Thus
obligation arises mainly from items which enter the operating cycle: payables from
acquiring materials and supplies used in production, and accruals of wages, salaries
and rentals.

Provisions include mainly tax provision, provisions for provident fund, proviso for
pension and gratuity /privilege/, and provision for proposed dividends.

Assets

The assets side of the balance sheet shows how funds have been used in the business.
The major asset components may be described briefly.

Fixed assets are tangible long-lived resources ordinarily used for producing goods and
services. They are shown at original cost less depreciation.

Investments represent financial securities owned by the firm

Current assets, loans and advances consists of cash, debtors, inventories of different
kinds, and loans and advances made by the firm.

Miscellaneous expenditures and losses represent outlays not covered by the previously
described asset accounts and losses, if any.
35
For preparing the projected balance sheet at the end of year n+1, we need information
about the following:

The balance sheet at the end of year n

The projected income statement and distribution of earnings for the year n+1

The sources of external financing proposed to be tapped in the year n+1

The proposed payments of debt capital (long-term, intermediate term, and short-term)
during the year n+1

The outlays and disposal of fixed assets during the year n+1

The changes in the level of current assets during the year n+1

The changes in other assets and certain outlays like preoperative and preliminary
expenses (which are capitalized) during the year n+1

The cash balance at the end of year n+1

Income Statement

Income statement ( or financial statement, or accounting statement) shows the

projected profit and loss statement. Therefore, in order to prepare income statement in
each year there is a need to project profitability of the project based on the following
information. That is:

Profitability Projections (or Estimates of Working Results)

Given the estimates of sales revenues & cost of production, the next step is to prepare
the profitability projections or estimates of working results (as they are referred to by
term-lending financial in Ethiopia). The estimates of working results may be prepared
along the following lines:

Cost of production ( cost of materials , labor, utilities, factory overheads--)

Total administration expenses (salaries, remuneration , professional fees light,

telephone, printing------)

Total sales expenses ( commission payable, packaging & forwarding charges, salary
of sales staff, and other miscellaneous expenses----------)
36
Royalty & know-how-payable ( it may be 2-5 percent of sale and is payable often for
a limited number of years, say 5 to 10 years)

Total cost of production (A+B+C+D)

Expected sales (estimated sales)

Gross profit before interest (difference between expected sales & total cost of
production)

Total financial expenses (interest expenses on terms loans, interest on bank

borrowing--------)

Deprecation ( the depreciation rates of the company law presupposes; building 3.4%,
plant and machinery 8.09% and miscellaneous fixed asset 5.15%)

-Operating profit (G-H-I)

-Other income

-Preliminary (beginning) expenses written off

-Profit (loss before taxation (J+K-L)

-Provision for taxation ( may be 45%)

-Profit after tax (M-N)

-Less: Dividend on

-Preference capital

-Equity capital

-Retained profit

-Net cash accrual (accumulation) (P+I+L)

Example : Projected Profit and Loss Statements of APC Flower Farm Enterprise
(million birr)

37
38
Chapter 4
PLANNING FOR MONITORING AND EVALUATION

4.1 INTRODUCTION

WHY MONITOR AND EVALUATE?

Monitoring and evaluation serve several purposes. In the absence of effective

monitor-ing and evaluation, it would be difficult to know whether the intended results
are being achieved as planned, what corrective action may be needed to ensure
delivery of the intended results, and whether initiatives are ma king

positive contributions towards human development. Monitoring and evaluation

always relate to pre-identified results in the development plan. They are driven by the
need to account for the achievement of intended results and provide a fact base to
inform corrective decision making. They are an essential management tool to support
the UNDP commitment to accountability for results, resources entrusted to it, and
organizational learning. Furthermore, both feed into the overall programmed
management processes and make an essential contribution to the ability to manage for
development results.22

39
Monitoring, as well as evaluation, provides opportunities at regular
predetermined points to validate the logic of a programme , its activities and
their implementation and to make adjustments a s needed. Good planning and
designs a lone do not ensure results. Progress towards achieving results needs
to be monitored. Equally, no amount of good monitoring a lone will correct
poor programme designs, plans and results. Information from monitoring
needs to be used to en courage improvements or reinforce plans. Information
from systematic monitoring also provides critical input to evaluation. It is very
difficult to evaluate a programme that is not well designed and that does not
systematically monitor its progress.

The key questions that monitoring seeks to answer includes the following:

-Are the preidentified outputs being produced as planned and efficiently?

-What are the issues, risks and challenges that we face or foresee that need to
be taken into account to ensure the achievement of results?

-What decisions need to be made concerning changes to the already planned

work in subsequent stages?

-Will the planned and delivered outputs continue to be relevant for the
achieve-ment of the envisioned outcomes?

-Are the outcomes we envisaged remaining relevant and effective for

achieving the overall national priorities, goals and impacts?

-What are we learning?

Like monitoring, evaluation is an integral part of programme management and

a critical management too l. Evaluation complements monitoring by pro
viding an independent and in-depth assessment of what worked and what did
not work, and why this was the ca se. After implementing and monitoring a n
initiative for some time, it is an important management discipline to take stock
of the situation through an external evaluation.

WHY PLAN FOR MONITORING A ND EVALUATION?

Effective and timely decision making requires information from regular and
planned monitoring and eva luation activities. Planning for monitoring and
evaluation must start at the ti me of p rogramme o r project design , and they
40
must be p lanned together. While monitoring provides real-time information
on ongoing programme or project implementation required by ma nagement,
evaluation provides more in-depth assessments. The monitoring process can
generate questions to be answered by evalua-tion. Also, evaluation draws
heavily on data generated through monitoring, including baseline data,
information on the programme or project implementation process, and
measurements of progress towards the planned results through indicators.

Planning for monitoring must be done with evaluation in mind: The

availability of a clearly defined results o r outcome model and monitoring da
ta, among other things, determine the ‗evaluability‘23 of the subject to be
evaluated.

3.2 MONITORING AND E VALUATION FRAME WORK

A clear framework, agreed among the key stakeholders at the en d o f the

planning stage, is essential in order to carry out monitoring and evaluation
systematically. This framework serves as a plan for monitoring and
evaluation, and should clarify:

 What is to be monitored and evaluated?

 The activities needed to monitor and evaluate
 Who is responsible for monitoring and evaluation activities
 When monitoring and evaluation activities are planned (timing) How
monitoring and evaluation are carried out (methods)
 What resources are required and where they are committed

In addition, relevant risks and assumptions in car rying out planned

monitoring and evaluation activities should be seriously c onsidered,
anticipated and included in the M&E framework.

In general, the M&E framework has three main components:

1. Narrative component—This describes how the partners will undertake `

3 9JJ V and eva1a
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww luation a
nd the a ccountabilities assigned to different in dividuals a nd agencies. For
example, at the UNDAF or national result level, it is n ecessary to engage with

41
 national monitoring committees or outcome level groups (e.g. sector
arrangements) a s well a s with U N intera gency monitoring w orking groups.
If these do not exist, there might be a need to establish such structures for
effective monitoring and evaluation. In addition the narrative should also
reflect:

Plans that may be in place to strengthen national or s ub-national monitoring

and evaluation capacities

Evaluability can be defined by clarity in the intent of the subject to be

evaluated, sufficient measurable indicators, accessible reliable information
sources, and no major factor hindering an impartial evaluation process.

Existing monitoring and evaluation capacities and an estimate of the

human, financial and material resource requirements for its
implementation

Results framework—This should be prepared in the planning stage as

described in Chapter 2.

Planning matrices for monitoring and evaluation—These are strategic and

consoli-date the information required for monitoring and evaluation for eas
refer

42
4
3
of an evaluation and effective use of evaluation information, the evaluation
should be made available in a timely manner so that decisions can be made
informed by evaluative evidence.26

Resources invested—An area (thematic or programmatic area, outcome or

project) in which UNDP has invested significant resources may be subject to
an evaluation as there may be greater accountability requirements.

The likelihood of future initiatives in the same area—Evaluations are an

important means o f gener ating r ecommendations to guid e future w ork. A n
ev aluation enables the programme unit to take stock of whether the outputs
have contributed to the outcome and whether UNDP has crafted an effective
partnership strategy. When selecting an initiative to be evaluated, look for one
in an area that UNDP will continue to support.

Anticipated pr oblems—Evaluations c an help prevent problems a nd p rovide

a n independent perspe ctive on e xisting p roblems. Wh en se lecting a n o
utcome for evaluation, look for those with problems or where complications
are likely to arise because the outcome is within a sensitive area with a
number of partners.

Need for lessons learned—What kinds of lessons are needed to help guide
activities in this country or other countries or regions in the region?

Alignment and har monization—Planned e valuations sho uld b e aligned w

ith national, regional and global development priorities and UNDP corporate
priori-ties (for ex ample, the U NDP Stra tegic Plan), and sh ould be
harmonized with evaluations of UN system o rganizations and othe r inte
rnational pa rtners. This ensures that pr oposed ev aluations wi ll generate
important info rmation to h elp

3.3 RESOURCES FOR MONITORING AND EVALUATION

Inadequate resources lead to poor quality mo nitoring and evaluation. To en sure

effective and quality monitoring a nd eval uation, it is critical to set a side a dequate

44
financial and human resources at the planning stage. The required financial
and human resources for monitoring and evaluation should be considered
within the overall costs of delivering the agreed results and not as additional
costs.

Financial resources for monitoring and evaluation should be estimated

realistically at the time o f planning for monitoring and evaluation. While it is
critical to plan for monitoring and evaluation together, resources for each
function should be separate. In practice, each project should have two separate
budget lines for its mon itoring and evaluation agreed in advance with
partners. This will help UNDP and its partners be more realistic in budgeting.
It will also reduce the risk of running out of resources for evaluation, which
often takes place towards the end of implementation.

Monitoring and evaluation costs associated with projects can be id entified

relatively easily and be charged directly to the respective project budgets with
prior agreement among partners through inclusion in the project budget or
Annual Work Plan (AWP) signed by partners.

Sourcing and securing financial resources for monitoring and evaluation of

outcomes or programmes can pose additional challenges, as there is not one
project where these costs can be directly charged. The most commonly
observed financing mechanism is to draw resources together from relevant
projects. Some additional possibilities include:

Create a sepa rate monitoring a nd ev aluation fund, facility or p roject a

ssociatedwith an outc ome o r a p rogramme to which all the constitue nt
projects w ould

45
contribute through transfer of some project funds. This facility could be
located in the same entity that manages the outcome or programme.

Mobilize funds from partners directly for an outcome or p rogramme

monitoring and evaluation facility.

Allocate required funds annually for each outcome on the basis of planned
costs of monitoring and evaluation from overall programme budget to the
facility or fund.It is important that partners consider the resources needed for
monitoring and evaluation and agree on a practical arrangement to finance the
associated activities. Such arrange-ments should be documented at the
beginning of the programme to enable partners to transfer necessary funds in
accordance with their procedures, which could take consid-erable time and
effort.

Human resources are c ritical for effective monitoring and eva luation, even
after securing adeq uate fi nancial r esources. For h igh-quality monitoring a
nd ev aluation, there should be:

Dedicated staff time—For ef fective mo nitoring a nd eva luation, sta ff sh

ould be dedicated for the function. The practices of deployment of personnel
for monitor-ing var y am ong or ganizations. Som e U NDP country offices ha
ve esta blished monitoring and eval uation un its with specif ic t erms o f r
eferences (T oRs), dedicated skilled staff, work plans and other resources.

Skilled personnel—Staff entrusted with monitoring should have required

technical expertise i n the area. A num ber of UNDP country offic es have a
dedic ated monitoring and ev aluation sp ecialist. Where necessary, skill levels
should be augmented to meet the needs and with ongoing inve stments in dev
eloping such capacity within the office as necessary.

Each monitoring and evaluation entity that functions at diffe rent levels, for
example at the project, programme or outcome level, should have a clear ToR
outlining its role and responsibilities. In general, these responsibilities should
include:

Setting up systematic monitoring frameworks and developing an evaluation

plan

46
 Meeting regularly with key partners and stakeholders to a ssess progress
towards achieving the results

Conducting joint field monitoring and evaluation missions to assess

achievements and constraints

Identifying any lessons or good practices

Reflecting on how well the results being achieved are addressing gender, and
the interests and rights of marginalized and vulnerable groups in the society

Identifying additional capacity development needs among stakeholders and

partners

Reporting r egularly to th e lea d i ndividuals or a gencies for th e par ticular

res ult areas and seeking opportunities to influence policy and decision-
making processes

Ensuring the quality of monitoring and evaluation work and providing

guidance as needed
47
Assessing the r elevance of th e M&E framework on a re gular basis based
on emerging development priorities and changing context

ENGAGEMENT OF STAKEHOLDERS IN MONITORING AND

EVALUATION

The engagement of stakeholders enlisted during planning and described in

continues to be relevant for monitoring and evaluation stages for the
following reasons:

The stakeholders, who set th e vision and the prioritized resu lts to r ealize
that vision during the planning stage, have the best ideas on how the results
would continue to remain relevant to them. They must therefore be
involved in identify-ing the information or feedback that is needed during
implementation, which determines the parameters for monitoring and
evaluation.

Having set the visi on, priority results and initial parameters for monitoring
and evaluation, the key stakeholders are best placed to e nsure that the
programmatic initiatives planned would deliver what was intended and the
way it was intended.

Stakeholder participation in monitoring and evalu ation can p roduce eff

ective communication for v arious other objectives. These include: fa
cilitate communication of ‗early wi ns‘ to increase s upport a nd enlist
engagement of th ose who are n ot yet engaged, ensure access of early
products and services of initiatives for intended benefi-ciaries, mobilize
additional resources to fill resource gaps, and ensure effective use of
lessons learned in future decision making.

Stakeholder p articipation th roughout the p rogramming cycle ensures o

wnership, learning a nd su stainability of results. Continued stakehold er pa
rticipation in monitoring and ev aluation cannot be assumed. It must be ins
titutionalized.

Specific measures have to be built into programme and project

management processes to ensure continued and effective involvement of
stakeholders.

INSTITU TIONA L ARRANGEMENTS

48
Is there a documented institutional or sector programme monitoring and
evalua-tion policy that clarifies the mandates of monitoring a nd evaluation
entities and programme or project teams, their responsibilities, and
accountability measures for effective data collection and data management
of public programmes or projects?

Does th e i nstitutional a nd s ector policy manda te r equire: establishing

sta ndard tools and templates, aligning organizational data with the
national data collectionand mana gement, defining s tandards for
monitoring and e valuating sk ills, a nd ensuring proper training?

Are sufficient resources, including availability of skilled sta ff a nd

financial resources, allocated for monitoring and evaluation activities in
respective monitor-ing and evaluation entities? Do monitoring staff have
proper statistical and analyt-ical skills to compile and analyse sample and
snapshot data?

Is there an independent evaluation entity? Is the institution responsible for

evalua-tion truly ‗independent‘ from management and subject to
evaluation? What is the reporting line of those responsible for carrying out
evaluations? What mechanisms are there to safeguard the independence of
the evaluation function?

LEADERSHIP

Does high-level management support evidence-based decision making

through-out the organization?

KNOWLEDGE

Can high-quality information be disaggregated by relevant factors (such as

gender, age and geography) to assess progress and analyse performance?

Do the respective monitoring and evaluation entities h ave access to all

relevantprogramme or p roject i nformation to be ga thered? Do the sta
keholders h ave access to data collected and analysed (for example through
the Internet)?

Do the monitoring a nd evaluation entities have easy-to-understand

formats for data collection and reporting? Is there a systematic and doc
umented process of ensuring data quality control at all levels of collection,
analysis and aggregation?

49
Is there sufficient evaluation technical expertise in the national system?
Are there national professional evaluation associations?

MONITORING FOR RESULTS

The previous chapter provided guidance on how to plan for monitoring and
evaluation including developing an M&E framework and effectively
addressing other planning needs, such as securi ng re sources and ca pacities
for implemen ting mo nitoring and evaluation activ ities. This c hapter
provides s tep-by-step guidance on ho w to implement planned m onitoring a
ctivities. I t also p resents useful to ols a nd tips fo r effective monitoring and
use of monitoring evidence in decision making.

The chapter follows the general steps of implementation of monitoring:Have a

clear common understanding of the following:

The monitoring policies applicable to the respective monitoring entity

Relevant roles and responsibilities and how they are applied in monitoring for
both ou tcomes a nd outputs, and m anagement entities in p rojects and
programmes

Commonly used monitoring tools and approaches

Reinforce and elaborate the initial monitoring framework (described in

Chapter 3) with detailed information needed to implement monitoring actions.
This includes finalizing reference points for periodic monitoring s uch as

50
indicators, baselines, risks, and annual targets, and locking them in monitoring
information systems.

Implement monitoring actions: organize, plan and implement monitoring

actions, using selected tools for collection and analysis of data and reporting.

Use monitoring data objectively for management action and decision making.

Project and output level

The project is the entity that uses inputs and resources and converts them to
activities and ou tputs. It i s also th e enti ty from whi ch mon itoring actions
beg in. Outputs generated by projects are always connected directly to an
outcome.28 UNDP projects normally operate in complex development settings
and it is impor tant to be c lear on each project‘s role, deliverables and
outputs, and their connections to other projects to avoid mix ups.

There is a critical responsibility at each project level with regards to the

generation of the planned output through a carefully planned set of relevant
and effective activities, and proper use of resources allocated for those
activities. Both these aspects must be monitored. The primary responsibility
for monitoring at the project or output level lies

In some cases, it is also possible that an output may be con nected to more
than one outcome. For example, a database on displaced co mmunities
generated by o ne project could serve n ot only an outcome on safety of the
displaced, but also other outcomes relating, inter alia, to their education and
nutrition and health standards, etc

51
with the project manager. Primary monitoring tools used at the project level
by UNDP are: the corporate project management system (Atla s); field visits,
consultations and reviews with stakeholders; Annual (and quarterly) Projec t
Repor ts; and the A nnual Project Review Process.

Outcome level

The outcomes are achieved by the generation of outputs thr ough projects (and
other related activities such as soft advocacy). These projects and related
activities could be supported by UNDP or others.

4.3 MONITOR: COLLECTION OF DATA, ANALYSIS AND RE

PORTING

SCOPE OF MONITORIN G

Monitoring aims to identify progress towards results, precipitate decisions that

would increase the likelihood of achieving results, enhance accountability and
learning. All monitoring efforts should, at a minimum, address the following:

Progress towar ds outcomes—This e ntails peri odically a nalysing th e exten t

to which intended outcomes have actually been achieved or are being
achieved.

Factors contributing to or impeding achievement of the outcomes—This

necessitates monitoring the country context and the economic,
sociological,political and other developments simultaneously taking place and
is closely linked to risk management.

Individual partner contributions to the outcomes through outputs—These

outputsmay be g enerated by programmes, projects, p olicy a dvice, ad vocacy
and other activities. T heir m onitoring a nd evaluation entails a nalysing w
hether o r n ot

52
outputs are in the process of being delivered as planned a nd whether or not the
outputs are contributing to the outcome.

Partnership strategy—This requires the review of current pa rtnership str

ategies and their functioning as well as formation of new pa rtnerships as
needed. This helps to ensure that partners who are concerned with an outcome
have a common appreciation of problems and needs, and that they share a
synchronized strategy.

Lessons being learned and creation of knowledge products for wider sharing.

Partners may add additional elements where needed for management or

analysis, while keeping a realistic scope in view of available capacities.
Monitoring usually provides raw data that requires fur ther analysis and
synthesis prior to reporting for decision making. Using information gained
through monitoring, p rogramme managers must analyse and take action on the
programme and project activities to ensure that the intended results—results
that are in the agreed results and resources frameworks—are being achieved.
Managers of programmes also monito r and doc ument the contribu-tions of
soft development initiatives and strategic partnerships.

PRIOR ITIZIN G MO NITORING

In practice, it is nec essary to pri oritize monitoring. Two fac tors ca n h elp as
sign monitoring p riority: criticality of a UNDP contribution to t he a ttainment
o f the overall result; and the severity of risks it faces. As the criticality and
severity of risks change, the corresponding priority attached monitoring of an
initiative also changes.

Criticality of a UNDP project or an initiative is considered high when: it is

connected with a tight time-bound high national priority; there is critica l
reliance on relevant UNDP comparative s trengths, e xpertise and comp
etencies for the achievement of planned results; or it involves a critical UNDP
coordination role entrusted by govern-ment and other partners.

Risks are initially identified in th e results frameworks with th eir potential

impacts. However, during programme and project implementation, additional
risks may arise from a cha nging o perational e nvironment (such as a c risis)
th at may h ave to be factored in when prioritizing monitoring.
53
Based on the two criteria of criticality and risks, as indicated in Figure 15, it is
possible to determine four broad categories to assign priority in monitoring. It
is also possible to identify which of the two aspects should be followed more
closely.

Chapter Six

EVALUATING FOR RESULTS

CHAPTER 5

This chapter presents a holistic view of the UNDP evaluation function in order
to help managers and staff of programme units and partners make strategic
decisions about evaluations. The chapter describes why evaluation is important
for UNDP and how evaluative information should be used, then briefly
presents the UNDP evaluation policy, types of evaluations that are commonly
conducted in UNDP, eyk roles and responsibilities in evaluation, and
evaluation requirements as stipulated in the evaluation policy.

5.1 WHY EVALUATE? USES OF EVALUATION

Evaluation is critical for UNDP to progress towards advancing human

development. Through the genera tion o f ‗evidence‘ and objec tive
information, evaluations en able managers to make informed decisions and
plan strategically. UNDP success depends, in part, on the ability of UNDP and
its counterparts to carry out credible evaluations and use th em to make
evidenced-based decisions. The effective conduct and use o f evaluation
requires adequate human and financial resources, sound understanding of
evaluation and most importantly, a culture of results-orientation, learning,
inquiry and evidence-based decision making. Everyone in UNDP and its
stakeholders have to share the same vision and be open to change.

When evaluations ar e us ed e ffectively, they sup port progra mme impro

vements, knowledge generation and accountability.

54
 Supporting programme improvements—Did it work or not, and why? How
could it be done differently for better results?

The interest is on what works, why and in what context. Decision makers, such
as managers, use eva luations to ma ke nec essary imp rovements, a djustments
to the implementation a pproach or s trategies, and to d ecide on a lternatives.
E valuations addressing these qu estions need to p rovide c oncrete informa
tion on how improve-ments could be made or what alternatives exist to address
the necessary improvements

Building knowledge for generalizability and wider-application—What can

we learn from the evaluation? How can we apply this knowledge to other
contexts?

The main interest is in the development of knowledge for global use and for
general-ization to other contexts and situations. When the interest is on
knowledge generation, evaluations generally apply more rigorous
methodology to ensure a higher level of accuracy in the evaluation and the
information being produced to allow for generalizability and wider
application beyond a particular context.

Evaluations should not be seen as an event but as part of an exercise

whereby different stakeholders are able to pa rticipate i n th e continuous
process of generating and applying evalua tive know ledge. UNDP man
agers, tog ether w ith g overnment and other stak eholders, deci de w ho p
articipates i n w hat pa rt o f th is pr ocess ( analysing findings and lessons,
developing a management response to an evaluation, disseminat-ing
knowledge) and to what extent they will be involved (informed, consulted,
actively involved, eq ual pa rtners or key d ecision ma kers). These ar e s
trategic decisions for UN DP m anagers tha t h ave a di rect bearing o n th e
l earning a nd ow nership of evaluation findings. An evalua tion framework
th at gen erates k nowledge, pro motes learning and guides action is an
important means of capacity development and sustain-ability of results.
55
Supporting accountability—Is UNDP doing the right things? Is UNDP
doing things right? Did UNDP do what it said it would do?

The interest here is on determining the merit or worth and value of an

initiative and its qu ality. An e ffective ac countability f ramework r equires
c redible a nd obje ctive information, and eva luations ca n de liver s uch in
formation. Evaluations h elp ensu re that UND P goa ls a nd i nitiatives are
aligned w ith a nd s upport the Millennium Declaration, M DGs, and g
lobal, national a nd c orporate pri orities. U NDP is accountable fo r pr
oviding ev aluative e vidence th at l inks U NDP c ontributions to the
achievement of development results in a given country and for delivering
services that are b ased on the principles of human development. By
providing such objective and i ndependent a ssessments, ev aluations in UN
DP su pport th e or ganization‘s accountability tow ards its Executi ve
Board, do nors, g overnments, national partners and beneficiaries.

128

EVALUATION POLIC Y: PRINCIPLES, NORMS AND STANDARDS FOR

EVALUATION

The evaluation policy was adopted in 2006 to strengthen the evaluation function in
UNDP. The guiding principles, norms and standards as expressed in the policy and the
UNEG Norms and Standards for Evaluation in the UN system guide the practice and
use of evaluation i n UN DP. Norms for evaluation—how evaluation should be
conducted in order to meet the required quality standards and its intended role.

The remaining evaluation section of this Handbook aims to provide practical guidance
on how these norms and principles can be applied throughout the evaluation process.

56
5.3 TYPES OF EVALUATION IN UNDP

INDEP ENDENT AND DECEN TRALIZ ED EVALUATIONS

UNDP support and services consist of programmes, projects, partnerships and

‗soft assistance‘ such as advocacy, policy advice and coordination support,
which may or may not be delivered within a project framework. Programmes
and projects have results frameworks that detail the results map and intended
results at the output and outcome levels. Evaluations in UNDP are carried out
to adequately cover this wide range of UNDP initiatives in order to assess their
worth and merit and support the organization‘s learning efforts and
accountability. The architecture of evaluation in UNDP, therefore, corresponds
to the UNDP programmatic structure and its components.

There are two categories of evaluations in U NDP: in dependent and

decentralized evaluations. The UNDP Evaluation Office is mandated by the
Executive Board to carry out independent evaluations. They are referred to as
independent since the Evaluation Office is independent from programme
management and is not part of subsequent decision-making processes
regarding the subject o f an evaluation. The Evaluation Office is also required
to conduct country programme evaluations (known as Assessments of
Development Results or ADRs), regional and global programme evaluations,
and thematic evaluations in accordance with the programme of work that is
approved by the Executive Board.

57
The programme units carry out various types of decentralized evaluations and
ensure that they provide adequate information about the overall performance
of UNDP support in a given context. In doing so, the programme units draw
from a range of evaluation types that are based on business units of their
development assistance at the country, regional or global levels. These
include: UNDAF; country, regional or global programmes; outcomes;
thematic areas; and projects. The most common decentralized evaluations are
project and outcome evaluations. The programme units do not c onduct these
evaluations themselves, but rather commission external evaluation consultants
to do so.

Together, these two categories of evaluations are intended to pr ovide

comprehensive information about UNDP performance at the project,
programme, corporate and UN system levels, with a view to supporting sound
management of UNDP initiatives and strategic direction.

OUTCOME EVALUATION

Outcome evaluations i n U NDP assess UND P contributions towards the

progress made on outcome achievements. These outcomes are generally
identified in the programme or project results frameworks to which UNDP
initiatives contribute.

Outcome evaluations are undertaken to:

Provide evidence to support accountability of programmes and for UNDP to

use in its accountability requirements to its investors

Provide evidence of the UNDP contribution to outcomes

Meta-evaluation is an evaluation of evaluations. It uses findings from a series

of evaluatio ns and requires a robust quality assurance mechanism to ensure
that the evaluations used as secondary data are credible and of good quality.

58
131

2
3
1

59
PROJECT EVALUATION

UNDP programme units ma y commission evaluations of their respective projects as

needed. Manag ing for r esults r equires, as a sta rting po int, a g ood k nowledge of
projects, their effectiveness, internal and external factors affecting effectiveness, their
added val ue a nd their cont ribution to h igher lev el o utcomes. A p roject ev aluation
assesses the performance of a project in achieving its intended results. It yields useful
information on project implementation arrangements and the achievement of outputs.
It is at this level that direct cause and attribution can be a ddressed given the cl ose
causal linkage between the initiatives and the outputs.

The primary purpose of a project evaluation is to make improvements, to continue or

upscale an initiative, to assess replicability in other settings, or to consider alternatives.
Therefore, although project evaluations are mandatory only when required by
partnership protocols, programme units are strongly recommended to commission
evaluations, particularly of pilot programmes, before replication or upscaling, projects
that are going into a next phase, and projects more than five years in duration.
Increasingly, project evaluations play an important role in accountability to donors and
governments involved in financing projects.

IMPACT EVALUATION

An impact evaluation is an evaluation of the effects—positive or negative, intended or

not—on individual households and institutions, and the environment caused by a given
development activity such as a programme or project. Such an evaluation refers to the
final (long-term) impact as well as to the (medium-term) effects at the outcome level.

By identifying i f d evelopment a ssistance is wo rking or n ot, impact e valuation al so

serves the accountability function. Hence, impact evaluation is aligned with RBM and
monitoring the contribution of development assistance towards meeting the MDGs. An
impact evaluation is useful when:
60
REAL TIME EVALUATIONS

Real time evaluations are often undertaken at an early stage of an initiative

to provide managers with timely feedback in order to make an immediate
difference to the initia-tive. They are commonly applied in humanitarian or
post-conflict contexts to provide implementing sta ff with th e o pportunity
to an alyse w hether th e initia l response o r recovery is appropriate in
terms of desired results and process. They can also be used in crisis settings
where there may be constraints in conducting lengthier evaluations. These
constraints include the absence of baseline data, limited data collection
efforts due to a r apid turnover of s taff members (for example, lack of
institutional memory) and difficulty conducting interviews and surveys due
to security issues.

JOINT EVALUATION

Joint evaluation i s on e modali ty of car rying out a n e valuation to wh ich

d ifferent partners contribute. Any evaluation can be conducted as a joint
evaluation. Increasingly, UNDP is en gaged in joi nt evaluations and t here
a re various degrees of ‗jointne ss‘ depending on the extent to which i
ndividual partners coop erate i n th e e valuation process, merge their
evaluation resources and combine their evaluation reporting.

5.4 ROLES AND RESPONSIBILITIES IN E VALUATION

The UNDP evaluation policy outlines the roles and responsibilities of key constituents
of the organization in evaluation. Programme units and the UNDP Evaluation Office in
Headquarters carry out different types of evaluations in order to objectively assess
UNDP contributions to development results.

Senior m anagers o f the programme u nits are respo nsible for commissioning
decentralized evaluations in the programmatic areas for which they are responsible and
using the information in managing for results. In order to enhance the impartial-ity an
61
 d o bjectivity of de centralized ev aluations, the p rogramme u nits h ire exter nal
experts a nd institutions to ca rry o ut an e valuation. Decentralized eva luations hel p
ensure that UNDP r emains accountable to th e relevant programme country and its
people and is responsible for contributing to development results in the most relevant
and efficient way.

and follow up with technical guidance, and support to enhance the quality
of their work. In terms of evaluation, to enhance its independence and
technical rigour, it is advised that the M&E speci alists manage the
evaluation in c lose consultation with programme staff who are responsible
for the subject of evaluation.

Due to different organizational and programme structures at the

decentralized level, organizational relationships cannot be generalized and
prescribed to all programme units. Howev er, i t is recommended that the
M&E specialists report to seni or management on evaluation-related
matters in order to ensure effective coherence, coordination and
independence of the function.

management system to support management accountability for evaluation.

It provides timely data on the status of evaluations in the evaluation plans,
management responses and follow-up. The Evaluation Office reports on
evaluation practices and compliance, using the data in the ERC in its
Annual Report on Evaluation to the Executive Board. Regional bureaux and
other oversight units also use the ERC data. ERC is a public website.

62
140

FORMULATING PE RFORMANCE INDICATORS

Indicators are signposts of change along the path to development. They describe the
way to track intended results and are critical for monitoring and evaluation.

Good performance indicators are a critical part of the results framework. In particular,
indicators can help to:

Inform decision making for ongoing programme or project management

Measure progress and achievements, as understood by the different stakeholders

Clarify consistency between activities, outputs, outcomes and impacts

Ensure legitimacy and accountability to all stakeholders by demonstrating progress

Assess project and staff performance18

61

63
Indicators may be used at a ny point al ong the results chain o f a ctivities,
outputs, outcomes and i mpacts, but must always directly relate to the result
being measured. Some important points include the following:

Who sets indicators is fundamental, not only to owner ship and transparency
but also to the effectiveness of the indicators. Setting objectives and indicators
should be a participatory process.

A variety of indicator types is more likely to be effective. The demand for

objective verification may mean that focus is given to th e quantitative or
simplistic at th e expense of indicators that are harder to verify but may better
capture the essence of the change taking place.

The fewer the indica tors the be tter. Me asuring cha nge is c ostly so use as
few indicators as possible. However, there must be indicators in sufficient
number to measure the breadth of changes happening and to provide cross-
checking.

A frequent weakness seen in formulating indicators is the tendency to use

general and purely quantitative indicators that measure number or percentage
of something, for example, ―number of new policies passed.‖ These are often
weak indicators as they merely communicate that something has happened but
not whether what has happened is an important measure of the objective. For
example, take a situation where an audit report finds 10 weaknesses in a
business unit, 3 of which are considered serious and the other 7 routine. If the
7 routine issues were dealt with, an indicator that measures performance as
―number or percentage of recommendations acted on‖ may capture the fact
that some action has been taken but not convey a sense of whether these are
the important actions.

In general, indicators should direct focus to what is critical. For example,

there could be different ways of measuring whether an outcome relating to
greater commitment by govern-ment partners to HIV/AIDS concerns is being
realized.

Examine the following indicator: ―Number of government ministries that have

an HIV/AIDS sector strategy.‖

64
 Now compare it with another quantitative indicator such as: ―Number of
government ministries that have an HIV/AIDS sector strategy developed in
consultation with non-governmental stakeholders.‖

And further compare it with a possible qualitative indicator: ―Number of

government ministries that have a strong HIV/AIDS sector strategy.‖

Measured by:

Strategy was developed in consultation with non-government stakeholders (X

points)

Ministry‘s senior officials involved in strategy development and

implementation processes (X points)

Ministry has in place a budget to finance implementation of strategy (X

points)

In the first case, a strategy could have been designed with no stakeholder
involvement, no senior management engagement and no budget. Simply
counting the number of ministries that have done this would not be a measure
of real progress against the outcome that deals with the real commitment of
the government partners.

62

65
Box 14. SMART i ndicators

Specific: Is the indicator specific enough to measure progress towards the

results?

Measurable: Is the indicator a reliable and clear measure of results?

Attainable: Are the results in which the indicator seeks to chart progress
realistic?

Relevant: Is the indicator relevant to the intended outputs and outcomes?

Time-bound: Are data available at reasonable cost and effort?

The process of formulating indicators should begin with the following

questions:

How can we measure that the expected results are being achieved? What type
of information can demonstrate a positive change?

What can be feasibly monitored with given resource and capacity constraints?
Will timely information be available for the different monitoring exercises?
What will the system of data collection be and who will be responsible?

Can national systems be used or augmented? Can government indicators be

used?19

Quantitative and qualitative indicators

Indicators can either be quantitative or qualitative. Quantitative indicators are

statistical measures that measure results in terms of:

Number Percentage

Rate (example: birth rate—births per 1,000 population)

Ratio (example: sex ratio—number of males per number of females)

Qualitative indicators reflect people‘s judgements, opinions, perceptions and

attitudes towards a given situation or subject. They can include changes in
sensitivity, satisfaction, influence, awareness, understanding, attitudes, quality,
perception, dialogue or sense of well-being.

66
 Qualitative indicators measure results in terms of:

Compliance with… Quality of…Extent of… Level of …

Note that in the example used in Box 13 on the commitment of government

partners, subindicators are being used to assess the quality of the strategy,
―Did it benefit from

63

the inv olvement o f o ther s takeholders?‖; the extent of sen ior ma

nagement e ngage-ment; and the level of commitment, ―Is there also a
budget in place?‖

As far as possible, indicators should be disaggregated. Averages tend to

hide disparities, and recognizing disparities is essential for programming to
address the special needs of groups such as women, indigenous groups and
marginalized populations. Indicators can be disaggregated by sex, age,
geographic area and ethnicity, among other things.

The key to good indicators is credibility—not volume of data or precision

in measurement. Large volumes of data can confuse rather than bring focus
and a quantitative observation is no more inherently objective than a
qualitative observation. An indicator‘s suitability depends on how it relates
to the result it intends to describe.

Proxy indicators

In some instances, data wil not be available for the most suitable indicators
of a particular result. In these situations, stakeholders should use proxy
indicators. Proxy indicators are a less direct way of measuring progress
against a result.
67
For example, take the outcome: ―improved capacity of local government
authorities to deliver solid waste management services in an effective and
efficient manner.‖ Some possible direct indicators could include:

Hours o f d own tim e ( out-of-service time) of solid was te vehicle

fleet due to maintenance and other problems

Percentage change in number of households serviced weekly

Percentage change in number of commercial properties serviced weekly

Percentage of on-time pick-ups of solid waste matter in [specify] region

within last six-month period

Assuming no system is in p lace to track these indicators, a possible proxy

or indirect indicator could be:

68
 Means of verification

Results statements and indicators should be SMART. The ‗M‘ in SMART

stands for ‗measurable‘, which implies that data should be readily available in
order to ascertain the progress made in achieving results. In defining results
and corresponding indica-tors, it is th us important to consider how data will
be obtained through monitoring and evaluation processes.

Means of verification play a key role in grounding an initiative in the realities

of a particular setting. Plans that are too ambitious or developed too hastily
often fail to recognize the difficulties in obtaining evidence that will allow
programme managers to demonstrate the success of an initiative. Without
clearly defining the kind of evidence that will be required to ascertain the
achievement of results, without fully considering the implications of obtaining
such evidence in ter ms of effor t and cost, planners put the in tegrity of th e
pro gramme at risk. If results an d indic ators ar e n ot ba sed on measurable,
independently verifiable data, the extent to which an initiative is realistic or
achievable is questionable.

70

69
70