First International Conference on Construction In Developing Countries (ICCIDC–I)

“Advancing and Integrating Construction Education, Research & Practice”

August 4-5, 2008, Karachi,, Pakistan

A Framework to Assess Sustainability of Community-based Water Projects

Using Multi-Criteria Analysis
Kamalesh Panthi
Ph.D. Candidate, Florida International University, Miami, FL, USA
kpant003@fiu.edu

Shashi Bhattarai
C.E.O., Development Dynamic, Kathmandu, Nepal

Abstract
The application of Monitoring and Evaluation systems to assess water facility projects has provided
decision makers to plan for the sustanability of the future projects based on the performance of the
existing projects. There are many subjective and objective opinions while determing the effectiveness of
these projects. For the community–based projects, sustainability is a major cause of concern to all the
stakeholders. Inorder to monitor and evaluate the sustainability of these projects, different indicators are
identified for measuring their effectivenes. A framework for an integrated evaluation system is developed
in this study using analytical hierarchy process for multiple-criteria decision making. There is much
subjective information that needs to be quantified inorder to remove any bias in evaluator’s assesment of
qualitative measures. A framework developed in this study in this study is then applied to assess the
sustainability of the sixteen chosen water facility projects in Nepal. The results have shown that there is
significant value of such framework in providing information and input for different decision-making
levels.

Keywords
AHP, Sustainability, Evaluation, Monitoring, Multi-Criteria Analysis

1. Introduction

Poverty assessment research has conistently showed that improvements in water services are a core
element in most strategies deisgned to alleviate poverty. It was until recently that these water utility
projects were considered to be one time investment by the government and there was little participation
from the community. These projects suffered from poor maintenance and disuse. This has led to a demand
for more community involvement so that the long term sustainability of such projects are ensured.
Development that meets the needs of the present without compromising the ability of future generations
to meet their own needs is the general definition of sustainability (WCED, 1987). Based on objectives set
by the stakeholders of the project, there may be different views of looking at the sustainability aspect of
the project. In this study, sustainabilty of the project is viewed as an amalgam of Technical,
Social/Environmental, Financial, and Institutional criterias as shown in Figure 1. Projects are evaluated
based on these multiple criterias for their sustainabilty.

464
 Technical

Sustainable
Social / Projects
Environmental Financial

Institutional

Figure 1: Components of a Sustainable Project

Multi Criteria Analysis (MCA) is a process of integrated assessment of projects, alternatives or options
for ranking or selecting, priority setting among the finite set of projects, alternatives or options. MCA is a
structured approach to determine overall preference among alternatives, where the alternatives
accomplish several objectives. The advantage of the MCA processes is that it enables an integrated
assessment of subjective and objective information with stakeholders’ values in a single framework.
Different MCA or Multi-Criteria Decision Making (MCDM) methods have been widely used in the area
of environmental resources planning and management. Recico et al. (1999) developed a system for water
evaluation and monitoring that was applied to an aquifer in Spain. Raju et al. (2000) used MCDM
analysis for a case study of an irrigation area to rank different alternatives using economic, environmental
and social factors as criterias. Of all the MCDM tools, Analytical Hierarcy Process (AHP) is being used
widely because of the nature of the problem and the structure of the relevant criteria (Karamouz et al.,
2002).

The Analytical Hierarchy Process (AHP) based multiple criteria analysis deals with the relative priority of
importance of each factor by pairwise comparison of all factors with respect to a certain criteria. A
hierarchical structure of these factors is formed by grouping them into different levels. The application of
the hierarchical structure allows the factors to be broken down into details. AHP based multiple criteria
analysis starts from building tree like structure with criteria at higher level and factors and sub-factors are
at the lower level. Objective of evaluation lies at the top and the options or alternatives to be evaluated are
placed at the lowest level of the hierarchy. The AHP simplifies the process of identifcation and
assessment of criteria, factors and sub-factors related to a problem. The importance or weight of the
factors can either be directly applied or could be generated by making pair-wise judgment between the
various factors. Performance of project is applied at the lowest level of the hierarchy. Criteria are the
highest level of sustainability monitoring indicators, where as sub-factors are at the lower level.

465
2. Methodology

This sustainability monitoring study process is broadly divided into two parts. The first part is to design
the monitoring framework and the second part is to utilize the designed framework for the selected
projects’ sustainability monitoring.

2.1 Monitoring Framework

The monitoring framework task consists of study of indicators and scoring system. The framework is the
basis of sustainability monitoring and generating sustainability score by analytic hierarchy process (AHP)
method.

The sustainability monitoring framework consists of four criteria, thirteen factors and twenty six sub-
factors for water facility. The framework constitutes sets of sustainability monitoring indicators. The
information generated in isolation is to be integrated with the scoring system, making it comparatively
easy to judge whether the project under consideration is sustainable or not. Further, each score is
classified as one of the three situations; sustained, partially sustained and not sustaining. A high end MCA
tool based on AHP has been utilized to generate sustainability score of each individual or group of
projects. Figure 2 presents a hierarchial structure that is formed by grouping factors into different level.

On the top of the hierarcy is the goal. The goal of this framework is to assess the sustainability of the
projects. On the next lower level are the major criterias which are used to evaluate the sustainaility of the
projects. In Figure 2, A1 – A4 are criteria and A11 – A43 are factors; sub-factors are below the factor
level. Factors and sub-factors in Figure 2 are represented by a letter and numeral. Please refer to Table 1
for their complete meaning. Sub-factors are measured or judged or rated for each project, while
monitoring sustainability in the field. Criteria are sub-divided into factors and factors are further sub-
divided into sub-factors. For example, technical, socio-environmental, financial and institutional aspects
are the criteria looked in this study. The technical criterion is divided into four factors for water supply
project monitoring, namely source, physical condition of system, water point functioning and meeting
demand. The factor "Water Point (WP) Functioning" is further divided into three sub-factors, namely
maintaining design flow, water quality and surrounding condition. The sub-factors are the indicators
under each criterion or factors are measured/judged or rated at project site while conducting sustainability
monitoring. Indicator may be a single sub-factor or factor consisting of more than one sub-factor. Each of
these sub-factors at the lowest level of the hierarcy are rated for each individual project on a five point
scale of excellent (80-100%), very good (70-79%), good (50-69%), fair (30-49%) and poor (<30%).
Based on the weights of the factors and sub-factors obtained from pairwise comparison from AHP are
multiplied with the respective ratings for sub-factors, total score is obtained for the particular project that
indicates its sustainability. Projects are then ranked according to their sustainability score.Sustainability
sensitivity analysis conducted with the weights to factors and sub-factors using AHP based software, to
get more insights on the sustainability status of projects, to identify most sensitive factor good or bad
governing the sustainability.

466
 Table 1: Components of Sustainability Assessment Criteria, Factors and Sub-factors

Criteria Factors Sub factors

A11: Source Yield & Quality A111: Reliability, adequacy, depletion
A112: Water quality at source
A113: Accessibility, chance of contamination &
Conflict

A12: Physical Condition of A121: Design adequacy, site & technology

System A122: Condition & functionality of system
Technical

A123: Natural threat to physical system

A1:

AS13: Water point functioning A131: Maintaining Design Flow

A132: Water quality
A133: Surrounding condition/Drainage system

A14: Meeting Demand A141: Water fetching time

A142: Status of meeting additional demand

A21: Use of water facility A211: Status of use by targeted population

Environmental

A22: Community participation A221: Decision making and O&M

A2: Social/

A23: Environmental A231: Mitigation measure & Drainage

A24: Social Inclusion & A241: Inclusion (ethnic group)

Equity A242: Equity (men, women, reach & poor)

A31: Availability of fund A311: Establishment of O&M fund & saving

Financial

A312: Regularity & transparency

A3:

A32: Use of fund A321: Use of saving / surplus fund

A41: Users’ committee A411: Existence, functioning & meetings

A412: Ownership & activities
A413: Representation on UC
Institutional
A4:

A42: Maintenance committee / A421: Existence

Care taker A422: Functioning

A43: Coordination and A431: With local authority & other agencies
Linkage A432: Training & external support

467
 Goal Criteria Factors Sub-factors Sustainability
Score
A111(0.054)
A11(0.1) A112(0.023)

A113(0.023)

A121(0.008)
A12(0.1) A122(0.054)
A123(0.038)
A1:Technical
(0.5) A131(0.12)

A13(0.2) A132(0.04)
A133(0.04)

A141(0.07)
A14(0.1) Not
A142(0.03) Sustainable
(<30%)

A21(0.1) A211(0.1)

A221(0.05)
A22(0.05)
A2: Social
/Environmental A23(0.05)
A231(0.05)
(0.25) Partially
Goal A24(0.05) A241(0.031) Sustainable
(Project (30-70%)
A242(0.019)
Sustainability
Assessment)
A311(0.025)
A31(0.01)
A312(0.015)
A3: Financial
(0.05) A32(0.01)
A321(0.01)

Sustainable
A411(0.054)
(70-100%)
A41(0.5) A412(0.023)
A413(0.023)

A4:Institutional A42(0.25) A421(0.025)

(0.2) A422(0.025)

A431(0.025)
A43(0.25)
A432(0.025)

Figure 2: Hierarchical Structure of Criteria, Factors and Sub-factors and their Weights

468
2.2 Application of the framework

A multi criterion approach of sustainability assessment study is undertaken whose objective is to monitor
the sustainability status of a range of completed projects which have completed three years of operation.
This was carried out by reviewing the physical functionality and usage of hardware in projects, by
appraising the effectiveness of institutional, operational and maintenance systems. A total of 16 projects
were selected for sustainability monitoring, taking care to balance the project selection process by
topography, technology, extended facility type, rural, urban etc. The objective of the sampling of projects
is not only to generalize the sustainability outcome, but to capture all possible cases of projects in the
sustainability monitoring framework, ensuring the applicability of the sustainability framework to all the
existing project types.

Systematic Secondary Information Collection and Analysis (SSICA) approach was used to collect and
analyze all documented information on sustainability monitoring. SSICA was conducted also to
understand the different aspects of integrated sustainability, specially for sustainability monitoring
framework development, the first part of the study. The study team identified the tools to be used to
solicit information from the project site. In this context, various participatory tools were evaluated against
the information to be collected to arrive at the best combination of methodological tools to utilize during
the fieldwork. The participatory tools assessed include observation (infrastructure and document); focus
group discussions; key informant interviews; and photo capture among others. Summary of activities
involved in second part of the study is presented in Figure 3.

Arrival at Project Service Area

Planning & Arrangement for Assessment

Transact Village Walk

With senior community / UC
members

Key points Observation, Photo

Capture and Sample Survey

On the spot interviews with

HH Owners / Students / Locals
passing by

Discussion & Meetings with

Beneficiaries & UC Members

Completion of Field Judgment and

Assessment

Figure 3: Systematic Field Assessment

The fieldwork consisted of random water point inspection (sample surveys), focus group discussions, key
informant interviews; transact walks, observations and photo capture. The field team provided input to
sustainability assessment sheets from the field interviews, focus group discussions and key informant

469
interview. Village walks were guided by the monitoring indicators on the sample survey matrix (SSM)
and field assessment sheets. The field team appraised the project beneficiaries and their impression of the
project and benefit rendered were collected.

Based on the f information collected for all the projects, each of the sub-factors are given a rating. For
example, if the reliability, adequacy, depletion of the source yield of the water facility is found to be
excellent, the score for this sub-factor would probably range from 80-100%. If the score given was, say
90%, then this score is multiplied with the relative weight of the “Reliability, adequacy, depletion” sub-
factor which in this case is 0.054. The score for this sub-factor for this project is therefore, product of
90% and 0.054, which is 4.86%. Similarly, score for all the other sub-factors are determined in a similar
manner and finally all the individual scores are added to get the total score for that particular project. If
the total score lies within 70-100%, then the project is considered sustainable. This methodology is
followed for other projects and finally the ranking of the projects based on the total sustainability score is
obtained as shown in Table 3.

3. Results

Analysis of the information was conducted for each project to generate the sustainability scores. The
range of score utilized to demarcate the sustainability status is presented in Table 2.

Table 2: Cut-off Score for Sustainability Status Determination

Typical five point Score Distribution for different grading Sustainability

grading nomenclature Five point Four point Three point status
Excellent 80-100% 70-100% 70-100% Sustained
Very Good 70-79%
Good 50-69% 50-69% 30-69% Partially
Fair 30-49% 30-49% sustained
Poor <30% <30% <30% Not sustained

Based on the scores obtained by utilizing the above methodology, projects are categorized as to whether
they are sustainable, partially sustainable or not-sustainable as shown in Table 3.

Table 3: Sustainability Score for Projects

Project Region Age Composite Sustainability Status

Name Hill/Tarai/Urban (Years) (Score)
District
Suntalachour WDR – Hill 6 Sustainable (82.0)
Baglung
Chapdanda WDR – Hill 6 Partially Sustainable (66.6)
Syangja
Belbhanjyang CDR – Hill 6 Sustainable (87.0)
Dhading
Lamachour CDR – Hill 7 Sustainable (84.3)
Nuwakot
Neupanegaun CDR – Hill 7 Sustainable (82.2)
Dhading
Aptari EDR – Hill 6 Sustainable (87.0)
Bhojpur

470
 Jagretar EDR – Hill 7 Sustainable (71.8)
Udayapur
Dhauri EDR – Hill 6 Partially Sustainable (69.6)
Sakaura Udayapur
Kawachit WDR – Hill 7 Sustainable (76.1)
Tanahu
Lyaku Urban 5 -
Bhakthapur
Kapan Urban 6 Sustainable (71.9)
Kathmandu
Tadhoka Urban 4 Sustainable (83.5)
Lalitpur
Ghoghanpur EDR – Tarai 6 Partially Sustainable (67.8)
Saptari
Taragana EDR – Tarai 5 Partially Sustainable (68.4)
Siraha
Jandol EDR – Tarai 8 Partially Sustainable (57.2)
Saptari
Jitpur CDR – Tarai 3 Sustainable (77.0)
Chitwan

4. Conclusion

The study developed a framework for the sustainability monitoring and evaluation of projects. The
framework results sustainability status of projects based on their performance with various indicators
included in the framework. Since the sustainability status of project is dependent on the indicators used
and weight and score distribution to the various indicators, the first and the foremost thing is that there
should be consensus on indicators and weight distribution in the framework among all the concerned
agencies that are using the framework in future.

From the result of the study, it can be seen that most of the selected projects are in the sustainable range
and the remaining are partially sustainable. For the partially sustainable projects, effort should be put to
those causes (sub-factors) that have lower score but have higher weights (importance). The utility of the
framework can be improved by carrying out the sensitivity analysis to see the effects of changes in
weights of different sub-factors to the overall sustainability score. This is recommended for the future
studies.

5. References

Karamouz, M., Kerachian, R., Zahraie, B., and Nejhad, S.A.(2002). “Monitoring and Evaluation Scheme
using the Multiple-Criteria-Decision-Making Technique: Application to Irrigation Projects”. Journal
of Irrigation and Drainage Engineering, Vol. 128, No. 6, pp 341-350.
Raju, K. S., Dukstein, L., and Arondel, C. (2000). ‘‘Multi-criterion analysis for sustainable water
resources planning: A case study in Spain.’’ Water Resources Management, Vol. 14, No. 6, pp 435–
456.
Recio, B., Rubio, F., Lomban, J., and Ibaiiez, J. (1999). ‘‘An econometric irrigated crop allocation model
for analyzing the impact of water restriction policies.’’ Journal of Agriculture Water Management,
Vol. 42, No.16, pp 47–63.

471
WAN (2007). “Multi-Criterion Approach to Sustainability Monitoring.” Summary Report- Long Term
Sustainability Monitoring Study. Water Aid Nepal, Kathmandu.
World Commission on Environment and Development, Earth Summit, Rio De Janeiro, Brazil, 1987.

472