You are on page 1of 13

Validation Report

Reference Number: PCV: PAK 2010-69 Project Number: 32058 Loan Number: 1891 December 2010

Pakistan: Road Sector Development Program

Independent Evaluation Department

performance evaluation. IED C. asian development bank. the Independent Evaluation Department does not intend to make any judgments as to the legal or other status of any territory or area. independent evaluation department. Senior Evaluation Officer. Independent Evaluation Department (IED) R. IED Team leader Team members In preparing any evaluation report. Independent Evaluation Division 2. Independent Evaluation Division 2. Key Words pakistan. Lumain. or by making any designation of or reference to a particular territory or geographic area in this document. Hettige. sector development program Director H. Roldan. “$” refers to US dollars. roads. Bolt. Assistant Operations Evaluation Analyst. highways. Office of the Director General.ABBREVIATIONS ADB ADF IED MOC NHA OCR PCR RRP RSDP TA – – – – – – – – – – Asian Development Bank Asian Development Fund Independent Evaluation Department Ministry of Communications National Highway Authority ordinary capital resources program completion report report and recommendation of the President Road Sector Development Program technical assistance NOTE In this report. lessons. Independent Evaluation Division 2. Southeast Asia Department R. adb. Advisor. .

00 0. IED2 December 2010 Approved Actual 83. (d) technical (ii) (iii) 1 2 Formerly with the Independent Evaluation Department at the time this validation was carried out. Consultant Validator: Director: R. Norojono ADB headquarters T. (d) strengthened road network institutions in professional and management skills. Basic Program Data PCR Validation Date: Program Number: 32058 1891 (program loan component only) Loan Number: Road Sector Development Program Name: Total Program Cost Program ($ million): Islamic Republic of Pakistan Loan/Grant ($ million) Country: Transport (Roads Transport) Total Cofinancing ($ million): Sector: and Information and Communications Technology ADF: 0. ADF = Asian Development Fund. Impact.2 50. and 1893PAK(SF). In this context. the Asian Development Bank was to finance a portion of the adjustment costs associated with the national road sector policy reforms.00 0.00 2 May 2002 27 Jun 2002 30 Jun 2007 30 Jun 2007 From To 2002 2003 2003 2004 2004 2005 2005 2007 2007 2008 H.00 0. PCR = program completion report. OCR = ordinary capital resources. The Road Sector Development Program (RSDP) was to improve management of the provincial and rural access roads network.76 0. Reforms initiated at the national level provided the framework for overall sector development involving (a) increased capital investments to develop an efficient transport network. Objectives or expected outcomes. ADB. approved on 19 December).00 0. Policy reforms were critically needed to increase the efficiency of Pakistan’s road sector at the national and provincial levels. Report and Recommendation of the President to the Board of Directors: Proposed Loans to the Islamic Republic of Pakistan for the Road Sector Development Program. Jones III. Masood ADB headquarters T. Southeast Peer Reviewers: Asia Department1 R.00 − 50. IED2 ADB = Asian Development Bank.0 Borrower ($ million): ADB Financing ($ million): OCR: 50. The expected outcomes were (a) preservation of road assets. for $200 million. Lumain. Policy and institutional changes in the National Highway Authority (NHA) would result in the effective delivery of services in the road sector. This was then expected to help reduce rural poverty and contribute to national economic growth. Program Description (summarized from the report and recommendation of the President)2 (i) Rationale. and assistance in establishing an effective policy and regulatory enabling environment. Hettige. Manila (Loans 1891-PAK.00 0.0 Beneficiaries ($ million): None Others ($ million): Cofinanciers: 19 December 2001 Approval Date: Effectiveness Date: 1 February 2002 Signing Date: Closing Date: Project Officers: Name: Location: H. Advisor. Bolt. F. (b) an efficient internal network to support regional cooperation. B. (c) operational efficiency improvements. Carlsson ADB headquarters H. (e) initiatives to improve traffic management and road safety.1892-PAK. 2001.00 0.PROGRAM COMPLETION REPORT VALIDATION A. Nakazaki ADB headquarters S. Muramoto ADB headquarters O. (b) governance and institutional development.00 0. (c) increased resources and improved institutional mechanisms to address inadequate road rehabilitation and maintenance. Office of the Quality Control/ Director General. and (f) support for measures to control excessive axle loads and vehicle weights. Senior Evaluation Officer. .

(b) developing the NHA Code. which were consistent with the policy themes of the national policy reform program. (f) preserving assets through an axle-load control program involving installation of weigh stations and enhancing enforcement procedures. and (h) adopting a comprehensive approach to road safety. The accomplishments of the RSDP policy loan relative to the five themes above are summarized in Table 1 (see also PCR. and (e) reduced incidence and severity of road accidents. and (g) poverty reduction and community development. These themes. (e) road safety. and a provincial sector development project.2 efficiency improvements. This ADB-financed policy loan was mainly intended to support continuing efforts of the NHA to adapt business processes to those of contemporary road agencies. and standard operating procedures. 3 4 . (ii) Program outputs. (g) ensuring transparency in procurement processes. (c) reduced burden on public sector resources by alternative financing mechanisms. The policy reform component of the RSDP (and its timing) is very relevant. with only two of ADB. Conception of the RSDP was generally appropriate. 2008. C.3 The PCR adequately assesses the relevance of program design and formulation. were highly relevant and remain so. Implementation of the road safety theme was an issue. In particular. 10–21). This validation covers the program completion report (PCR) on the national policy component of the RSDP. (iv) Components and/or outputs. road sector resource management (8). (c) enhancing staff accountability in financial management.e. three were complied with while a fourth is judged to be generally complied with. while the second and third (footnote 2) were for the improvement of provincial roads in Pakistan’s Sindh Province. which covered the implementation of the national road sector reform program. It is aimed at building the capacity of the NHA and transforming it into an efficient service provider to road users (i. Specific program targets include (a) improved network condition through better institutional management and maintenance practices. the PCR frankly comments that the policy loan should have been implemented with accompanying technical assistance (TA). For the institutional efficiency improvement theme. to ensure safe and comfortable travel on the national highways). Its stated aim and purpose are consistent with the government’s development strategy and the country assistance program of ADB for Pakistan from 2001 to 2003. The Asian Development Bank (ADB) assistance comprised three loans using a sector development program modality.3% of total) were either complied or generally complied with. (d) prioritizing investment based on economic principles. and (e) road safety. the RSDP was to support the continuation of the reform process with the following elements: (a) rationalizing staffing. On the former. One at $50 million equivalent (Loan 1891-PAK) was for development of the policy reform program. (b) road sector resource management. including public awareness and stakeholder consultations. Evaluation of Design and Implementation (program completion report assessment and validation) (i) Relevance of design and formulation. financial manual. Manila. The closing date for Loans 1892 and 1893 was extended from 30 June 2007 to 30 June 2009. paras. there was room for improvement. and road asset preservation (5) themes were carried out. particularly for the transport policy and road safety themes (section C[iv]). (d) improved access and lower transport costs. The final status of associated policy actions (for tranche releases) relative to the national policy reform program policy matrix are described in Appendix 4 of the PCR. Of the twenty-four policy actions for tranche release. (e) self-financing maintenance needs through toll receipts. (f) private sector participation including involvement of knowledge-based industries. All policy actions for transport policy (1). (c) road asset preservation. 4 Its program interventions were built around a comprehensive set of five policy themes involving (a) transport policy.. An ongoing national policy reform program largely reflected the commitment of the NHA to change and adapt. While the design and formulation of the RSDP was suitable. Completion Report: Road Sector Development Program in the Islamic Republic of Pakistan. The RSDP comprised two components to address structural inefficiencies in the national and provincial road network: national road sector policy reform. (d) institutional efficiency improvement. twenty (83. (b) improved local government capacity to become involved in the management of road networks within their jurisdiction.

as much of this would have been in kind. As this was a program loan. (iii) Program cost. which operates as a complete business entity that executes two areas of operations: (a) development of the national highway network. which generates revenues for this purpose but this is still inadequate. NHA = National Highway Authority. Generally achieved. and (d) design and feasibility studies. Road Asset Preservation Institutional Efficiency Improvement Road Safety Partly achieved.079. To some extent. Of 112 planned vehicle weighbridges.300 because of depreciation of the US dollar versus the yen. A road safety cell was established and road design safety audits were introduced as required by external funding agencies. Since the loan was to be used to pay for approved imports. The National Highway Safety Council was established in 2001 as part of the road safety theme. The total appraised program cost was estimated at $83. the loan was closed as originally scheduled. The NHA initiated rightsizing and reduced staff in 2003 and later increased it in accordance with increasing work demands. while the institutional reform and corporate strengthening program are under way. and (b) maintenance and upgrading of the developed network.3 million or $50 million equivalent) increased to $50. policy actions were carried out and objectives achieved. Although disbursement of the first and second tranches was delayed. Manila. The National Highway and Motorway Police was created under this ordinance. as did a lack of commitment at higher government levels and a lack of coordination among institutions involved. The NHA outsources functions such as (a) maintenance. As the ADB loan portion was fixed at loan approval. Further actions regarding axle weight control are being considered for implementation in the future. Some progress has been achieved with the enactment of a national highway safety ordinance.172 was debited to ADB as the front-end fee. Source: Independent Evaluation Department staff based on Asian Development Bank. 2008. an ADB communication with the government on the status of the national reform program matrix (dated August 2002) do indicate these were complied with at the time. (b) construction. but it has become a dormant organization without a permanent secretariat because funds were not available.3 the six tranche release conditions either fully complied with or generally complied with. Generally achieved.763. Inadequate funding retarded some policy actions from being implemented. while the total loan value (denominated in Y6. (c) toll collection. 42 have been constructed and are operational with noticeable effect on the motorways. Transport policy reform involved the development of a long-term national transport policy framework that will combine strategic vision and ensure sustainability. The toll rate is under review with a view to help increase revenues to the needed level. On the other hand. The policy’s objectives are being achieved and actions have been implemented. Efforts by the National Highway and Motorway Police were partly responsible for the reduced number of road accidents. none of the loan went directly to financing the reform program. most of the national highways have been brought under a tolling regime. while the balance was not complied with because of lack of resources. The loan was approved on 19 December 2001 and closed on 30 June 2007.5 Two were partly complied with. The first 5 Three policy actions (on the road safety theme) required for the release of the first tranche were reportedly not fully met at program completion. Table 1: Assessing Program Outputs Policy Theme Transport Policy Road Sector Resource Management Assessment Achieved. disbursements. borrower contribution. PCR = program completion report. and conformance to schedule (as relevant to project performance). The PCR does not report on the actual costs incurred by the government. Completion Report: Road Sector Development Program in the Islamic Republic of Pakistan. the release criteria seem to have been subject to meeting release schedules rather than tranche release requirements. . Table 1). Achieved. The NHA has prepared a long-term financial business plan for 2007–2020 for approval by the government. A total of $486. According to the PCR. the total amount was disbursed with two adjustments. These were focused on the NHA.2 million (PCR. To help ensure that national highway maintenance is sustainable. The PCR relates that policy actions have been carried out and objectives achieved. it was disbursed in two tranches based on fulfillment of tranche conditions set out in the national policy reform program policy matrix (section C[ii]).

transport policy TA was later sponsored upon government request. coordination among agencies was often poor. para. The NHA remains committed to continuing improvements in institutional efficiency and reforms in the management of the national road network (PCR. related technical assistance. The first part in the amount of $15. First. This was partly caused by lack of funds. conditions and covenants. for $290. only one of the 18 loan covenants was partly complied with. The PCR indicated that the original schedule of first tranche release was February 2002 and the second tranche release on June 2002. In general. A review of the covenants reveals that only 1 of the 18 program-specific covenants was partly complied with. and some actions were not initiated during the program period. the day after the loan was made effective. Several factors also detract from borrower performance. To a certain extent. while the balance was judged generally complied with (section C[iv]). As identified at appraisal. thus completing full disbursement under the program. (iv) Implementation arrangements.602 equivalent was disbursed on 28 June 2002. Conditions for tranche release are given in Appendix 5 of the PCR. ADB. particularly for implementing the road safety actions. For example.4 tranche in the amount of $34. The partly complied with covenant involved the provision of funds in a timely manner for carrying out the program.6 The second and final tranche was disbursed in two parts. Funds were not always readily available. implementation of some of the road safety components had not had the expected impact and sustainability. with the support of the Ministry of Communications (MOC) and the NHA. There was no TA attached to the loan and no consultants were required. the program loan was implemented as conceived. 59). For example. The performance of the borrower and the MOF was satisfactory in terms of using the loan amount for the purchase of supplies and materials. . the Ministry of Finance (MOF). the actual compliance was less than the intended spirit of the covenant. Manila (TA 4400-PAK. (v) Performance of the borrower and executing agency.000. 63). target dates were often missed.G of the PCR states that the delays in disbursing the two tranches were due to the government’s delay in completing requirements for loan effectiveness (footnote 6) and completing the second tranche policy actions. Compliance with the requirements for loan effectiveness. Technical Assistance to the Islamic Republic of Pakistan for Transport Policy Support. The three agencies involved (MOF. In the conditions and covenants section. particularly for the transport policy and road safety themes (section C[i]). this observation implies that some of the agencies responsible for implementing the program were not fully committed to instituting reforms over the 5-year implementation period. the program was implemented through the executing agency. and NHA) performed satisfactorily in establishing and implementing policy reforms. section II. ADB’s program performance ratings for development objectives and implementation progress from June 2002 to June 2007 were satisfactory. approved on 30 September). and this contributed to the national transport policy document (PCR. From this validation’s point of view.065. The rest of the covenants were either complied with (8 covenants) or generally complied with (9 covenants). while the second part in the amount of $1. Loan release was delayed by 4 months compared with the original program period. As explained above. the PCR rightfully points out that the borrower did not fully analyze the resources 6 7 The report and recommendation of the President stated that the period of the policy loan was from January 2002 to June 2007. the assessment of generally complied with seems to mean that while attempts were made to be in compliance.210. as a condition for the first tranche release. Eight of the loan covenants were complied with. was delayed. In hindsight. and procurement and consultant performance. In general.000 equivalent was disbursed on 14 November 2003.7 Procurement of the various goods and materials purchased by the government followed normal government procurement procedures. actions met at one point in time were not continued. There was a 17-month delay in disbursing the second tranche because of delayed government action on release conditions. para. MOC.000.986 equivalent was disbursed on 28 November 2003. suitable TA should have been included. 2004.

review mission (November 2002). This leaves questions as to whether ADB provided adequate assistance during program implementation. the program remains important and very relevant to the country’s road sector. the government must ensure that the policy reforms are continued for sustainable operations. particularly for the transport policy and road safety components (section C[iv]). However. The PCR assessed ADB performance satisfactory. it states “The Program was administered by different officers in ADB and monitoring could have been better. any possible positive effect might be lost as if nothing These comprised a consultation mission (February 2002). taking into consideration the program’s timely completion and successes in a challenging reform process. 34). Their continuity is now dependent on establishing a stable source of funds to finance the components. The program loan mission leader had no part in program implementation. especially in terms of field supervision. (vi) Performance of the Asian Development Bank. All components were appropriate. 67 follows these up with “The program administration and review by ADB should emphasize close monitoring of the executing agency and ensure continuity of staff in the administration team. while the transport policy theme reportedly benefited from a subsequent stand-alone transport policy TA project. In this sense. this validation rates the program relevant. para. D. Although the tranche conditions were satisfied. some basic problems in the sector related to poor interministerial coordination and lack of needed legislation prevailed during implementation. The NHA is said to be committed to continuing improvements in institutional efficiency and reforms in the management of the national road network. continued implementation of some of the reforms have already suffered from lack of funds (section D[iv]). Second. While ADB headquarters support appears to have been sufficient to ensure that project management issues were addressed. it is unclear from the PCR whether field-level dialogue was adequate. The PCR rightfully points out that this policy program should have been implemented with TA. From a short-term perspective. In particular. supervision missions fielded were seemingly not sufficient. In para. These were started but could not be continued because of lack of funds. The PCR assessment of ADB performance is contradictory. as the aims and purposes were consistent with the government’s development strategy and ADB’s country assistance plan (PCR.” Para. para. This validation agrees that the program was satisfactorily conceived and is reasonably comprehensive (section C[i]). This validation considers ADB headquarters-based administrative support to have been adequate before and after tranche release. and a special program administration mission (November 2003). and program inputs were sufficient to generate the planned outputs.5 required for implementation of some components (PCR. The PCR rated the program very relevant. Evaluation of Performance (program completion report assessment and validation) (i) Relevance. Taking into consideration the overall program quality-at-entry at both approval and completion. which helped to facilitate actions taken by the government agencies in complying with tranche conditions required for the release of loan tranches” (PCR. Overall. In addition. During the 5-year implementation period. The PCR states that the lack of resources brings the continued implementation of all aspects of the policy reform into doubt. 58). This observed design weakness detracts from a full relevance rating at completion. this validation agrees with the PCR rating on ADB performance of satisfactory. 33).” The PCR goes on to emphasize that “ADB provided assistance in a timely manner. this validation agrees with the PCR’s overall performance rating of satisfactory. Effectiveness in achieving outcome. The November 2002 mission was a multiproject mission covering three other loans and a TA project. since they are largely on paper or the responsibility of newly established agencies. Therefore. 8 This validation considers this level of supervision inadequate for a program. in the longer term. (ii) 8 . 33 of the PCR. As cited above. ADB’s review could have been more periodic to avoid partial compliance actions by the EA. However. implementation of the road safety components did not meet expectations. ADB undertook only three review missions led by two different staff members. to be effective. while the initial efforts to set up new policies might have been largely successful. para. program-related achievements need to be sustainable.

so that an appropriate balance between revenue sources and appropriate allocation for the roads sector is maintained. An axle load control program was introduced to protect and increase the service life of road assets. which have not had the expected impact and sustainability. as well as professional-level accident recognition and implementation of remedial measures such as accident blackspotc identification and rectification. can now supplement government revenues through off-budget initiatives such as tolls on the national highway system. Implementation of the policy will be coordinated by the Planning Commission. Table 2: Assessing Effectiveness in Achieving Program Outcomes Policy Theme Transport Policy Rating Effective Assessment ADB intervention was successful and effective in developing a draft transport policy document for submission to the cabinet after interministerial discussion. More recently. Source: Independent Evaluation Department staff based on ADB. Less Effective .6 ever happened without proper follow-up. approved on 17 December). The NHA has moved toward becoming a commercial agency with a view to economy and efficiency in operations by operating on a fee-for-service basis. Manila. The NHA. c In terms of road safety management. Thus. Associated penalties for violation of load levels were set and are being levied. and personnel training. its approval is lingering for a long time due to evaluation required by many ministries and departments. approved on 5 February 2006). the result for each of the major themes is summarized in Table 2. ADB = Asian Development Bank. for $10 million. the NHA changed its organizational structure and is involved in institutional capacity building under b follow-on ADB-financed programs. budget allocations need to be rationalized through revenue sources. several attempts were made by the government to articulate a comprehensive transport policy that encompasses all subsectors. this refers to a place where road accidents have historically been concentrated. Road Asset Management Effective Road Asset Preservation Effective Institutional Efficiency Improvement Effective Road Safety Concerning the road safety theme. The final document was submitted to the executing agency for further processing and approval by the government. The executing agency is still coordinating with the Planning Commission and the Cabinet for their final review and approval. and lack of needed legislation. 2008. An enforcement procedure and institutional framework were established. for $180 million. The PCR states several positive efforts in this section. approved on 13 December and Loan 2231-PAK [Tranche 1]. there is no such legislation on the provincial highways.a The road asset management system has been developed and helps ensure that the road sector agencies carry out asset management and preservation effectively. Report and Recommendation of the President to the Board of Directors: Proposed Multitranche Financing Facility and Loan to the Islamic Republic of Pakistan: National Highway Development Sector Investment Program. Community awareness and stakeholder participation is being pursued. b ADB. Completion Report: Road Sector Development Program in the Islamic Republic of Pakistan. At completion. While effectiveness is relatively positive up to program completion. and ADB. a A comprehensive National Transport Policy for Pakistan was completed on 30 May 2008. However. the proof will come after the cabinet and further approvals and adequate funding. capability. PCR = program completion report. This was due to lack of funds. Manila (Loan 2210-PAK[SF]. but in section C(v) shortcomings are identified such as the implementation of some of the road safety policy components. for $545 million. The policy reform is being monitored through the development of awareness programs and measuring axle load levels at regular intervals on the network. Manila (Loan 2400-PAK. The draft policy document was presented to the interministerial National Trade Corridor (NTC) Task Force. The National Highway and Motorway Police (NH&MP) were responsible for identifying the operational parameters for enforcement. The NTC core group will take up the policy development process further and to formalize it as the government document. various measures were introduced to help reduce road accidents in Pakistan. poor interministerial coordination. as one of the major road agencies. 2007. However. A departmental review was conducted until early 2009. The validation agrees with the PCR rating of effective inasmuch as the desired effect is still in progress. The NHA and NH&MP are improving their knowledge. Previously. approved on 17 December and Loan 2401-PAK[SF]. Report and Recommendation of the President to the Board of Directors: Proposed Multitranche Financing Facility and Technical Assistance Grant to the Islamic Republic of Pakistan: National Trade Corridor Highway investment Program. for $3 million. the NHA is more capable of undertaking both highway upgrading and maintenance. While legislation was enacted to restrict maximum axle loads on the national highway network. 2005. NHA = National Highway Authority. Achieved outcomes here will probably be sustained considering follow-on assistance.

According to the PCR. there has been little effect. Because of the lack of resources and TA. PCR = program completion report. NHA standard operating procedures have been revised and updated. . enforcing road safety and axle load control on national highways is being extended regularly in phases. paras. Despite these delays. The NHA has acquired financial management expertise. The PCR rated the program efficient. The PCR does not suggest exactly how far these phases have progressed. the first to qualify for tranche release. approved on 18 November and Loan 2104-PAK[SF]. the NHA has undertaken rightsizing exercises but with additions and subtractions. Manila (Loan 2103-PAK. the impacts section (PCR. 2004.a MOC = Ministry of Communications.2 million. Weigh stations have been installed at strategic locations throughout the national highway system and on motorways and were integrated with the fine collection system. but has apparently failed in meeting its obligations (section C[v]). for $5 million. Completion Report: Road Sector Development Program in the Islamic Republic of Pakistan. Based on the PCR findings at completion. Manila. The National Highway Council has approved a medium-term road resource management plan. and subsequent revisions have been made to reflect the views of the various agencies involved. In terms of implementation efficiency. there seems to be little case for an expectation of high sustainability. NHA = National Highway Authority. Road Asset Management Highly Efficient Road Asset Preservation Efficient Institutional Efficiency Improvement Less Efficient The lack of achievements under the road safety theme is discussed in section C of the PCR. and intended benefits. The NHA and other concerned agencies will likely sustain actions where they see a clear benefit and will generally eschew actions deemed less beneficial or politically unacceptable. the loan was closed as originally scheduled. While the PCR does not specifically rate sustainability in the sustainability section (PCR.7 (iii) Efficiency in achieving outcome and outputs. Road Safety Inefficient (iv) Preliminary assessment of sustainability.9 The first tranche was delayed by only 4 months while the second tranche disbursements were late by 17 months because of delayed government action on release conditions. Lack of funds is cited as the main reason for inaction. a Asian Development Bank (ADB). which should improve the efficiency of in-house financial management capability. 50): “The government is trying its best to sustain the actions initiated under this Program. In broad terms. 49–50). This initiative was reportedly being pursued under the Northwest Frontier Province Road Development Sector Project. para. Actions have been undertaken to improve road asset preservation. Table 3 briefly summarizes PCR assessments on program efficiency under the reform program. but final approval has not been achieved. To achieve institutional efficiency improvement. Report and Recommendation of the President to the Board of Directors: Proposed Loan to the Islamic Republic of Pakistan for the Northwest Frontier Province Road Development Sector and Subregional Connectivity Project. which has improved steadily since 2001. Aside from external financial assistance. 9 Program loans do not fit the typical measure of efficiency as indicated by the economic internal rate of return of an infrastructure project. MOC has finalized the national transport policy. 2008. sustainability. 54) states that the policy program is expected to result in a “sustainable” impact over the long term (considering the effectiveness and efficiency of the outputs and outcomes). for $296. some actions were not implemented and some objectives could not be achieved. this validation agrees with that assessment. This initiative seems to be working efficiently as evidenced by the condition of the national highway and motorway network. Table 3: Assessing Efficiency in Achieving Program Outcomes Policy Theme Transport Policy Rating Efficient Assessment The transport policy has gone through several drafts. but was dropped by the government citing difficult conditions.” The government committed itself to supporting this program before loan approval. the policy loan cost was in line with approved loan amount while tranche releases were delayed. Source: Independent Evaluation Department staff based on Asian Development Bank. This would improve the Program’s performance. approved on 28 November). Additional external assistance would be necessary for complete implementation of all policy actions and establishment of alternative funding source for their continuity. the PCR explains sustainability this way (para.

6 for scoring successful projects/programs. Accordingly.8 falls within the range of 2. However. para. the validation rates the program at the low end of successful. the overall progress made under difficult conditions. On the negative side. The PCR concludes that the policy program’s socioeconomic impact can be perceived as moderate. and less likely to be sustainable. this validation sees mixed results in the long-term sustainability of the program actions. (b) Support for the implementation of the policy program could have been achieved with TA. . This validation agrees with this rating. the PCR states that the government should ensure that these initiatives are continued in the long term to ensure sustainable operations. ADB’s support in the endorsement process and implementation would go further in achieving the goals set out in this intervention. which is not fully operational.10 For the program to be fully effective. the PCR rated the policy loan satisfactory. and these are presented and/or summarized below. The PCR continually refers to policy objectives as having been generally achieved. Understanding the existing policies or lack thereof takes considerable time and interaction between ADB staff and concerned government officials. 11 The program is assessed to be relevant. Lessons.e. Overall Assessment. up from 38% (PCR. continued donor engagement in pursuing program actions will be critical for proper follow-up. Considering their importance. Likewise. The PCR states many positive impacts such as an increased maintenance allocation now covering about 50% of the development budget. In the likelihood that program sustainability is to become supply-driven or donor-dependent. The PCR also made a number of sound recommendations. stakeholder consultations are being carried out. this validation rates the sustainability less likely to be sustainable. 51 and Appendix 4). Recommendations. except for road safety. The installation of weigh stations at strategic locations on national highways has been partly accomplished. and Recommendations (validation of program completion report assessment) (i) Overall assessment.. and the level of donor interest. The implementation structure should reflect the organizational arrangement existing within the government. the reasons for this failure must be understood and ways to ensure coordination developed. the main issue outstanding is full enforcement of legal loading. it would require additional assistance in the development of implementation strategy and development of subsector specific guidelines and regulations. ADB is discussing ways to support various components. There are several important lessons cited in the PCR. effective. collaboration. Its equivalent overall weighted average of 1. i. the capacity of the executing agency to coordinate with all the involved ministries is crucial. they are highlighted below. Since this is an important requirement for success. Regarding road infrastructure. the government did not create the necessary organizations for the efficient treatment of road accident data. One of the basic problems in the sector is the lack of coordination. particularly for transport policy and road safety. the objectives of the policies were achieved with most agreed policy actions being carried out. The government and ADB recognized for some time that a comprehensive program of policy development and reform was necessary within the transport sector.7<S≥1. Maintenance planning is better executed and a maintenance manual developed.11 Lessons. and cooperation among the government agencies involved. This sector development program was well conceived and at the beginning was readily agreed by the government. At program completion. as stated in the PCR.8 Accordingly. the national transport policy formulated by MOC with assistance from ADB is being submitted for cabinet approval at some future unspecified date. (ii) (iii) 10 For example. E. However. (v) Impact (both intended and unintended). The conditional “generally” is not defined in the PCR but may imply something less than full achievement (section C[iv]). at program completion. Based on program information at completion. particularly within the highways and roads subsectors. efficient. (a) In the implementation of an interministerial policy. such a program is only as effective as the government wishes it to be. If the national transport policy document is endorsed by the government. depending on the interest of the donors in pursuing specific policy actions.

Better coordination was required at the highest levels of government. Corruption was not reported to be an issue. Monitoring and Evaluation Design. To maintain an active dialogue. a review mission in 2002. There should be a set of mechanisms associated with covenants to guide and ensure their compliance. fiduciary aspects. the loan was financially closed as originally scheduled.9 (a) (b) (c) (d) (e) Three first tranche conditions require further actions to ensure full compliance. H. initial efforts to set up the new policies appear successful (although largely on paper). To sustain this. there seems to be little case for optimistic expectations on sustainability (section D[iv]). there should be continuity of ADB staff involvement both in review frequency and staff. this input on the part of ADB was inadequate. The policy loan is rated at the low end of successful. implementation of the road safety program on the entire road network. actions will be where government see a clear benefit and will generally eschew actions deemed less beneficial or politically unacceptable). 60). para.g. While tranche disbursements were delayed. as applicable) (program completion report assessment and validation) Specific safeguard measures were not needed as part of the program. Implementation. G. and Utilization (program completion report assessment and validation) Monitoring was left to a consultation mission in 2002. including governance and anticorruption. For such a complex and interactive initiative to be undertaken comprehensively. grants. F. The government should accelerate efforts to establish the accident reporting center. it is essential that the executing agency keep involved staff engaged through the duration of the program.. Ratings Relevance: Effectiveness in Achieving Outcome: Efficiency in Achieving Outcome and Outputs: Preliminary Assessment of Sustainability: PCR Very Relevant Effective Efficient Not Rated IED Review Relevant Effective Efficient Less Likely Reason for Disagreement/Comments A weakness in program design relates to use of attached TA projects to support the transport policy and road safety themes (sections C[iv] and D[i]). Given funding constraints. through relevant government organizations. PCR rating at completion confirmed based on program achievements amidst a challenging reform process. sustainability of policy actions will be mixed (i. Other (e. At program completion.. In the absence of external assistance. No missions followed. Borrower and Executing Agency: Performance of ADB: Impact: Overall Assessment: Quality of PCR: Satisfactory Satisfactory Satisfactory Moderate Satisfactory Satisfactory Moderate Successful Satisfactory . government assessment of the program. and accident evaluation laboratory (PCR. longer term government performance must ensure the policy reforms are continued. while the one in 2003 was led by another ADB staff.e. safeguards. The government did not provide the necessary operation and maintenance funds for the many activities required to sustain reform. The two missions in 2002 focused on tranche approval and had one mission leader. additional assistance is needed in the form of loans. and TA. To improve the program’s performance and help make it sustainable. and a special program administration mission in 2003. as the two tranches had been released upon general compliance with the covenanted reforms. The reform process would have also benefited further from closer field supervision by ADB staff (section C[vi]). Similarly. driver instructor/training school. Poor governance must be concluded. There is a need to coordinate.

To a certain extent. Staff Review Committee. some questions emerged on program effectiveness. J. Comments on PCR Quality The PCR appears thorough and comprehensive. K. and discussions of ADB’s Board of Directors. the lack of easily understood criteria for meeting reform requirements as covenanted caused some confusion in the PCR.10 I. In particular. . this validation appreciates the PCR’s candid approach in pointing out some of the program’s shortcomings during implementation. as general compliance rather than specific compliance was acceptable. Data Sources for Validation The sources for this validation report included the PCR. as well as back-to-office reports of supervisory missions. It generally conformed to IED guidelines. report and recommendation of the President. this validation also takes note of the good lessons identified by the PCR and some of the good recommendations made. Where assessment of the program and the PCR proved difficult when general compliance was not clear. minutes of the Management Review Meeting. Recommendation for Independent Evaluation Department Follow-Up IED may want to perform a detailed evaluation of the sector development program given the many difficulties the policy loan faced and the difficulty that sustainability seems to be facing. However.

. received the following response from the Transport and Communications Division.REGIONAL DEPARTMENT’S RESPONSE TO THE PROGRAM COMPLETION REPORT VALIDATION REPORT On 14 December 2009. Central and West Asia Department. We have no comment. IED2. Independent Evaluation Department. Director.