Public Disclosure Authorized

Document of
The World Bank
Public Disclosure Authorized

Report No: 32561

IMPLEMENTATION COMPLETION REPORT

(PPFI-P9160 IDA-29210)

ON A

CREDIT

IN THE AMOUNT OF US$ 28.8 MILLION

TO
Public Disclosure Authorized

PAKISTAN

FOR

IMPROVEMENT TO FINANCIAL REPORTING AND AUDITING PROJECT

November 14, 2005

Public Disclosure Authorized

Financial Management Unit (SARFM)

South Asia Region
 CURRENCY EQUIVALENTS
(Exchange Rate Effective May 2005)
Currency Unit = Rupee (Rs)
Rs 1.00 = US$ 0.0168
US$ 1.00 = Rs. 59.5

FISCAL YEAR
July 1 June 30

ABBREVIATIONS AND ACRONYMS

AATI Audit and Accounts Training Institute

AG Accountant General
AGP Auditor General of Pakistan
AGPR Accountant General Pakistan Revenue
AMI Audit Management Information System
CAAT Computer Aided Audit Techniques
CAS Country Assistance Strategy
CBR Central Board of Revenue
CDMD Course Design & Materials Development
CGA Controller General of Accounts
CoA Chart of Accounts
DAGP Deputy Auditor General of Pakistan
DAOs District Accounts Offices
DDO Drawing and Disbursement Offices
DO Development Objectives
DPL Development Policy Loan
FABS Financial Accounting and Budgeting System
FAM Financial Audit Manual
FDs Finance Departments
FY Fiscal Year
GFS Government Finance Statistics
GOP Government of Pakistan
HRM Human Resource Management
ICB International Competitive Bidding
IMF International Monetary Fund
IT Information Technology
LAN Local Area Network
MIS Management Information Systems
MOF Ministry of Finance
NAM New Accounting Model
NWFP Northwest Frontier Province
OAG Office of the Auditor General
OAGP Office of the Auditor General of Pakistan
PA&AS Pakistan Audits & Accounts Service
PAC Public Accounts Committee
PAD Pakistan Audit Department
PIFRA Improvement to Financial Reporting and Auditing Project
PPF Project Preparation Facility
QAG Quality Assurance Group
RFMA Regional Financial Management Advisor
RFP Request for Proposal
SAR Staff Appraisal Report
TTL Task Team Leader
WAPDA Water and Power Development Authority
WAN Wide Area Network

Vice President: Praful C. Patel

Country Director John W. Wall
Sector Manager Robert Saum
Task Team Leader/Task Manager: Pazhayannur K. Subramanian
 PAKISTAN
IMPROVEMENT TO FINANCIAL REPORTING AND AUDITING

CONTENTS

Page No.
1. Project Data 1
2. Principal Performance Ratings 1
3. Assessment of Development Objective and Design, and of Quality at Entry 2
4. Achievement of Objective and Outputs 8
5. Major Factors Affecting Implementation and Outcome 19
6. Sustainability 22
7. Bank and Borrower Performance 23
8. Lessons Learned 28
9. Partner Comments 31
10. Additional Information
Annex 1. Key Performance Indicators/Log Frame Matrix 32
Annex 2. Project Costs and Financing 34
Annex 3. Economic Costs and Benefits 36
Annex 4. Bank Inputs 37
Annex 5. Ratings for Achievement of Objectives/Outputs of Components 39
Annex 6. Ratings of Bank and Borrower Performance 40
Annex 7. List of Supporting Documents 41
Annex 8. Comments from the Borrower 42
Project ID: P036015 Project Name: IMPR FIN REP & AUDIT
Team Leader: Pazhayannur K. Subramanian TL Unit: SARFM
ICR Type: Core ICR Report Date: November 14, 2005

1. Project Data
Name: IMPR FIN REP & AUDIT L/C/TF Number: PPFI-P9160; IDA-29210
Country/Department: PAKISTAN Region: South Asia Regional
Office
Sector/subsector: Central government administration (100%)
Theme: Public expenditure, financial management and procurement (P);
Other accountability/anti-corruption (P); Standards and financial
reporting (P); Other economic management (S)

KEY DATES Original Revised/Actual

PCD: 04/05/1993 Effective: 09/20/1996 02/18/1997
Appraisal: 03/03/1995 MTR: 06/15/1999 12/12/2000
Approval: 09/17/1996 Closing: 06/30/2002 05/31/2005

Borrower/Implementing Agency: Islamic Republic of Pakistan/ Pakistan Audit Department; Ministry of Finance
and The Planning Commission
Other Partners:

STAFF Current At Appraisal

Vice President: Praful C. Patel Joseph Wood
Country Director: John W. Wall Sadiq Ahmed
Sector Manager: Robert J. Saum Marie G. Zamor
Team Leader at ICR: Pazhayannur K. Subramanian Anthony Graeme Lee
ICR Primary Author: Pazhayannur K. Subramanian;
Sati Achath; Ali Hashim

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely,
HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S
Sustainability: L
Institutional Development Impact: SU
Bank Performance: S
Borrower Performance: S

QAG (if available) ICR

Quality at Entry: S
Project at Risk at Any Time: Yes
3. Assessment of Development Objective and Design, and of Quality at Entry
3.1 Original Objective:

The objectives of the US$37.2 million Improvement to Financial Reporting and Auditing Project
(PIFRA) (Credit US$28.8 million equivalent) were to: (a) improve public sector accounting and financial
systems; (b) provide the basis for enhancing public sector accountability; and (c) support improved
institutional capacity for economic policy-making and management.

The objectives were clear and important to the country’s improvements in governance and
accountability. They were also timely and appropriate to the needs of the Borrower, in light of the
following factors:

a) Due to serious deficiencies in the financial systems, financial data for financial planning,
budgeting, reporting and control was unreliable. Financial reports and final accounts, mostly
based on manual accounting systems, were neither timely, complete nor designed to meet user
needs.
b) Reports produced by the Pakistan Audit Department (PAD) did not meet the objectives of a fiscal
reporting system. Crucial financial information such as budget data and revenue collection details
were not reported. Financial information was fragmented among various agencies and a reliable
cohesive picture could not be obtained. As a result, the reporting system did not provide the
government with an effective method to: (i) monitor the fiscal deficit and its financing for aggregate
control; (ii) manage its cash and debt position; (iii) monitor expenditures in relation to
appropriations; and (iv) provide analyses of spending and revenue trends to assist with policy
formulation and performance measurement.
c) PAD, the supreme audit institution of Pakistan, was compromised due to the agency's dual
responsibility of maintaining the accounts of entities and being responsible for their audit.
Separation of the accounting and auditing functions within government was viewed as a primary
requirement for establishing the independence and credibility of audit.
d) PAD's audit function was viewed as a means of detecting irregularities and fraud rather than as a
management control function. The focus was to verify transactions regardless of their value.
Reviews of internal controls for system weaknesses were normally not carried out. Audits had
limited coverage due to lack of resources and quality was suffering due to a lack of expertise.
Revenue audits did not have a major role in PAD's audit function. Only a small proportion of
Drawing and Disbursement Offices (DDOs) were audited each year and purposeful parameters,
such as potential risks or weaknesses in internal controls, were not used as selection criteria for
audits. Certification audit had made little progress due to the volume of transactions and lack of
expertise within PAD. In the absence of auditing standards and quality assurance, the quality of
audit work was low.
e) The human resource management functions in PAD were limited in scope. Personnel at the
accounts offices were frequently not adequately qualified or trained. The effectiveness of training
programs was undermined by a lack of application of training.

The project was consistent with the Bank's Country Assistance Strategy (CAS) for Pakistan
discussed on December 15, 1995. The CAS stressed the centrality of fiscal adjustment to Pakistan's reform

-2-
and development efforts. Addressing these concerns involved, inter alia, improved resource mobilization,
public expenditure reform, more effective use of government resources, and improved public sector
management. The project focused on developing more effective financial accounting, reporting, control and
monitoring systems, and thereby providing an essential support for these activities. The policy support
component aimed to achieve CAS objectives by contributing to improvements in macroeconomic
management. CAS, in support of the establishment of a more competitive enabling environment also
sought to improve governance in Pakistan. The project’s emphasis on improved accounting and financial
reporting was expected to have a direct impact in this regard. Governance would also be assisted by the
improved transparency of the linkages among budgeting, internal controls and accountability resulting from
the new systems. The improved financial systems would help ensure the routine application of all
prescribed controls on financial transactions. In the longer term, the measures to enhance the auditor's
independence and strengthen audit capacity would also contribute to improved governance.

The project was also in line with the government's ‘Vision Statement’ and strategy to address
problems in accounting and auditing by introducing policy changes relating to: (a) adoption of improved
accounting principles and standards for the government sector; (b) a phased transfer of responsibilities for
government accounting from the Auditor General to the executive arm of government; (c) enhanced
independence, standards and capacity for government audit, including greater use of private sector firms in
public sector finance; and, (d) development of the skills and competencies of government finance staff,
including revised human resource management and training policies.

The project design tried to reflect lessons learned from similar projects which showed the need for:
(a) a high level of commitment on the part of the member country to project objectives; (b) well-defined
objectives, consistent with the country's absorptive capacity; (c) thorough preparation and accurate
assessment of needs; and (d) rigorous supervision.

3.2 Revised Objective:

The objectives were not revised.

3.3 Original Components:

The project comprised components for public sector financial management relating to accounting,
auditing, institutional development (human resource management, training and administration), economic
policy-making and management. They were all related to achieving the project’s objectives.

Component I: Government Accounting and Financial Reporting (US$12.3 million)

This component provided for the development of accounting standards, reporting systems, and
financial administration procedures conforming, as far as possible, to internationally accepted principles
and the implementation of a set of high priority computer based financial systems based on these standards.
The systems would enable efficient processing of the very large number of government expenditure and
receipt transactions, enable the uniform application of prescribed controls for these transactions and
support information retrieval needs from government finance managers at the center and in the provinces.
The project provided for consultancy services to define and implement the new set of accounting standards,
and computer hardware, software, consultancies and training required for the design, development and
implementation of the following high priority financial systems at 21 out of a total of 95 sites:

a) Core Accounting and Reporting System: This system would provide integrated

-3-
government-wide accounting (when fully implemented across all sites), encompassing the functional
requirements for budget preparation and budget execution, and covering the appropriation, commitment,
funds allocation, and payment processes for investment and current budgets.

The system would be comprehensive in terms of coverage and would provide a credible source of
reliable and timely data. Accordingly, the Core Accounting System would be fully automated, interfacing
with external data sources, such as external debt, on a periodic basis. This system, introduced along with a
modern budget classification system and an appropriate chart of accounts, would enable expenditures and
revenues to be recorded at a detailed level and to relate these to specific programs and projects. The data
would be amenable to cross classification for financial and economic analyses.

The system would be used by: (i) PAD (including its subordinate units and its successor
organization) to perform the basic accounting functions; (ii) the federal Ministry of Finance (MOF) and the
provincial Finance Departments (FDs) to perform the processes associated with budget execution,
monitoring and control; and implement cash limits, as may be required; and (iii) the federal Planning
Division and the provincial Planning Departments to obtain the status of actual expenditures on ongoing
programs and projects. Data would also be directly available for use by agencies/spending units for
program and project management.

b) Payroll and Pension Systems: The payroll of government employees constitutes a large portion of
government expenditures. The need for an automated payroll and pension system was evident in view of
the increasing volume of transactions. The system would perform the full range of the payroll functions,
and the expenditures processed would automatically update the core accounting and reporting system.

c) Cash Flow Forecasting and Financial Analysis System: One of MOF's main information needs
was to establish a system for forecasting likely flows of payments and receipts, and the consequent impact
on cash balances and issue of debt instruments. The cash flow and financial analysis system would be
designed to meet this requirement. It would provide a powerful facility for retrieving current and historical
data from the accounting system, and it would allow end-users to manipulate the data for forecasting and
analysis purposes.

Component II: Government Auditing (US$6.6 million)

This component focused on audit systems improvement and systems development. It included:

a) Audit Standards and Techniques: Consultant services to: (i) prepare guidelines for the
introduction of international auditing standards; (ii) introduce modern auditing techniques and concepts;
(iii) review PAD's audit policies, processes and procedures, output, workload, efficiency and quality
assurance for the audit function; (iv) advise on methodology to enhance PAD's communications with stake
holders, for internal audit of PAD's systems under development; (v) to assist in the use of automated audit
tools and standard audit programs wherever appropriate; and (vi) improve communication of audit results
and to develop quality assurance programs.

b) Automated Tools: Provision of hardware and audit software for audit planning, sampling and
documentation, and for audit of automated accounting systems.

c) Capacity Development: Consultancy to support revised human resource management procedures

through international training in a modern financial system environment and through arranging attachments

-4-
with other Supreme Audit Institutions.

d) Use of Private Sector Firms: Assistance with developing modality for utilizing private sector
firms, and finance for private audit firms' fees.

Component III: Human Resource Management (HRM) (US$1.3 million)

The HRM component involved organization and systems development. To reflect its need for
increased professionalism and specialization, PAD would implement revised HRM policies relating to the
career paths of accountants and auditors, defining standard job descriptions, enhancing the effectiveness of
training developing a transparent and quantifiable system of evaluation and manpower planning. The
project provided consultant assistance for: (i) managing the process of change; (ii) preparation of job
descriptions and performance standards, and (iii) installing an MIS system for HRM planning and
monitoring.

Component IV: Training (US$6.0 million)

The training component included: (i) moving towards professional-level training; (ii) providing
retraining associated with the changes introduced by the project; (iii) strengthening the quality of training
and training facilities, and (iv) expanding the circle of beneficiaries. The project provided refurbishment
and some extension of existing facilities, equipment, computer hardware, materials, and consultants to
assist in training-needs assessments, course design, materials development and training of trainers. The
component also included international scholarships for faculty trainers, and seminars. In addition to
including provincial staff in its training programs, PAD would share training course materials with
provincial training institutes as appropriate.

Component V: Administration (US$6.3 million)

The project included administrative capacity development as follows:

(i) MIS Wing: The project supported the creation of an MIS Wing and provided for: (a) a Software
Development Center; (b) selected posts to be staffed by contract professionals (such as the General
Manager (MIS) and other technical specialists); (c) local training for PAD staff in systems development;
and (d) recurrent costs for systems maintenance.

(ii) Project Management: Strengthening the Project Directorate, through provision of staff, facilities,
hardware and software, consultants and training. Assistance was also to be provided for technical review
of computerized information systems' development, and for a senior level advisory panel of international
experts, which would at least meet annually.

(iii) Future Project Preparation: Assistance for the preparation of the follow-up project if grant funds
were not available.

(iv) Office Automation: Provision for a local area network and associated office software for PAD.

Component VI: Other Policy Support for Economic Management (US$1.7 million)

MOF and the Federal Planning Commission managed selected studies on economic and financial
policy issues related to the on-going macroeconomic and structural reform program, a diagnostic study of

-5-
the institutional arrangements for macroeconomic policy-making, and technical assistance/training to
upgrade staff skills. Specific areas included international capital markets, foreign asset/liability
management, improving the revenue yield of certain taxes, on-the-job specialized training in macro
modeling, and assistance for establishment of a nucleus in the Planning Commission for analyzing and
monitoring poverty. The project financed the costs of (mostly local) consultancy services required for the
studies and short-term technical assistance for staff training.

3.4 Revised Components:

The components were not revised.

3.5 Quality at Entry:

There was no official assessment of the quality at entry by the Quality Assurance Group (QAG).
The ICR considers the quality at entry to be moderately satisfactory for reasons explained below.

The project objectives were consistent with the CAS and the government priorities. The project
had a sound results framework at entry to enable tracking the operation’s progress towards meeting its
Development Objectives (DOs). Staff Appraisal Report (SAR) sets out a useful framework covering both
output/process and outcome indicators. The project’s assumptions about reaching the DOs were realistic
in terms of the technical feasibility of the components and their relationships to the planned outcomes. The
project design/team was realistic about the time required to implement such a complex and challenging
project, and expected that the full impact would be shown only over a 10-15 year period. Intermediate
project outcomes to be used for project monitoring provided a realistic framework to measure project
progress.

During preparation of the project, the following risk factors were identified:

• Vested interests. By improving the quality of financial systems and reporting, the project would
introduce controls which would affect those who benefit from weaknesses in present arrangements.
These interests may act to delay the project, or to divert it from its objectives.
• Resistance to change. Organizational risks included PAD's capacity for managing the process of
change, especially those aspects related to its union and staff concerns arising from the introduction
of new procedures, technology and the functional separation of accounting and audit.
• Technology development and operation. Risks related to the development and operation of
computer-based financial information systems would arise as staff have limited technical skills and
experience of technically more complex financial systems.
• The risk of inadequate skills for obtaining the benefits of improved technology by controlling the
quality of financial data inputs, and applying the financial information generated.
• Project management. The complexity of the project, and the likelihood of delays in decision
making and implementation, increased project management risks.
• Policy Changes. There was a risk that policy changes or revised administrative arrangements
could be introduced which would disrupt the project.

Though measures to mitigate the above risks were incorporated into the project design, the

-6-
enforcement of these measures proved to be very difficult during implementation. The project design
catered adequately to technical aspects but under estimated considerations of the political economy. Thus,
for example, even though the SAR had adequately stressed the need for skilled and qualified technical staff
required for implementation of the automated accounting systems (including a legal covenant for the
appointment of a GM-MIS at an appropriate level and with adequate remuneration), a reasonable level of
technical staffing could only be achieved toward the end of the project. This was due to the reluctance of
generalist Pakistan Audits & Accounts Service (PA&AS) officers who viewed this position as unnecessary
and did not recognize the need for technical staff as permanent members of the PAD. The separation of the
functions of audit and accounts, though acceptable from a technical stand point, was resisted by PA&AS
officers who viewed this as a threat to their career prospects. Though the need for an automated
accounting was recognized by every body, the introduction of the system being designed under the project
was resisted by the provinces which used this as an opportunity to re-open the long standing issue of
provincial control over accounts (provincialization of accounts).

In practice, the project design proved to be too complex and expectations about implementation
capacity and the will of the Government to improve this capacity during preparation proved optimistic.
This led to difficulties in implementation. In hindsight it is felt that perhaps too many reform activities were
bundled into a single project. Though project activities reflected essential reform priorities and the SAR did
state explicitly that the overall reform would take 10-15 years or more, perhaps the first tranche of the
reform effort could have been further sliced to initially concentrate on repairing the accounting system and
later addressing larger institutional reform issues.

In addition, it appears that the quality at entry could have been further enhanced in the following
areas:

(a) Issues such as the separation of auditing and accounts may have been better addressed under the
umbrella of the overall country development policy dialogue and by establishing linkages with
other possible instruments such as Development Policy Loans (DPL). In the second half of the
duration of this project such linkages were in fact established with parallel Bank structural
adjustment operations and the IMF program and this reinforced the Government's commitment to
meeting key policy milestones such as the approval and introduction of the new NAM and CoA.
(b) MOF approval/sanctions required for the creation of specialist IT positions required for the
implementation of PIFRA systems should have been obtained prior to project start-up.
(c) The need for more frequent and broad-based discussions with all stakeholders, and a more
participatory approach involving all relevant departments and players and a framework (e.g.
specified schedule) for such discussions through out the life cycle of the project, should have been
specified more clearly in the project design. These discussions could have removed the
misperceptions of key stakeholders regarding some of the issues mentioned above. In practice, as
project implementation proceeded, the PAD team did brief counterparts in other departments about
project essentials. However, due to a government policy of frequent transfers within departments,
incumbents changed frequently and often key officers of the Provincial Governments, the Federal
MOF and even the field officers of the PAD in place, at a particular point in time, were not always
familiar with project scope, advantages and implementation program. This resulted in an erosion
of stakeholder commitment and an inadequate ongoing involvement of key players on whom
successful implementation depended.
(d) The project design should have elaborated the change management activity in greater detail and
adopted a more in-house approach to implementation, instead of relying heavily on outside

-7-
 consultants. During project implementation, this task was eventually re-allocated to a team of
experienced PAD staff and when this was done, progress improved considerably.
The ICR therefore deems the quality at entry to be moderately satisfactory.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:

Satisfactory. The Bank’s project team had recognized at the outset the difficulty in implementing
such projects and the long time required for achieving all objectives and outcomes. The SAR stated that,
this is a 10 to 15 year reform program and PIFRA is the first project in this reform program.

In practice, the project proved to be extremely demanding and complex for the Borrower because
of: (a) the scale of the change involved in implementing reforms across multiple agencies and levels of
government; (b) the involvement of multiple stakeholders with differing agendum; and (c) resistance to
change from several of the stake holders on matters mentioned above in section 3. 5.

To aggravate matters, the capacity of the PAD in terms of administrative resources and qualified
staff proved to be insufficient to carry out a project with a large Information Technology (IT) component
which included introducing modern accounting system country wide. The lack of adequate hiring
mechanisms within the government to hire highly qualified technical professionals at market-based salaries
further impeded the implementation process.

The combination of the above factors coupled with the internal security situation in Pakistan, post
September 11, (as discussed in section 5.1) led to significant project delays and the project closing date had
to be extended twice. The project closed about three years later than originally scheduled.

However, after the coming into power of the present government and with its focus on reforms,
opposition from several groups with vested interests in one or more aspects of the currrent system, was
neutralized. Over a period of time, especially after the current Auditor General assumed office, the
professional project management capacity of the PAD also improved. These factors led to an improvement
in the pace of implementation and significant gains were made during the later years of project
implementation. Most of the key project outputs, as designed, were delivered and they contributed to the
achievement of project development objectives. Accordingly, in view of the project achievements detailed
below, achievement of the DOs is rated satisfactory.

Objectives I and II: Improving public sector accounting and financial systems, and providing the basis
for enhancing public sector accountability:

Project outputs from the following components contributed to these objectives.

A. Government Accounting and Financial Reporting

The main achievements of the project in this respect are:

l Responsibilities for government accounting have been transferred from the Auditor General to the
executive arm of the government, namely the Controller General of Accounts (CGA), through the

-8-
 enactment of the CGA Ordinance.
l A road map has been laid for better public sector accounting and financial reporting systems
through the:
l adoption of improved accounting principles and standards by the Government of Pakistan
(GOP) through the New Accounting Model (NAM) which inter alia involves a move from
single entry to double entry accounting procedures, and introduction of commitment
accounting.
l design and implementation of a new budget classification system and chart of accounts (COA)
conforming to international standards.
l design and implementation of a system for automation of budgeting, accounting and reporting
functions of the federal and provincial governments and implementation of this system at 30
sites.
The new budget classification system and the CoA conforming to the International Monetary
Fund’s (IMF) Government Finance Statistics (GFS) has been adopted by the government. The Federal
budget and the budgets of three provinces (NWFP, Punjab and Sindh) for the year FY04-05 were prepared
according to the new CoA. For FY05-06, it is expected that all provinces would also convert to the new
CoA. After an extensive training program, about 20,000 Drawing and Disbursement Officers (DDOs) and
accounting staff have been trained in the use of the system and Government finance managers are now
using it as an essential tool for budget preparation, budget execution and fiscal control.

B. Government Auditing

The project has strengthened the capacity of the Office of Auditor General (OAG) through the
introduction of enhanced standards, systems, and better trained staff. After the separation of accounting
and auditing functions, PAD is split into OAG for auditing functions, and CGA for accounting functions.
The government auditing component completed all the seven elements. Trial field implementation of the
manual in seven areas included very significant audits for live testing of the manual – Revenue, Railways,
Works, Water and Power, accounting for domestic debt, accounts of Abbotobad District, the Public
Account of interdepartmental and judicial deposits. The testing included use of the computer aided audit
techniques (CAATS) software ACL for acquisition of data from the SAP system in the Abbotobad District
Office and the office of Deputy Auditor General of Pakistan (DAGP). Live audit testing was extended to
all areas of the DAGP as an implementation strategy for the new audit manual.

Training in the new financial audit manual and in the new CAATs and audit management software
ADM PLUS have been rated very highly by participants. The training was supported by three
international consultants assisted by local consultants and DAGP counterparts to ensure sustainability of
the training.

Objective III: Supporting improved institutional capacity for economic policy-making and management

The project was instrumental in improving the competence of government finance staff through
training and exposure to modern tools and techniques. The outputs from the component “Other Policy
Support for Economic Management” contributed to this objective.

MOF: Nearly fifty officers were sent on training and short courses abroad to attend various short-term
courses on Debt Management, Government Regulations, Macroeconomic Policy and Management, Foreign
Exchange Management, Budgeting and Financial Management in the Public Sector. About 5 to 7

-9-
higher-level officers attended various workshops or training programs (on topics such as GFS budget
classification, fiscal reporting etc.) at the World Bank and IMF in Washington. Two consultants were
hired and equipment was purchased for the Economic Advisory Board which played a very important role
in the formulation of Interim Poverty Reduction Strategy Paper, later upgraded to Poverty Reduction
Strategy Paper.

Two officers of the Central Board of Revenue (CBR) officials were sent to Columbia University
for Masters in Economic Policy Management, and to UK for Masters in Development Management. The
project financed setting up of computer laboratories in Karachi, Islamabad and Faisalabad. These
laboratories are functioning and CBR staff at all levels are receiving training in these laboratories. It also
financed sending CBR officials for certain selected overseas training.

4.2 Outputs by components:

After initial delays, the project components were able to deliver most of the key outputs as
designed.

Component 1: Government Accounting and Financial Reporting

The main achievements of this component from a functional standpoint have been described in
section 4.1. The physical outputs from the component which have enabled these achievements are
summarized here:

l Design and implementation of NAM, a new CoA and budget classification structure, accounting
standards and procedures based on international accounting principles
l Design of functional specifications for a new budget preparation and accounting system based on
these standards. The system also includes modules for payroll, pensions, provident fund
accounting and cash forecasting
l Acquisition, testing and implementation of application software (SAP/R3) based on these
functional specifications
l Acquisition, testing and implementation of hardware and network equipment required for the
implementation of the new software
l Implementation of the new systems at 30 sites to deliver the functionality as described in section
4.1 above
The new automated system enables:

l Compilation of the federal and provincial budgets based on proposals received from line ministries
and provincial line departments respectively
l Recording of cash allocations and commitments
l Approval and processing of expenditure transactions – the approved budgets for line ministries and
departments are available in the system and this enables an automatic budget check in the system
prior to authorizing any commitments and payments
l Recording of receipt transactions
l Reconciliation of transactions with the government bank (National Bank)
l Preparation of a full set of fiscal and accounting reports
l modules for payroll, pensions, and provident fund accounting and cash forecasting
The system has the full functionality envisaged in the original project design and is being used to
process government expenditure and receipt transactions at those offices where it has been implemented.

- 10 -
The payroll, provident fund and pensions of government employees serviced from these centers are also
being processed by the system.

As envisaged in the SAR, the system was implemented in a phased manner. In Phase I, it was first
tested at a test site in Islamabad; in Phase II, it was piloted at the AGPR Islamabad, the AG-NWFP and
two DAOs reporting to the AG; and in Phase III, it was replicated to a total of 30 sites, which include all
provincial AGs, and selected DAOs in each province. The sites were selected to achieve maximum
coverage in at least one province. The SAR had identified this province to be Sindh. However, during
implementation this was changed to NWFP due to a greater buy-in by the provincial government there,
and its proximity to Islamabad from where the PIFRA team operated. However, as implementation
proceeded, major sites in all other provinces (Sindh, Baluchistan and Punjab) were also covered during
the replication phase.

The original number of sites to be automated in PIFRA was envisaged to be 21 in the SAR.
Though PIFRA wanted to cover 52 sites, by the closing date of the project it completed systems
implementation at 30 sites. The total percentage of expenditure transactions covered by these sites is
approximately 40% of the Federal and provincial budgets. The remaining roll-out sites will be picked up
as part of work under PIFRA II. The process of replication is on-going and PIFRA II provides for
replication at all remaining sites. Retroactive financing provided under PIFRA II has enabled the
government to carry on implementation activities over the interim period until PIFRA II becomes
effective.

Several ancillary features have also been developed which are described below:

l Modification of the Legacy system and change of manual procedures to accommodate the new
CoA: Recognizing that each province will have three categories of sites [(i) operating the new
SAP/R3 system; (ii) using the old Cobol based legacy system; and (iii) continuing to work on a
manual basis)] during project implementation, the PIFRA team initiated a program to modify the
legacy COBOL system to accommodate the new CoA. This has enabled the new CoA to be
implemented at even those sites where the new SAP/R3 system is still not in use and has
tremendously facilitated the transition to the new SAP-based system at these sites. In addition,
those DAOs that were operating a manual system have changed their ledger forms used for
recording expenditure and receipt transactions to accommodate the new CoA All DAO staff,
country wide have been trained in the use of the new forms and the new CoA. In addition, a
module has been developed that enables the provincial accounts to be prepared on the basis of
inputs provided by the SAP /R3 system, the legacy system, and from sites that still operate on a
manual basis. Taken together, these measures have enabled the universal implementation of the
new CoA and budget classification structure through out the country.
l The PIFRA systems were originally intended to cover federal and provincial budgets and
expenditures. However, while the project was being implemented, the government announced a
major policy shift which involved devolution of financial powers to the district and sub-district
levels of government. This required a change in the system implementation strategy and after this
devolution of fiscal authority to the district level, the systems were expanded to include a module
that covers District Government Functionality. This Module enables the new SAP-based system
to process district level transactions and produce district accounts. This module was completed
during project implementation and has been included as part of the standard functionality in the
system.
l Telecommunications: A Wide Area Network (WAN) has been implemented connecting various

- 11 -
 sites to their respective central nodes at the provincial headquarters and at the center. This has
enabled the implementation of a centralized architecture with the main application servers located
at provincial headquarters. This enables the District Accounts Offices (DAOs) to process their
transactions using the SAP/R3 system operating at the servers located at provincial headquarters.
l Technical Staffing: A team of 17 technical staff experienced in the design and implementation of
automated accounting systems in using SAP/R3 has been selected and hired by PIFRA at market
salaries. This staff has now been deployed at various sites and form part of the core systems
implementation effort.
The component has delivered its designed outputs. On this basis, the component is rated
satisfactory.

Component II: Government Auditing

Sub-components:

(i) Completion of the Financial Audit Manual, Supervision Instruments, Working Paper Kits and
Specific Guidelines

A general financial audit manual was first developed, and later, supervision instruments and
working paper kits with audit guides for 21 accounting cycles were developed. Once this was completed,
16 instruments were accepted. These instruments give tailored audit guidelines for each of the specific
audit areas on how to use the Supervision Instrument and Working Paper in their own scenario. These
Guidelines were for the audit of: (1) Federal Government; (2) provincial governments; (3) district
governments; (4) procurement; (5) foreign-aided projects; (6) Customs and Sales Tax; (7) Income Tax; (8)
civil works; (9) Railways; (10) Water and Power Development Authority (WAPDA); (11) commercial
enterprises; (12) Defense Services; (13) Zakat Collection; (14) privatizations; (15) Environment; and (16)
Foreign Missions.

(ii) Procurement of Hardware and Software

ADM Plus was procured for the purpose of Audit Management and ACL was procured for
CAATs. These two items of software were used in the test implementation field audits. Hardware worth
US$1.3 million was purchased through International Competitive Bidding (ICB).

(iii) Test implementation of the audit manual

In this component DAGP identified seven audit teams and assigned audit objectives, as follows:

(a) Federal Audit team – audit a portion of Repayment of Domestic Debt (a significant component of
the federal appropriation account);
(b) Provincial Audit team – audit a portion of Interdepartmental and Judicial Deposits (a significant
portion of the Financial Statements of Public Accounts);
(c) District Audit – audit a portion of payroll data from the SAP R/3 accounting system pertaining to
the District of Abbottabad (to enable use of CAATs software tool against payroll which is a
significant component of District expenditures);
(d) Railways Audit team – audit consumption of fuel (a significant element of Railways expense);
(e) WAPDA Audit team – audit of Civil Works (a major activity of WAPDA);
(f) Works Audit – Audit of Pak PWD (a major activity);

- 12 -
(g) Revenue Audit – Audit of Customs receipts pertaining to Vehicle Imports (an important Revenue
activity).

The Field Audits achieved the following results:

(a) It was demonstrated that the new audit methodology and associated tools could be applied by
DAGP auditors to audit components of the federal, provincial and district government financial
statements;
(b) The DAGP auditors involved, and their supervisors, gained valuable exposure to the methodology
and tools developed for DAGP, as well as to the conceptual underpinnings of the certification audit
and the practical considerations that have to be accommodated.

The Field Audits provided: (a) a valuable training opportunity; (b) exposure to the issues of
collaborating with the private sector; and (c) also demonstrated conclusively that the new audit
methodology and the supporting documents and tools are applicable in the Pakistan context. It also proved
that the tools can be used effectively by DAGP audit teams as long as they receive appropriate training and
support.

(iv) Seminars on changes

Five stakeholder seminars were arranged in April 2004. One seminar was held in the federal
capital and four seminars were arranged in provincial capitals. The Seminars were aimed at important
stakeholders of the Audit Department and were also attended by Members of Cabinet, Members of the
Public Accounts Committees and Principal Accounting Officials. The stakeholders were informed of the
New Audit Methodology being adopted by the DAGP and its impact on the audited organizations.

(v) Training of a core group of auditors

Audit training materials were completed using the new audit manual and on-the-job training was
provided through the pilot testing activity. The inclusion of DAGP master trainer counterparts and local
consultants in the training program provided a strong source of expertise for further training.

(vi) Internal audit guidance

The completed deliverables relating to the Internal Audit Component are: (a) Internal Audit Plan;
(b) Internal Audit Manual; and (c) Draft Internal Audit Legislation. After the separation of accounting and
auditing, the CGA is entrusted with the internal audit function and is responsible for implementation and
roll out of the materials developed during the project. Five internal auditors were posted to large
departments and guidance has been provided to them for use of the manual.

(vii) Policy framework for using the private sector

The DAGP and the Private Sector Auditing Profession discussed the opportunity for working
together. They concluded that, rather than looking at collaboration with the private sector from the narrow
perspective of engaging private sector firms to perform public sector audits, there is a broader set of
options for DAGP to consider in the context of an integrated capacity building initiative to fulfill its
evolving responsibilities. An agreement was reached with the Pakistan Institute of Chartered Accountants
on cooperation with the PAD. This deliverable was for the first time tested during the test implementation
audits of the seven sites described above. The field audit teams were supervised by the private sector audit

- 13 -
firms, and it was concluded that private sector auditors can provide valuable assistance in coaching audit
teams whose members are new to certification audits and in helping with the audit work where tasks are
unfamiliar. The full impact of the use of the private sector would be achieved as these modalities of
cooperation are put into more general use in the Department in the more complex certification audits.
Achievement of this component is satisfactory.

Component III: Human Resource Management

The project provided consultant assistance for: (i) managing the process of change; (ii) preparing
job descriptions and performance standards, and (iii) installing an MIS system for HRM planning and
monitoring.

Outputs from each of these areas are discussed below:

(a) Change Management

The objectives of the change management process were to: (i) promote stakeholder awareness and
acceptance of PIFRA’s objectives; (ii) align various components of PIFRA; and (iii) anchor the change
process in the organization of CGA and AGP.

Outputs included change workshops in support of site implementation, videos showing

implementation progress (two in English and one in Urdu), posters about the project, and a quarterly
newsletter describing the project and the impact of its implementation. A final review report has been
prepared by the project consultants and provides an overview of the change process at the end of the
project.

(b) Modernization of the HRM function

(i) Assessment of the changed role and functions of HR management. The Status Quo Evaluation Report
identified scope for a multitude of issues to be resolved including professionalization of entry to the
PA&AS and the departmental cadre, the need for career paths in the audit and in the accounting
organizations, the need for adequate staff remuneration and performance incentives, the need to base
promotion on performance output, and the need for more rational manpower estimating. The report
assesses the changed role of the HRM function in the context of PIFRA and also proposed workable
changes in HR governance. Implementation of these changes will be taken up during PIFRA-2.

(ii) Manpower Plan. The workload for the next 3, 5 and 10 year horizons was estimated and a career
chart and re-deployment plan were proposed.

(iii) Detailed job descriptions and work performance standards. Detailed job descriptions and
corresponding work performance standards have been finalized in consultation with the CGA and the field
audit offices to incorporate all levels of hierarchy. This document links job description with performance
standards and the performance evaluation system. The job description serves three purposes: recruitment,
performance management and the assessment of training requirements of individuals.

(iv) Performance evaluation system. An output-oriented performance appraisal system which could be an
improvement on the current system has been prepared. It develops a structure of subjective standard
general skill performance and the so-called job performance factors, which are to be assessed against

- 14 -
achievements set out in the job descriptions papers. Roll out of the new system requires more stakeholder
discussions and involvement which are proposed during the next phase of the reform program.

(c) Report on Implementation of an HR MIS

This document suggests the scope of a possible Human Resource Management Information System
(HR MIS) for OAG . It identifies the users of the system and covers potential functions and reporting
structure of the MIS. OAG intends to use the HR modules of the SAP/ R3 system which are being used for
the Accounting and Reporting function to implement the functionality of the HRMIS as envisaged in this
report. Operationalization of this functionality will be taken up during PIFRA-2.

Overall assessment of the HRM Component:

After being taken in-house, the PIFRA Change Management element of this component has been
effective in creating stakeholder awareness and understanding and in mobilizing top leadership support.
The focus of Change Management is now shifting towards promoting and supporting stakeholder
involvement and thereby allowing stakeholders to share responsibility and ownership for PIFRA’s
implementation.

The HRM subcomponent was delayed and initial reports submitted by the consultants were
considered unsatisfactory, having to undergo substantial amendment. The recommendations of the reports
are to be implemented in the follow on PIFRA-2 project. Some of the proposals are at variance with civil
service norms and external approvals are required for implementation. This has not been addressed by
senior management, and so implementation of the acceptable HR proposals will need to follow later. The
HRMIS system which is now available through SAP R/3 also needs to be put into full use by PIFRA-2.

The PAD expected that PIFRA would implement revised HRM policies relating to the career paths
of accountants and auditors, defining standard job descriptions, enhancing the effectiveness of training and
developing a transparent and quantifiable system of evaluation and manpower planning. While the project
had prepared proposals in all of these areas, many of the implementation of the outputs got shifted to
PIFRA-2 due to delay in delivery by the consultants. So the achievement of this component is considered
to be only moderately satisfactory on an overall basis.

Component IV: Training

(i) Training of DDOs and others in NAM and CoA: Over 21,000 DDOs of federal, provincial and district
governments, DAOs/TOs and Finance department staff were trained in NAM and the new CoA. Of late,
officers of State Bank of Pakistan and National Bank of Pakistan have also been trained in NAM and CoA.
The effectiveness of this training is reflected by the fact that the change over to the new system/CoA in the
federal and NWFP governments for the year 2004-05 took place smoothly and without any major
problems. Budget call letters for other provincial governments for the year 2005-06 were issued according
to the new CoA and NAM which will now be followed all over the country. DDOs’ training is an on-going
activity and the Accounts and Audit Training Institute (AATI) has plans and resources to provide refresher
or additional training courses to the staff concerned as necessary. A feedback system is in place to watch
and ensure quality and effectiveness of the training programs.

(ii) Training of Auditors: About 120 officers who have been trained in Financial Audit Manual (FAM)
acquired new knowledge and skills in auditing, which will play an important role in successful

- 15 -
implementation of FAM. 137 officers were trained in CAATs. Introductory training on the Audit
Management Information System (AMIS) and audit software was also conducted along with CAATs
training. The effectiveness of this training is yet to be assessed, but it is hoped that familiarization with
CAATs will help adoption of new techniques at the level of field audit offices, and will lead to improvement
in the quality of audit results.

(iii) Course Design & Materials Development (CDMD) and testing for SAP(FI). The training materials
developed by the consultant were tested in the class-room and feedback obtained through evaluation done
by the participants, informal discussions and comments by observers. Changes were made where required
by the training expert and the subject matter expert. SAP(FI) was tested in the computer environment of
the office of AG Punjab.

(iv) Revising the modules already tested. As stated above, changes in the tested modules were made in the
light of feedback obtained and all revised training materials have been approved.

(v) MIS Report. This is a part of the package for the management and quality assurance of the training
function. The consultant has submitted a report, which has been approved.

(vi) Quality Assurance. Quality assurance on the deliverables prepared by the consultant was achieved
through setting up a Technical Committee for Evaluation and Acceptance of Deliverables. In the matter of
quality assurance of the training provided by AATI itself, regular feedback will be obtained from the
trainees, their supervisors and their departments.

(vii) Training Impact Report. The consultant has prepared and submitted the training impact report in the
light of guidelines suggested by OAG.

The Training function in PAD has moved towards a truly professional level, and has successfully
provided training and retraining associated with the changes introduced by PIFRA. AATI has been
refurbished and the existing facilities have been extended by providing equipment, computer hardware, and
materials and by engaging consultants to assist in training needs assessments, course design, materials
development and training of trainers. These have led to a strengthening of the quality of training and
training facilities. The circle of beneficiaries of training has been expanded to cover the staff of the
executive departments as well the provincial governments. The training materials are being shared with
other training institutes. The project had provided for international scholarships for faculty trainers which
could not be arranged because of various reasons.

Since the component has delivered most of its designed outputs, achievement of this component is
rated as satisfactory.

Component V: Administration

This component has delivered most of the outputs as per design and is rated satisfactory.

(i) MIS Wing:

This sub-component has been able to deliver its outputs as designed:

(a) The software development center envisaged in the project was completed in 1998 at a cost of Rs.12.29

- 16 -
millions.

(b) The position of GM MIS was created and staffed by a retired Government official for a few years at
the start of the project. However, when this individual retired, the post remained unfilled for several years.
It was only after difficult negotiations with the MOF, that this position was filled up again. However, this
could happen only towards the end of the project. The creation of contract posts for qualified technical staff
was similarly resisted by the MOF and required similar negotiations with the project team. These posts
were however eventually agreed and 17 technical staff members have been hired at market based salaries.
During the interim phase, PIFRA relied on contractor staff to maintain the systems. However, the new staff
members have now been integrated in the government project team and are now gradually taking over these
responsibilities from the contractor. To avoid a similar situation in PIFRA II the provision of technical staff
has been specified clearly in the PC- I (the Government project document that inter- alia also provides the
basis for obtaining MOF sanction for creation of these posts).

(c) The project provided financing for PAD staff to attend local training courses.

(d) Financed re-current costs for system maintenance of hardware and software through contract with
suppliers.

(ii) Project Management:

The project provided financing for office facilities, hardware and software required by the PIFRA
directorate for project management purposes. It also financed related consulting services and training. PAD
was reluctant to use the expert panel provided for in the project due to the exorbitant costs associated with
hiring international experts to staff the panel. This panel was eventually set up and one meeting was held.
But the Panel did not operate on a regular basis since the government felt that the costs exceeded the
benefits.

(iii) Future Project Preparation:

It was not found necessary to use the financing provided by PIFRA I for the preparation of PIFRA
II.

(iv) Office Automation:

PIFRA procured and implemented a Local Area Network, including personal computers, servers
and a Local Area Network (LAN) and associated office management software to connect the main officers
and staff of the PAD.

Achievement of this component is satisfactory.

Component VI: Other Policy Support for Economic Management

MOF: See Section 4.1, Objective III

Planning Commission: Strengthening of development and application of macroeconomic models was

achieved by purchase of computers and equipment from funds allocated under this component. Similarly,
Poverty Alleviation Section strengthening was achieved through purchase of computers. On these two

- 17 -
fronts, the “Other Policy Support Component” managed to achieve its objectives.

CBR: The project financed setting up of computer laboratories in Karachi, Islamabad and Faisalabad.
These laboratories are functioning and CBR staff at all levels are receiving training in these laboratories. It
also financed sending CBR officials for overseas training.

However there was no clear strategy for managing this component. It contributed in a piecemeal
manner to a wide range of activities, rather than in a focused way on economic policy support. While each
intervention may well be justified in itself, it finally comprised a diverse mix of international training and
computer purchases. Hence the component’s achievement is considered only moderately satisfactory.

4.3 Net Present Value/Economic rate of return:

Not applicable.

4.4 Financial rate of return:

Not applicable.

4.5 Institutional development impact:

The project has had a substantial institutional development impact.

Better fiscal control and financial planning: The project helped the federal and provincial governments
to have better control on financial resources and enabled better financial planning. For example, the new
CoA and budget classification structure gives a more logical breakdown of activities, and is therefore very
useful when preparing federal and provincial government budgets. The new systems have been
implemented at the key provincial and federal sites and are used routinely to process government
expenditure and receipt transactions and prepare accounting and fiscal reports. The systems enable uniform
application of prescribed controls and delivery of key fiscal and accounting reports. As system replication
has progressed, many user reservations about the systems have been removed and greater ownership has
developed amongst the staff of the Provincial AGs, the DAOs and the Accountant General Pakistan
Revenue (AGPR) sub offices in the provinces. Most importantly, offices of the Finance Secretaries of the
Provinces now view the new system as a helpful tool for the processes of budget preparation and budget
execution. The full impact of the systems would be seen when all sites are up and running.
Accounting and Financial Reporting. The CGA organization has been strengthened and annual final
accounts for the Federal Government have been produced within six months of year-end. Data reliability
has improved, and as a result, monthly and quarterly Federal reports are posted on the web and budget
execution data is sent to ministries by mid-month for reconciliation.
Auditing: The project resulted in the government accepting a new auditing methodology for financial
statement audits, and established a training program for that methodology. The project also led to changing
the selection criteria for auditors, and taking more auditors who have qualifications in accounting. The
software for audit management has improved the capacity of the OAG to program its audits and track
progress of audits and make recommendations. OAG has produced 16 audit working paper packages,
which were targeted to specific type or institutions, such as Railways, Defense, district governments,
provincial governments, and also environmental audits. These working papers have facilitated in making
these audits more clear and effective, and improving reliability on fiscal figures. Training provided to
auditors have, in the mean time, helped in enhancing audit quality and reducing the multiplicity of

- 18 -
non-viable audit observations.
There has been considerable progress in improving the timeliness of audits with audit reports
available within twelve months of the end of the fiscal year (i.e. six months after the accounts have been
prepared by the CGA). An Annual Audit Program that sets out specific goals and targets is being
published, and the Auditor General of Pakistan (AGP) has recently begun the process of recruiting over
800 qualified auditors to conduct audits under the District Audit Infrastructure. Timeliness of audit will be
improved under the PFRA II.
Training: Because of various types of training programs financed by the project, capacity of the staff of
OAG, CAG, MOF, CBR, provincial governments, district governments and spending units of all ministries
has increased. In addition, the new set of tools provided to managers has helped them to manage their
specific departments more effectively and efficiently. The training in NAM and CoA has facilitated their
adoption at federal and provincial levels. A pool of trained personnel has been created which will lead to a
smooth change-over to the NAM. Close liaison with the provincial finance departments has had a positive
effect towards collective ownership of the project. Other initiatives have resulted in a reorganization of
AATI, a move towards imparting professionalism in the training function itself, effective utilization of
trained personnel, and improvements in training materials and methods. The syllabus for the training of
probationers has been revised and implemented. A revised syllabus for the SAS studies and examination is
under consideration in OAG and on its approval will be used for the training program for direct recruits.
Project Management. After a long period of trial and error, the project implementation experience has
also contributed to strengthening the capacity of government and PAD in terms of managing large and
complex projects.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:

(i) International events. The aftermath of 9/11, war in Afghanistan, and the resulting insecurity in
Pakistan seriously affected project implementation. For example, the Canadian government issued a travel
advisory more than once and Canadian consultants had to leave Pakistan for three months in 2002. The
unexpected departure of these consultants at a crucial time affected implementation, by delaying the
preparation of the manuals, losing continuity of activities, and losing the confidence of national
government, which felt that the consultants were leaving them in the lurch. In addition, the deteriorating
internal security situation resulting in bomb explosions and other terrorist activities in certain provinces
such as Sindh and NWFP (where the first pilot sites were located) and border areas of Afghanistan also
adversely affected implementation. Major procurement contracts for IT system for implementing NAM
and consultancies for HRM, Training and Auditing components could not be signed as originally planned
owing to the post-September 11 international developments, as a result of which expatriate consultants
could not arrive for signing the contracts. This led to delays and necessitated an extension in the project’s
closing date.

(ii Change of government. Changes in the federal government resulted in the delay of the passing of
Auditor General Ordinance and CGA Ordinance which formed the basis of NAM. Similarly, the two-stage
bidding process for ICB of functional specifications was also delayed. The closing date had to be extended
twice because of various changes in government structure and technical environment, which led to
unavoidable delays and necessitated realignment of implementation plans.

- 19 -
5.2 Factors generally subject to government control:

(i) Delays in decision making. The project took more than five years to take root. This was mainly
because of the reluctance of policy makers to address issues in a timely manner. For example, the slow
implementation of the project can be attributed to the delays in getting approvals for the NAM and CoA,
resolution of political economy issues, such as control of the accounting function by the provincial
governments; reluctance to separate the accounting and auditing functions, the lack of good project
management, and other similar institutional issues. On the other hand, technical problems were found to be
more amenable to resolution, even for a complex system being implemented in a relatively under-developed
environment.

(ii) Insufficient MOF commitment. Even though the senior officials of the MOF did voice their support
for reform of the government and accounting system, they did not develop adequate knowledge about the
specifics of the reform program or what the implementation of such a reform program would entail. In the
absence of such support, vested interests, which viewed the implementation of the new system as a threat to
their position, were able to mount a successful opposition to the new system and cause significant delays in
its implementation.

In addition, to begin with, MOF considered this project as an accounting project, and as a result,
was less interested in it. In order to get the MOF’s interest and involvement, the project was presented as a
fiscal management project and linked with IMF benchmark and adjustment operations. Thereafter, the
MOF took an active interest and fell in line with this kind of approach. It took, however, a long time to get
their serious involvement.

(iii) Opposition from vested interests in existing systems who perceived the new systems to be a threat to
their career prospects. Major organizational changes and introduction of new systems instituted under the
project met with resistance partly because, they were perceived as being against the personal interests of
certain quarters within these organizations. In PIFRA, splitting of the audit and accounts function was
perceived as a possible threat to the scope of authority of the Auditor General’s organization and the career
prospects of officers of the PA&AS cadre. The introduction of the new budget classification structure was
resisted mainly by a small group of staff responsible for running the legacy budget preparation system who
feared that the new system would result in losing their jobs.

In PIFRA the major change requiring the separation of auditing and accounting functions was
supported at the design stage by an Auditor General who was not part of the PA&AS cadre and could see
beyond the cadre issues. However, no progress was made in this regard once he retired and a new Auditor
General, who was opposed to this change, took over. It was only after the coming to power of the present
government that wanted to overhaul the bureaucracy anyway, that this change was approved by the
government.

5.3 Factors generally subject to implementing agency control:

(i) High turnover of Project Directors. During the eight year period of project implementation, the
project director was changed 13 times, which resulted in periods of instability in the PIFRA Directorate and
affected implementation. During periods when a good project director was appointed, the project moved
quickly; in other periods it languished. The line of authority for this position was also not clear. In some
periods, the project director needed to get all decisions approved by another senior official in the
department who was not directly related to the project and had no interest in seeing it move quickly.

- 20 -
(ii) Capacity of Project Directorate. The PIFRA Directorate did not have the technical capacity to make
independent judgments on complex technical issues related to the installation of the computer based system.
These issues included custom developed solutions versus off-the-shelf software, centralized vs.
decentralized architectures, and over specified tender documents that restrained competition. The PIFRA
Directorate also lacked the necessary authority and expertise required for contract management. Payments
to consultants for delivered outputs were seriously delayed when clients expressed concerns about output
contents, leading to consultant withdrawals and dismissals, and non-completion of acceptable outputs.

(iii) Lack of knowledge of Bank procedures on the part of members of the committee constituted for
hiring consultants and contractor contributed to delays in procurement. They were unclear on the
evaluation criteria in the Request for Proposal (RFP) documents and therefore variations occurred while
appointing contractors and consultants. Similarly members of the technical evaluation committees did not
have the knowledge of the Bank procedures, which also led to delays. The unplanned demand from the
Regional Directors due to lack of awareness of the Bank’s procurement procedures was leading to undue
rush of work on Procurement Wing, thereby limiting its ability to plan effectively leading to delays. In
addition, frequent posting/transfer in the procurement wing was another factor which contributed to delay
in the implementation of the project.

(iii) Delays in contracting and delays in accepting consultant outputs. The Audit Component was
frequently disrupted due to disagreements between the client and consultants. As a result, the entire
contract period expired without making any headway to resolve the differences. Subsequently a new
consultant team was deployed upon extension of the contract in April 2003. The committee established by
the project deciding on service delivery contracts was unwieldy, and not quite responsive to quick decision
turn-around in respect of delivered project outputs from consultants. In the case of one large contract, each
task was dependent on the completion and approval of the preceding task. This made the activities of audit
component extremely inflexible and totally dependent on the quality of each deliverable.

5.4 Costs and financing:

The actual total cost of the project was about US$34.02 million compared with the SAR estimate
of US$37.2 million. The Bank financed US$28.00 million and the IDA credit of $28.82 million allocated
to the project was almost fully utilized. This constituted 82.3% of project costs. An amount of
approximately US $0.82 million was cancelled. The Government contributed US$6.02 million equivalent
in local costs which constituted 17.6% of project costs.

During the course of project implementation, the primary focus of the project became the
government accounting and reporting component, since progress on this component and design of the
accounting system had to precede work on some of the other components, such as auditing and training.
The bulk of the project costs was incurred under the government accounting and reporting component
which required $ 17.58 million of IDA financing and $1.66 million of government financing. These costs
were significantly higher (by about $5.9 million) than the appraisal estimate due to the following reasons:
(a) the delay in project implementation resulted in increased costs for consulting services provided by the
project supervisory consultants during the implementation phase; and (b) the contract with the hardware
and software vendor included the costs for a greater number of hardware and software licenses and for
implementation services than were required for just the 21 sites originally planned for PIFRA I. Instead of
the original estimate of 21, 30 sites were actually fully implemented and are in productive use, under
PIFRA I. Most of the additional equipment procured has been installed at the designated roll-out sites
during the period of the project. The Operationalization of these sites is under progress supported by
retroactive financing from PIFRA II. The other slight increase was in the HRM component which was due

- 21 -
to some unforeseen steps needed to be taken for information dissemination. Savings were made in training
and auditing components. The main saving in auditing component was due to the shifting of some of the
civil works to the next phase. Savings in training component resulted from certain economy measures
adopted through the use of in-house facilities.

6. Sustainability
6.1 Rationale for sustainability rating:

Likely. The sustainability of the project is rated as likely in light of the following factors:

• The government has demonstrated its strong commitment to sustain the achievements of the
project. The formal acceptance of core policy changes such as the separation of auditing and
accounting, the introduction of the NAM, the new CoA, the conversion of the federal and
provincial budgets to the new CoA, the implementation of new automated systems at 30 key sites
and creation of a core team of end user and technical staff through extensive training, are difficult
to reverse. The government has provided sufficient budget to meet recurrent costs and hired
adequately qualified technical people.

• The follow on project, PIFRA II, and also the possibility of PIFRA III (as envisioned in the SAR)
would enhance the sustainability of the project. For example, Financial Accounting and Budgeting
Systems (FABS) component of PIFRA II would follow up and extend the work done under PIFRA
I in implementing policies, systems and procedures that support overall project objectives of
improving the accuracy, completeness, reliability and timeliness of intra-year and year-end
government financial reports in Pakistan at the federal and provincial levels, and some district and
tehsil governments. Successful completion of this component would be instrumental to achieving
full government financial accountability and transparency in the new automated environment.

• The Office of the Auditor General of Pakistan (OAGP) component of PIFRA II would facilitate in
adopting modernized government audit procedures and internationally accepted auditing standards
that will contribute to eradicating deficiencies in program and evaluation capabilities in a timely
and effective manner. Likewise, PIFRA II’s CGA component would assist the CGA’s Office and
the AGs to professionally and adequately manage, and also to maximize the benefits from the
computerized accounting environment.

• The strong focus on training, and the international assistance would help sustainability and regular
operation. The AATI would continue to provide its regular blend of short courses and
probationers’ training. In some instances, such as the training of Drawing and Disbursing Officers
in NAM, it will also assist with mass training.

• The Development Unit under the restructured OAG will help to sustain all audit innovations. The
OAG’s office has identified a core team for implementing the new audit methodologies and trained
a set of master trainers in its office. These will provide continuous training on the new
methodologies and audit tools to all other staff.

However, it may be mentioned that sustained implementation of all new systems related to
accounting or auditing, is dependent to some degree on the continued buy-in by the AGP and CGA of
changes in ‘structure’, ‘style’, and ‘staffing’ introduced under the project. To derive the full benefits of the
change process, implementation of the proposals of PIFRA’s HRM and training components is particularly

- 22 -
necessary. This needs to be done in close collaboration between the PIFRA’s components and the top
management of the AGP and CGA as PIFRA II is implemented.

6.2 Transition arrangement to regular operations:

The transition to regular operations has already taken place. The government is already using the
NAM and the new CoA in its budgeting and accounting practices and major sites have already transitioned
to the new automated systems. Also see Section 6.1

7. Bank and Borrower Performance

Bank
7.1 Lending:

Satisfactory. The Bank's performance in the identification, preparation, and appraisal of the
project was satisfactory.

The project was based upon diagnostic analysis undertaken by PAD on the status of government
accounting and audit, which resulted in the preparation of the strategies underpinning the project. It
complemented and reinforced the government’s interest in financial reforms in the public sector. The
project also supported the government's effort to seek greater involvement of private sector firms in
government financial management. The Bank was well placed to provide assistance because of its
continuing dialogue with the government on fiscal adjustment, its involvement with executive departments
of both federal and provincial government, and its ability to provide, longer term support to undertake
subsequent stages of reform.

The task team ensured that the project was consistent with the CAS and the government’s
governance and financial management priorities. During preparation and appraisal, the Bank considered the
project design in terms of relevant aspects, such as technical, financial, economic, and institutional,
including procurement and financial management. During the appraisal, the Bank assessed the project's
risks and benefits, and also incorporated lessons learned from other projects both in Pakistan and other
countries. The task team was well-knit and had a good skill mix, and it was flexible and responsive to local
needs. The Bank consistently had a good working relationship with the Borrower during this period.

7.2 Supervision:

The Bank’ performance during supervision was mixed, ranging from satisfactory at the beginning
to moderately satisfactory during the middle years and finally improving to satisfactory during the last three
years. On an overall basis, the performance could be rated as satisfactory since some of the delays
encountered during the middle period were made up in the later years and most of the project objectives
were achieved.

Two separate QAG assessments carried out during the project, the first in July 1999, and the
second in September 2004, support this assessment.

The first panel of reviewers (QSA3) rated the Overall Supervision of the project as marginally
satisfactory. In addition, Supervision of Fiduciary Aspects, Adequacy of Supervision Inputs and Processes,
and Realism of Project Performance Ratings were also rated as marginally satisfactory. The Focus on
Development Impact was rated as satisfactory. This panel noted that:

- 23 -
 “The project has been given too little supervision since the key decision-makers reached agreement
to ensure its effective implementation. Too much dependence has been placed on gaining top down
support and too little attention to building understanding, support and motivation at the working
level. Since this project goes to the heart of the serious governance weaknesses in Pakistan, its
supervision merits higher priority than it appears to have been given in FY99.

The Technical Manager has made strenuous efforts to overcome the staffing problem which were
eventually successful. However, the management of the supervision of this project by the
Infrastructure Sector Unit must be questioned -- it would seem to fall more appropriately within
PREM's terms of reference, or with some kind of assistance from the Regional Accounting &
Audit Adviser. The project should be thoroughly reviewed as soon as possible, with a very well
staffed supervision mission. The active involvement of the Country Director was a positive aspect,
as is that of the IMF which hopefully will be continued and reinforced. Having the CD resident in
the field has greatly facilitated getting the top policy makers committed to the key project
objectives. It is questionable whether GoP should be permitted to proceed with the procurement of
millions of dollars worth of computer equipment before there is evidence of more convincing
government commitment to the project and agreement on a new work program and set of
monitorable benchmarks. Meanwhile, the supervision team needs to give more attention to
compliance on the financial covenants and establishing an effective outcomes oriented M&E
system. Lastly, it is important that someone on the supervision team should have experience
installing large accounting and auditing systems.

The supervision suffered from staffing problems. Both Country Director and Technical Manager
were active in following up on this project; the CD succeeded in getting the Minister of Finance
and AG to reach an agreement on both the Chart of Accounts and the splitting of accounts and
audit which was critical for the project to proceed. There was too little effort to build support and
understanding for the project among stakeholders at the working level and thereby to address the
many concerns of staff about the possible adverse impact of the changes on them personally.

Supervision was impeded by the prolonged absence of the Task Leader; this hiatus was not
corrected quickly enough. With regard to the Islamabad Field Office, their involvement appears to
have been relatively minor (except for the significant intervention of the CD to get agreement
between the MoF and the AG), whereas this is exactly the kind of project which would benefit
greatly from the intensive involvement of the Field Office in supervision, which is now planned.
Supervision was insufficient and incomplete given the slow implementation and the issues which
needed to be settled. Some components of the project were almost not supervised at all (e.g. the
components related to training and the Planning Commission).

This project was under-supervised. The managers substituted as best they could given their very
heavy other obligations, but more work is needed to help build stakeholder support for the project -
a successful project cannot be built simply by relying on top down decision-making.”

The problems identified in QSA3 led to various changes, including: (a) a revamping of the
supervision team to include two Financial Management Specialists as co-task leaders, one being stationed
in the field and another at HQ as a representative of the RFMA; (b) an increased level of supervision; and
(c) moves to build stakeholder support at the operational level. The new team also benefited from the
advice of a Panel of International Experts in accounting, auditing and change management apart from
constant input from the Cluster team leader and the Sector Director.

- 24 -
 The second panel (QSA6) rated the overall supervision of the project as satisfactory. ‘Adequacy
of Supervision Inputs and Processes’ was rated highly satisfactory, ‘Focus on Development Impact’ and
‘Supervision of Fiduciary Aspects’ were rated as satisfactory, and ‘Candor and Quality of Project
Performance Ratings’ was rated as marginally unsatisfactory. The panel noted that:

“This complex and high risk project has appropriately been well and intensively supervised, with
close involvement of Bank and client senior management. The significant supervision problems
identified in QSA 3 and other areas of vulnerability which could undermine implementation success
have been addressed. Certain key policy actions were buttressed through conditionality in parallel
adjustment operations. Implementation was greatly facilitated by an experienced, cohesive and
stable supervision team whose expertise covered the main components of the project.

The Bank drew the attention of top management to issues needing action in a prompt, detailed and
direct fashion. The improved dialogue with government resulted also from focusing the
government's attention more broadly on fiscal management objectives of the project rather than
narrowly on technical issues of accounting and auditing. Further increased focus on training and
workshops for operational staff helped to improve government commitment and to resolve some
implementation issues.

Bank supervision should have rated IP as U, in the panel's view, for FY02 and 03, given the delays
being experienced on various components. However, supervision did address the issues
substantively even if candor and quality of reporting could have been improved. Supervision
reporting could have made explicit comparisons of performance with targets.

The combination of regular supervision missions, regular video and audio conferences between the
Bank and the government team, and almost daily contact between the local FMS team and the
PIFRA Directorate has enabled close monitoring of implementation progress. This close attention
has been significantly enhanced by the appointment of a new international FMS to the Pakistan
office who assumed the role of co-TTL. All supervision mission reports have been clear and
detailed, with necessary follow up actions clearly set out. The team liaised closely with the IMF
and DFID, both of whom had incorporated conditions concerning this operation into their own
country operations, and had coordinated with ADB.

Communication with the authorities on problems needing resolution has been detailed, prompt and
frank, on an almost daily basis as well as through the more formal follow up process following
each supervision mission. The tying of this project into Bank adjustment lending operations and the
IMF program has provided the necessary incentive to the borrower to address identified concerns.
The CPPR of July 2003 also took stock of the project and identified key issues needing to be
addressed. Supervision attention to the quality and reliability of procurement administration was
satisfactory, as also to compliance with agreed procurement schedules.

The Country Director and the Operations Advisor were closely involved, including where
necessary in discussions at the highest levels with the authorities. Formal communication to the
borrower following supervision missions were generally signed by the CD. In addition both were
involved in meetings on the project with key officials.

Although inadequate funds were budgeted, this constraint was overcome by combining supervision
with missions for preparing the PIFRA 2 project. In addition it appears that the substantial time
devoted to supervision by the country based co-TTL has not been fully charged against this

- 25 -
 project”.

The ICR team concurs with the above findings of QAG and considers the Bank's supervision
performance during the implementation of the project as satisfactory on an overall basis.

7.3 Overall Bank performance:

Satisfactory. Overall, the Bank performance was satisfactory during project preparation, appraisal
and implementation.

Borrower
7.4 Preparation:

Satisfactory. The Borrower's performance in the preparation of the project was satisfactory.

Preparatory activities for the project, financed by a Project Preparation Facility (PPF), included:
(a) initiating the consultancy for developing accounting principles, standards and a financial reporting
framework to guide the detailed systems design; (b) establishment of the MIS organization structure by
appointment of a computer professional as General Manager MIS and other key staff in the MIS wing; (c)
undertaking a training needs analysis for provincial government staff including users of financial
information, accountants and auditors; (d) training for MOF, provincial and PAD's HRM, MIS and
training institute staff; (e) preparation of detailed designs and tender documents for civil works for site
preparation, and training facility renovation works; and (f) project management training and assistance
with setting up project management systems.

The project concept was widely discussed within PAD, to inform departmental staff of the
objectives of change and the opportunities generated thereof. A Steering Committee was formed to ensure
user involvement in the project and to perform the functions of interagency coordination. Representatives
from MOF, Economic Affairs Division, Planning Division, Pakistan Computer Bureau, and all four
Provinces assisted PAD to ensure that the financial accounting systems developed by the project were
responsive to user requirements and interests, that users' training needs were met, and that systems and
procedures relating to audit were effective. PAD also coordinated with the Federal Public Service
Commission and the Establishment Division on aspects relating to the implementation of its human
resource management strategy. Users were directly involved in the definition of the financial reporting
systems development, through consultation with the systems design consultants. PAD extended its training
to include users from outside the department. On these counts, the Borrower’s performance was
satisfactory.

7.5 Government implementation performance:

Satisfactory. The government’s implementation performance varied over the course of the project.
The government’s implementation performance was not satisfactory prior to FY02. Though the
institutional reforms were fully supported by the government during preparation, the government
commitment, particularly on institutional aspects, turned out to be half-hearted. During initial years of
project implementation, the MOF and AG were reluctant to give high priority to this project. On account of
insufficient commitment on the part of the MOF and the AG, opposition to the project intensified from key
stakeholders, such as the provincial governments, and this effected implementation and caused extensive
delays.

- 26 -
 These initial problems were, however, eventually overcome. The tying of this operation to a
parallel Bank SAL and the IMF's program assisted in enforcing Borrower commitment. In particular, MOF
and AG became fully committed to the objectives of the project and the implementation pace gained
momentum. The project did not suffer from any counterpart funding problems, as GOP took timely
corrective measures and made appropriate budget provisions. The overall assessment of government
performance is therefore satisfactory.

7.6 Implementing Agency:

Satisfactory.

PAD: The performance of PAD, which coordinated the project, also varied during the course of
the project. PAD was fairly enthusiastic in the very early years, especially during the design phase of the
CoA and the NAM. Difficulties surfaced after the Auditor General who negotiated the project retired, and
the incoming Auditor General, who initially said it would take him time to form a view on the project,
announced that he thought the separation of responsibilities to be unconstitutional. Implementation became
very slow for the next three years. Frequent changes of the project director also impeded project progress.
The project picked up momentum after the new government came to power in 1999, which resolved many
of the institutional issues. In 2002 a very competent professional was appointed to the position of project
director and he stayed in place until its closing. The new AG who took charge in July, 2002 provided good
leadership and direction to the reforms. This had a very positive impact on project progress. On an overall
basis the performance is rated satisfactory since most project objectives were achieved.

PIFRA Directorate: Exept for the high turnover of Project Directors, the performance of the
PIFRA Directorate was satisfactory. PAD assembled a core group of 20 committed young officers to
form the PIFRA Directorate, who became key drivers for project implementation. These members were
drawn from the officers and staff manning the accounting functions. Some members of this group were
very conversant with existing business processes and the functional requirements of the system as seen
from a user standpoint. They served as subject area specialists along with the vendor team responsible for
configuring the SAP/R3 system so that it would be able to meet functional requirements. They were also
involved in initial pilot deployments and testing of the system. The Directorate conducted proper
monitoring and evaluation activities as per the Credit Agreement and agreed indicators, and covenants were
mostly complied with. The team was receptive to the Bank’s advice, and collaborated with the Bank’s task
team in implementation.

The PIFRA Directorate operated in close coordination with the senior management of PAD.
Quality of financial management and procurement administration was moderately satisfactory. There were
were delays in procurement because of major issues in terms of selecting consultants for designing IT
systems. Submission of audit reports and response to some of the audit queries were delayed in a few
instances. The disbursement knowledge of the PIFRA staff and the quality of withdrawal applications
remained poor which resulted in ineligible expenses being claimed as expenditures from the Credit. About
US$319,937 were recovered from PIFRA against ineligible withdrawals.

7.7 Overall Borrower performance:

Satisfactory. The overall performance of the Borrower was satisfactory.

- 27 -
8. Lessons Learned
Lack of Commitment and Active Support from the MOF: Projects such as PIFRA should be framed
primarily as initiatives designed for reform of the expenditure management systems rather than
merely accounting systems reform. If reform measures are framed in terms of resolution of an
accounting problem (such as non-conformity with international accounting standards, use of an archaic
chart of accounts, lack of timely reporting etc.), the danger is that it will be focused only on the reform
of the organization and processes related to the accounting function. These are areas where it is
difficult to get active involvement of senior policy makers in the MOF. Framing the project primarily
as an expenditure management reform initiative raised the importance of the problem to a level where
senior level policy makers in the country and their counterparts in donor organizations could relate to
it. Linkages were established between the PIFRA program and requirements under policy based lending
from both the Bank and IMF, and the MOF started taking a more active interest in ensuring that
PIFRA milestones were met.

Scope of the system: Extending the central system to sub-national levels: If the central government
is the sponsor of the project, then the project should be restricted to central government payments
and receipts. If the central government wants to extend its control to provincial level payments then it
should have some financial leverage and control over provincial finances. In Pakistan, over 80% of
the provincial finances come from the Center. Under these circumstances, the MOF does have some
justification in insisting on the implementation of a central system. However, implementation of a
common system for central and provincial payments should only mean that a common set of standards
and controls are applied to both sets of transactions in terms of whether the payment request has
budgetary and cash cover. These controls would be part of any budgetary control system, be it central
or provincial. Particular care should be taken that the agency exercising the payment control function
does not exceed these limits. Specifically, use of the central system for provincial payments should not
result in any additional interference from the center. Reluctance to use the central system would be
further reduced if the system is designed in consultation with the provincial governments and ensures
that it can provide useful and timely information to the provinces for management of their provincial
resources.

Lack of adequate project management capacity:

l Implementing a new, automated government wide accounting system is a very complex

undertaking that requires successful and coordinated implementation of a myriad set of
activities. These projects require a core group of committed professionals who are trained in the
implementation of the new system and would act as change agents as the system is rolled out
across the country. They also need a project management structure with clear lines of authority.
l Projects such as PIFRA need to be headed by a competent public sector senior manager who
enjoys the respect of his/her colleagues and superiors in the government service system. He/she
should have full financial and administrative authority to make project related decisions and should
not be hampered by bureaucratic interference from other departments. He/she should be assisted
by a fully staffed project secretariat and staff who are expert in formal project and contract
management techniques and use these techniques to monitor project progress.
l For projects which involve the installation of complex country wide networks of computer systems,
the project design should ensure that the project secretariat has specialist advice on complex IT
procurements and project installation that both the client and the Bank can accept as independent
of the suppliers and vested interests.

- 28 -
l Since the project spans a variety of agencies, representatives of these agencies need to be
represented on the project management team and need to be actively involved in managing the
interface with their respective agencies.
l During the testing and pilot stages of the project, the new system will be managed by the project
implementation unit. However, as the new system is implemented across the country, the day to
day operation of the system will need to be handed over to the departments/offices that are
routinely responsible for operation of the accounting system. An effective transition strategy
needs to be worked out for this which ensures that the departments receiving the new system are
fully trained to operate and maintain it.
l It is best to get agreement with Government on these issues prior to the finalization of the
Loan/Credit agreement.

Resistance to change

l Adoption of the new accounting model and new budget classification structure, and other such
changes require the full support of the MOF policy makers. They should be convinced that these
changes are beneficial and necessary and be willing to carry through their implementation,
despite opposition from vested interests.
l Better communication with the affected staff could assure them that their fears are unfounded.
This could be done by citing examples where similar changes have been successfully implemented
without creating the sort of displacement or reduction in career prospects that is feared.
l Resistance to change may be reduced if the change agents would be assured that similar changes
have been implemented in this manner successfully in other cases so that they are more confident
of achieving a successful outcome. This would also need to be supplemented by putting in place
some incentives for successfully implementing the change. The Bank could help by providing
opportunities and arranging for the project team to present their experiences to international
audiences at the Bank and/or other countries engaged in similar projects. The government could
institute incentives such as special pay and bonuses for staff engaged in these exercises.

Orientation and Training

Successful implementation and operation of a country wide accounting systems project such as that
envisaged in PIFRA requires that:

l a large number of stakeholders in various departments and agencies become familiar with
project scope, methodology and above all, its advantages, i.e. what it would be able to do for
them.
l a very large number of staff be trained in the use of the new systems. These include finance
managers and a few relatively senior staff in the overall concepts of the new system; staff who
would be doing transaction processing and report production on a day to day basis, end users
in other departments agencies who would need to be trained in the use of new input documents
to be used with the new system, and staff who would be responsible for operating and
maintaining the system.
l most staff need to know only the specific features of the system that they would be using. This
has been found relatively easy to accomplish even for complicated systems such as SAP. Only

- 29 -
 a small number of staff need to have a comprehensive knowledge about the whole system
functionality.
l as with any new system, help desk facilities are essential to provide a source of ready
assistance to staff in the resolution of their problems on a day to day basis. Segmented in this
fashion, the training effort becomes more manageable.

Technical Staffing: The lack of career prospects for technical IT staff within the government has been
a major area of concern for the sustainability for the project. In order to remedy this situation, they
should be given sufficient incentives to remain in the government. It is necessary that this issue be
tackled at the design stage as conditionality prior to project implementation.

Importance of getting more emphasis on change management and human side of reforms: The
project should have put more emphasis on change management as a separate activity within the overall
ambit of the project. The SAR had only a one line mention on change management and no details were
given.

Policy changes such as separation of auditing and accounts, may be accomplished more easily
under the umbrella of the overall country poliy dialogue and by linkages with other possible
instruments such as budgetary support loans (e.g. DPL). Establishing such linkages has been has been
similarly successful in several operations for the implementation of Treasury systems in countries in
Eastern Europe and Central Asia region.

All organizations who are stakeholders in a program component should be involved in dialogue
and should be entrusted with the responsibility of implementing the program. As long as stakeholders
do not own a program by fully participating in all aspects of the program, the desired objective of the
program cannot be achieved and would often result in creating tension between different government
departments.
Complexity of project design: Though project activities reflected essential reform priorities, in practice
the project design proved to be too complex and expectations about implementation capacity and the
will of the Government to improve this capacity by undertaking actions agreed upon during
preperation, proved optimistic. This led to difficulties in implementation. In hindsight, it is felt that
perhaps too many reform activities were bundled into a single project. Though the SAR did state
explicitly that the overall reform would take 10-15 years or more, perhaps the first tranche of the
reform effort could have been further sliced to first concentrate on repairing the accounting system and
then addressing larger institutional reform issues.

Time required for Institutional building projects. The substantial time required for the introduction
of changes in legal framework, organization and functional processes, systems, and procedures across
multiple levels of government and agencies needs to be recognized while designing similar projects. A
recent study in ECA of Treasury systems in 13 countries has shown that the average time taken for
setting up a basic Treasury system (similar to the government accounting system under the Government
accounting component in PIFRA) was between seven to nine years.

The Bank's continued support for PIFRA in difficult times. The Bank recognized the criticality of
the reform measures envisaged under the project to the Government's development perspective and
continued to support the project through its dark years even though under normal circumstances the
project may have been declared non- performing and closed. The task team was allowed the additional

- 30 -
 time to try alternative approaches to assure government commitment, such as establishing linkages
with adjustment operations and much more intensive supervision with key technical inputs. However,
strict deadlines were established for completion of key milestones. These measures were eventually
successful. The Bank's flexibility in the application of operations policy eventually resulted in
satisfactory completion.

9. Partner Comments
(a) Borrower/implementing agency:

See Annex 8 (Borrower report attached).

(b) Cofinanciers:

Not applicable.
(c) Other partners (NGOs/private sector):

Not applicable.

10. Additional Information

A. The Bank’s ICR Team consisted of the following members:

P.K. Subramanian Task Team Leader

Ismail Ceesay Co- Task Team Leader
Anthony Graeme Lee Senior Municipal Financial Specialist
Hasan Saqib Senior Financial Management Specialist
Asif Ali Procurement Specialist
Riaz Mahmood Disbursement Analyst
Ali Hashim Consultant
Sati Achath Consultant
Michael Jacobs Consultant
Riyaz H. Bokhari Consultant
Altaf Ahmad Program Assistant
Marietta Visaya Program Assistant

B. List of Task Team Leaders of the project in chronological order:

(i) Anthony Graeme Lee

(ii) P.K. Subramanian

- 31 -
Annex 1. Key Performance Indicators/Log Frame Matrix
Outcome / Impact Indicators:
Indicator/Matrix
(a) Revised accounting standards adopted by
May, 1997.
(b) Pilot implementation of core financial
systems completed by December 1999;
(c) Province-wide replication of core financial
systems completed by December 2001;
(d) Identification of public/private audit
cooperation opportunities by December 1997;
(e) Introduction of Accounting Technicians
and 2 year Probationers' specialised training
by April 1998 and January 1999 respectively;
and
(f) Approval of proposals for functional
separation of accounting and audit by
December 1999.
(a) Improved financial reports, including
budget variance and project costs;
(b) Percentage of transactions automated, by
value;
(c) Cash-flow forecasts introduced and
efficiency of borrowing improved;
(d) PAD staff's job rotation with 3 years
reduced;
(e) More timely reports to Public Accounts
Committee.
(f) Improved training impact evaluation
results.
Output Indicators:
Indicator/Matrix
Separation of budget of CGA and OAG by
1998
Implementation of new computerized system
at 21 sites by Dec., 2001
Operations manuals prepared for internal
audit and certification audit by Dec. 1998
1
Projected in last PSR Actual/Latest Estimate
Adopted in November, 2000
Pilots completed in March 2005
Replication plan changed and the current
estimated completion date is June 2006.
Collaboration in field audit was tested during
live audit at 7 sites done in 2005
Substitution of SAS with PIPFA under active
consideration. New syllabus for SAS in
compliance with PIPFA prepared and
accepted. Probationer’s course of 2 years not
introduced yet.
Approval was done by the President in 2001
through the promulgation of CGA and AGP
Ordinance 2001.
Functionality provided to all the productive
sites. The productive use of the functionality
fully made at DAO Faisalabad. Use at other
productive sites being rolled out gradually.
An estimated 40 % of national transactions
automated.
Will be possible as more sites are rolled out.
Principle of job rotation after 3 years in place.
However, owing to administrative
contingencies, exceptions are made.
Improvement in timeliness and quality of audit
observations achieved. Number of audit
observations restricted to only high value,
high risk areas.
Training impact evaluations forms developed
by the consultants will be used for monitoring
quality of training.
1
Projected in last PSR Actual/Latest Estimate
Separated in 2001
Implemented at 30 sites by May, 2005
Preparations of all manuals completed by
2004.
- 32 -
Purchase and installation of audit software by Completed in 2004.
Sep. 1999

Approval of incentive pay and salaray Approved in 2005

supplements to project staff

Training materials developed and Training of Completed in 2004

Trainers completed
1
End of project

- 33 -
Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)

Appraisal Actual/Latest Percentage of
Estimate Estimate Appraisal
Component US$ million US$ million
Component 1: Government Accounting and Financial 13.33 19.24 144
Reporting
Component II: Government Auditing 7.26 2.95 41
Component III: Human Resource Management 1.39 2.14 154
Component IV: Training 6.52 2.83 43
Component V: Administration 6.82 5.51 81
Component VI: Other Policy Support for Economic 1.88 1.35 72
Management
Total Baseline Cost 37.20 34.02
Physical Contingencies 0.00
Price Contingencies 0.00
Total Project Costs 37.20 34.02
Total Financing Required 37.20 34.02

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

1
Procurement Method
Expenditure Category ICB 2 N.B.F. Total Cost
NCB Other
1. Works 0.00 0.90 0.00 0.00 0.90
(0.00) (0.80) (0.00) (0.00) (0.80)
2. Goods 6.90 2.80 0.30 0.00 10.00
(5.70) (2.30) (0.20) (0.00) (8.20)
3. Services 0.00 0.00 5.90 0.00 5.90
(0.00) (0.00) (5.90) (0.00) (5.90)
4. Technical Assistance 0.00 0.00 13.10 0.00 13.10
(Implementation (0.00) (0.00) (13.10) (0.00) (13.10)
Assistance, Institutional
Developmenet, Training
and Policy Support)
5. Incremental Staff Costs 0.00 0.00 6.50 6.50
and Operating Costs (0.00) (0.00) () (0.00) (0.00)
6. Refinancing PPF 0.00 0.00 0.80 0.00 0.80
(0.00) (0.00) (0.80) (0.00) (0.80)
Total 6.90 3.70 20.10 6.50 37.20
(5.70) (3.10) (20.00) (0.00) (28.80)

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

1
Procurement Method
Expenditure Category ICB N.B.F. Total Cost
NCB

- 34 -
 2
Other
1. Works 0.00 1.25 0.00 0.00 1.25
(0.00) (1.12) (0.00) (0.00) (1.12)
2. Goods 12.66 0.92 0.93 0.00 14.51
(12.66) (0.64) (0.65) (0.00) (13.95)
3. Services 0.00 0.00 8.29 0.00 8.29
(0.00) (0.00) (8.29) (0.00) (8.29)
4. Technical Assistance 0.00 0.00 5.64 0.00 5.64
(Implementation (0.00) (0.00) (4.36) (0.00) (4.36)
Assistance, Institutional
Developmenet, Training
and Policy Support)
5. Incremental Staff Costs 0.00 0.00 4.05 0.00 4.05
and Operating Costs (0.00) (0.00) (0.00) (0.00) (0.00)
6. Refinancing PPF 0.00 0.00 0.28 0.00 0.28
(0.00) (0.00) (0.28) (0.00) (0.28)
Total 12.66 2.17 19.19 0.00 34.02
(12.66) (1.76) (13.58) (0.00) (28.00)

1/
Figures in parenthesis are the amounts to be financed by the IDA Credit. All costs include contingencies.
2/
Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff
of the project management office, training, technical assistance services, and incremental operating costs related to (i)
managing the project, and (ii) re-lending project funds to local government units.

Project Financing by Component (in US$ million equivalent)

Percentage of Appraisal
Component Appraisal Estimate Actual/Latest Estimate
IDA Govt. CoF. IDA Govt. CoF. IDA Govt. CoF.
Component 1: Government 11.23 2.10 17.58 1.66 156.5 79.0
Accounting and Financial
Reporting
Component II: 6.43 0.83 2.92 0.03 45.4 3.6
Government Auditing

Component III: Human 1.17 0.22 2.10 0.04 179.5 18.2

Resource Management

Component IV: Training 4.00 2.52 2.65 0.18 66.3 7.1

Component V: 4.12 2.70 1.46 4.05 35.4 150.0

Administration

Component VI: Other 1.88 0.00 1.29 0.06 68.6 0.0

Policy Support for
Economic Management

TOTAL 28.83 8.37 28.00 6.02 97.1 71.9

- 35 -
Annex 3. Economic Costs and Benefits
Cost Benefit Analysis
(Indicate currency, units and base year)

Present Value of Flows

Economic Analysis Financial Analysis
Appraisal Latest Estimates Appraisal Latest Estimates

N/A
Benefits
Costs

Net Benefits
IRR/NPV

- 36 -
Annex 4. Bank Inputs
(a) Missions:
Stage of Project Cycle No. of Persons and Specialty Performance Rating
(e.g. 2 Economists, 1 FMS, etc.) Implementation Development
Month/Year Count Specialty Progress Objective

Supervision

03/08/1997 3 PROJECT ADVISOR, RMP (1); S S

MISSION LEADER (1);
INSTITUTIONAL DEVEL. (1)
05/12/1997 4 PROJECT ADVISOR, RMP (1); S S
TASK MANAGER (1);
INSTITUTIONAL DEVELOP.
(1); AUDIT (1)
10/02/1997 6 TEAM LEADER (1); PROJECT S S
ADVISOR (1);
INSTITUTIONAL DEV. (1);
ACCOUNTING (1); SYSTEMS
DEV. (1); LACI (1)
11/21/1998 5 SYS DEV/MISSION LEADER S S
(1); IMF (1); DISBUREMENTS
(1); AUDIT/TRAINING (1);
ACCOUNTING (1)
08/27/1999 7 TEAM LEADER (FMS) (1); S S
CO-TEAM LEADER (FMS) (1);
INFORMATICS (1);
DISBURSEMENT (1);
PROCUREMENT (1);
AUDITING/TRAINING (1);
FIN.MGMT./COMP.SYSTEMS
(1)
02/18/2000 7 TASK TEAM LEADER (1); S U
CO-TASK TEAM LEADER (1);
INFORMATICS (1);
ECONOMIST (1);
DISBURSEMENT (1);
PROCUREMENT (1);
ACCOUNTING & AUDIT (1)
05/30/2000 1 TASK TEAM LEADER (1) S U
09/13/2001 5 TASK TEAM LEADER (1); S S
CO-TASK TEAM LEADER (1);
FINANCIAL MANAGEMENT
(1); DISBURSEMENTS (1);
PROCUREMENT (1)
02/09/2002 7 TASK MANAGER (1); S S
CO-TASK MANAGER (1);
INFORMATICS SPECIALIST
(1); FIN.MGT. SPECIALIST (1);
DISBURSEMENTS (1);
PROCUREMENT SPECIALIST
(1); ACCOUNTABILITY

- 37 -
 SPLST. (1)
09/19/2002 8 TASK LEADER (1); CO-TASK S S
LEADER (1); INFORMATICS
SPECIALIST (1); TASK
LEADER-PIFRA 2 (1);
PROCUREMENT SPECIALIST
(1); FIN.MGT.SPECIALIST (1);
DISB. SPECIALIST (1); DFID
REPRESENTATIVE (1)
05/30/2003 7 Mission Leader (1); FM Team S S
leader (1); Co-Task Leader (1);
Audit Specialist (1); FMS (1);
Disbursement Splst. (1);
Procu.Splst. (1)
10/18/2003 7 TASK TEAM LEADER (1); S S
INFORMATICS (1); CO-TTL
(1); AUDITING (1); TRAINING
(1); TTL, PIFRA-2 (1); FMS (1)
05/22/2004 8 Team Leader (1); TTL-PIFRA-2 S S
(1); Informatics (1); Co-Team
Leader (1); Auditing (1);
Training (1); FMS (1); Team
Support (1)
11/23/2004 8 Team Leader (1); TTL-PIFRA-2 S S
(1); Co-Team Leader (1); Audit
Expert (1); Informatics Specialist
(1); Training Expert (1); FMS
(1); Disbursement (1)

ICR
04/20/2005 9 Team Leader (1); Co-Team S S
Leader (1); Audit Expert
(1); Informatics Specialist
(1); Training Expert (1);
FMS (1); Disbursement (1);
Procurement (1); Tean
Support (1)

(b) Staff:

Stage of Project Cycle Actual/Latest Estimate

No. Staff weeks US$ ('000)
Supervision 49.4 744.0
ICR 3.0 9.7
Total 116.6 965.8
Staff inputs -- also includes Bank-financed and TF consultants.

- 38 -
Annex 5. Ratings for Achievement of Objectives/Outputs of Components
(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)
Rating
Macro policies H SU M N NA
Sector Policies H SU M N NA
Physical H SU M N NA
Financial H SU M N NA
Institutional Development H SU M N NA
Environmental H SU M N NA

Social
Poverty Reduction H SU M N NA
Gender H SU M N NA
Other (Please specify) H SU M N NA
Private sector development H SU M N NA
Public sector management H SU M N NA
Other (Please specify) H SU M N NA
Ratings for achievement of objectives (relates to the objectives specified in the SAR. Outputs by
components (relates to the outputs specified in the PAD).

- 39 -
Annex 6. Ratings of Bank and Borrower Performance
(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HU
Supervision HS S U HU
Overall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HU
Government implementation performance HS S U HU
Implementation agency performance HS S U HU
Overall HS S U HU

- 40 -
Annex 7. List of Supporting Documents
1. Aide Memoires, Back-to-Office Reports, and Project Status Reports.

2. Project Progress Reports.

3. Consultant Study Reports financed under the Project.

4. Borrower's Evaluation Report dated October 14, 2005; and

5. Staff Appraisal Report for Pakistan: ‘IMPROVEMENT TO FINANCIAL REPORTING AND

AUDITING PROJECT’, dated August 5, 1996 (Report No. 14890-PAK)

- 41 -
Additional Annex 8. Implementation Completion Report prepared by PIFRA Directorate

PIFRA Directorate, Islamabad

No. FABS/EAD October 14, 2005

Maria Ejaz
Section Officer (WB)
EAD
Islamabad.

Subject: PIFRA Implementation Completion Report

Reference: Your letter dated September 2, 2005 on the subject noted above.

The Implementation Completion Report prepared by PIFRA is enclosed as requested.

Sd/-

(KHURAM FAROOQ)
DIRECTOR FABS

Cc:
• PD PIFRA
• Mr. Ismaila B. Ceesay, Team Leader (Financial Management), World Bank, Islamabad.

- 42 -
 Basic Data

Project Start Date 1996

Project End Date May 31, 2005

Number of extensions 2

No of World Bank Missions 12

Total Cost $ 37 m ($ 28 m IDA Funding)

Planned Actual Variance

Cost $ 37

Time 5 years 9 years 4 years

Total number of
project personnel

Total number of 2 15 13
operational
vehicles

Total number of 21 in SAR/24 in 30 + 9 as per SAR

FABS sites PC 1 and + 6 as per
implemented PC 1

Introduction

PIFRA was started in 1996 when, as a result of a Diagnostic Consultancy Study, financed under
the Third Technical Assistance Credit (Cr. 1755-PAK) , to review the government’s fiscal reporting
system and the institutional arrangements for accounting and auditing, it was assessed that the absence of
accurate, comprehensive and timely financial information in Pakistan's federal and provincial governments
hampered efforts to improve resource allocation, monitoring and control of expenditure, cash flow and debt
management, and casts uncertainty on the financial data underpinning public sector management decisions
and the macro economic dialogue.

One of the key dimensions of Good Governance, Accountability, was being undermined due to
inadequate internal controls, weaknesses in auditing in terms of professional capacity and use of modern
methodologies and tools. The Human Resource practices of PAD needed intervention to meet the ever
growing challenge of introducing professionalism at the institutional level to carry out its mandate

- 43 -
effectively.

The institutional arrangements for carrying out audit and preparing the accounts by PAD were not
conducive to ensuring independence of audit.

With this background, Government initiated PIFRA project with the vision to

l ‘adopt and implement a modem accounting system designed according to recognized accounting
principles and standards and based on modem information technology’;
l ‘implement a governance structure and legal framework for an independent and effective audit
function’;
l ‘improve the professional capacity of the elements of the civil service responsible for fiscal
management’;
l ‘make increasing use of the private sector to supplement public sector resources’; and
l ‘adopt improved standards for private sector financial disclosure’.
In order to help Government achieve this vision, PIFRA project was launched with the assistance
of the World Bank to support the Government’s vision as stated above.
Evaluation of Design and Implementation

1. Design and Formulation

The project design, comprising four key components (FABS, Audit, Training, HRM/Change
Management), was in great measure sound and relevant.

Two key assumptions made in the design still hold true ten years down the road:

l It will take a sequence of projects to achieve the vision of modern fiscal reporting and
management system, improved auditing procedures grounded in best practices and modern HR
function of PAD
l It will take 10-15 years to achieve the true improvements in a country-wide fiscal management
system.
FABS was by far the biggest component, attracted most of the attention and achieved
comparatively more progress as was envisaged in the project design.
Audit component envisioned formulating methodologies, guidelines and implementation of these
methodologies through an IT system.

Process re-engineering in terms of accounting and auditing and the implementation of these
re-engineered processes posed enormous challenges in terms of HRM and Training. Realizing the
magnitude of these challenges, these two support functions merited formulation as separate
components in the design of the project. Experience has proved the appropriateness of the design in
this respect.

In addition, key Performance Indicators and Outcomes were clearly defined in the project design as
per best practices.

However, some areas of the design that did not stand the test of time and reality merit discussion:

- 44 -
 l Project Management Structure: The proposed arrangement that PAD would implement most of the
components in-house through hiring of technical resources and quality review of systems done by
international experts at least annually was based on the assumption that PAD would have project
management capacity to manage these technical resources with the assistance of GM MIS. For this
purpose, a software development centre was conceived in the project design to be set up in PAD.
This assumption was more optimistic assessment of the project management capacity of PAD and
did not stand the test of reality. Although, this methodology has been adopted very successfully in
the private sector for some large international projects with enormous benefits of economy,
efficiency and ownership, this arrangement had to be drastically reviewed. PAD did not have
experience of implementing project of this size and complexity nor did it have adequate skills and
management resources required for this undertaking. Although GM MIS was conceived to
complement PAD project management capacity, eventual responsibility for overseeing the delivery
of outputs rested on PAD project management which for quite a long time struggled with finding
the right team led by Project Director with the qualities of leadership, integrity, communication,
persuasion, negotiation and willingness to take calculated risks. No wonder, 13 Project Directors
were changed during the lifecycle of the project.

Eventually, major implementations of FABS and Audit had to be outsourced to external

contractors/consultants. PAD managed to weave together a winning team and develop capacity
only very late (seven years after the start of the project) when the project was brought back from
the brink of being declared a ‘sick’ project and abandoned.

Not withstanding these weaknesses, the proposed arrangements for implementation through a core
group of end users at each individual site was one of the most sound judgments in the project
design. The end-user support group would be trained by the technical team and would thereafter be
responsible for training all other personnel at the AG offices as well as at DAOs. The entire
implementation of FABS followed this model and is considered key to the success.

l Technology Architecture: Detailed distributed technology architecture was proposed in the design
of the project However, later developments in technology necessitated revision in the technology
architecture to make it centralized at the provincial level. Owing to rapid developments in
technology, the details of technology should be avoided in future project designs. A more business
focused abstract outline of technology and architecture seems better suited to stand the test of time.

l HRM: ‘Developing a system of quantifiable and transparent evaluation and evolving an open,
merit-based, promotion and reward system’ seems an unachievable objective, given the enormous
constraints of culture in general and civil service structure in particular. The project did not
achieve its objective on this count as was expected. Even early steps could not be taken in this
direction.
Key stakeholders like Ministry of Finance, Provincial Finance Departments, Auditor General of
Pakistan, Federal Public Service Commission, Establishment Division were consulted in the formulation
process.

2. Project Outputs

The project outputs were clearly mentioned in the design of the project and a comparison of

- 45 -
planned (appraised) vs Actual outputs is given below.

Planned Actual
(a) Improved financial reports, including Functionality provided to all the productive
budget variance and project costs; sites. The productive use of the functionality
fully made at DAO Faisal Abad. Planned use at
other productive sites in July-August 2005.
(b) Percentage of transactions automated, An estimated 40 % of national transactions
by value; automated.
(c) Cash-flow forecasts introduced and Not completed yet
efficiency of borrowing improved;
(d) PAD staff's job rotation with 3 years Principle of job rotation after 3 years in place.
reduced; However, owing to administrative
contingencies, exceptions are made.
(e) More timely reports to Public Improvement in timeliness and quality of audit
Accounts Committee; observations achieved. Number of audit
observations restricted to only high value, high
risk areas.
(f) Improved training impact evaluation Consultants developed training impact
results. evaluations forms which will be used for quality
monitoring of training.

In retrospect, one important aspect, the transition strategy, should have been kept in view while
formulating the outputs and merited more explicit and elaborate discussion. While 21 sites were planned to
be made productive as per the design, most of the sites would still have been working on manual or legacy
systems. The AG offices that aggregate the summary reports from these offices would have needed reports
on the new classification from the legacy sites in order to achieve a structured roll-up of information at the
federal and provincial level as per the new classification structure. Till that data was made available in the
required format, an extended transition strategy was needed to be worked out to achieve the national and
provincial level reports on the new formats. This strategy was missing in the design. The output that new
cash forecasting reports would be used that would improve the efficiency of borrowing was not possible for
the transition period while majority of the sites were still working on the legacy. Even design consultants of
NAM missed it their documentation, which later led to difficulties in producing these reports and meeting
stakeholder expectations.

3. Key Performance Indicators

A comparison of Performance Indicators and their actual status at the Project Close is given below.
Most of the indicators have been achieved. However, the delay in achieving these indicators is glaring and
has been explained later.

Indicator Actual/Latest Estimate

(a) Revised accounting standards adopted by The revised accounting standard was
November 1998; adopted by 2000

(b) Pilot implementation of core financial March 2005

systems completed by December 1999;

- 46 -
 (c) Province-wide replication of core financial Planned completion date is June 2006
systems completed by December 2001;
(d) Identification of public/private audit Paper on collaboration with Private Sector
cooperation opportunities by December 1997; Auditing Firm, one of the deliverables, was
approved by AGP in 2004. Collaboration in
field audit was tested during live audit at 7
sites in which the actual audits were done in
2005 by field auditors of AGP and
supervised by private sector firm
(e) Introduction of Accounting Technicians Substitution of SAS with PIPFA under active
and 2 year Probationers' specialised training by consideration. New syllabus for SAS in
April 1998 and January 1999 respectively; and compliance with PIPFA prepared and
accepted. Probationer’s course of 2 years not
introduced yet.
(f) Approval of proposals for functional Approval was done by the President in 2001
separation of accounting and audit by through the promulgation of CGA and AGP
December 1999. Ordinance 2001.

4. Project Costs

A comparison of the planned and actual cost explains that the initial estimates were gravely upset
by the actual response to the bids. For example, the estimates for Hardware ICB for Roll Out sites as
prepared by the Consultants were around $ 12 m whereas the lowest bid received and eventually awarded
was for $ 4.5 m !. A cost comparison of planned and actual is given below.

5. Disbursements

The disbursements were made according to agreed arrangements with the World Bank. The
year-wise comparison of disbursements reflecting the great acceleration in the implementation after 2003 is
self-evident from the graph given below (The figures are up Sep 5, 2005)

Disbursements in PIFRA 1
7,000,000
6,000,000
5,000,000
4,000,000
3,000,000 USD
2,000,000
1,000,000
0
19 98

19 99

20 03

20 04

5
19 97
19 96

20 01

20 02
00

00
9

0
9
9

0
-1

-1

-2

-2

-2

-2
-1
-1

-2

-2
97

98

99

02

03

04
96
95

00

00
19

20

- 47 -
 Disbursements in Pak Rs

450,000,000

400,000,000

350,000,000

300,000,000

250,000,000

PKR
200,000,000

150,000,000

100,000,000

50,000,000

0
1995-1996 1996-1997 1997-1998 1998-1999 1999-2000 2000-2001 2000-2002 2002-2003 2003-2004 2004-2005

6. Project Schedule

The project had an overall delay of 5 years owing to various reasons. The project implementation
pace has been intermittent; satisfactory during the design of the system, very slow during validation of
design, slow during implementation in the early part and comparatively very fast during the last three years.
First test and pilot site, DAO Abbot Abad, could be made productive in 18 months

- 48 -
 These reasons are analyzed in detail below.

l Abstract nature of the project: It may be appreciated that PIFRA is introducing deep, wide ranging
and comprehensive reforms in the country’s archaic financial management system. Unlike any brick
and mortar project, the outcome of the project is abstract (better information for decision makers). As
in any software project, it was extremely difficult in elucidating user requirements upfront during the
requirement analysis phase. This difficulty could have been avoided by employing sound system
analysis and design techniques using modeling tools. Unfortunately, the consultants did not apply the
rigor this activity needed in order to document these requirements at the level of detail that makes
configuration and implementation a relatively straightforward task. Therefore, requirement analysis
had to be done well beyond the design phase during the prototyping and first pilot stage. Similarly, due
to abstract nature of requirements, users constantly kept changing requirements even after formal sign
off of Operational Acceptance Testing. They would only know the real impact of the configuration in
the live environment and would come back several times for enhancements, new user requirements or
improvements in the layout and formatting of reports and screen, consuming more resources and time
and upsetting the timelines. Scope disputes with both Audit and HRM consultants is another testimony
to this fact. Delays in user validation of the NAM design (it took 2 years) is yet another case in point.
Even highest decision makers take their own time to understand the issues and make project decisions
due to the abstract nature of the issues
l Context: It is difficult to implement projects of this nature at such a wide scale in such an antiquated
setting. Limited capacity to appreciate the criticality and urgency of making project related decisions
further contributed to the delays. It took 2 years to finalize the ICB and 7 months to finalize the
procurement arrangements for site preparation design consultants. Besides capacity, a relatively high
tolerance for delays without invoking accountability has been another factor important in this context.

l Restructuring and Ordinance: The project involved separation of audit from accounts, a contentious
administrative and policy issue with the potential of fundamental changes in the structure of the audit
and accounts services in Pakistan that involved promulgation of two separate ordinances (Auditor
General Of Pakistan and Controller General of Accounts Ordinance 2001), the delay in getting
consensus and working out the modalities by consulting all the stakeholders consumed considerable
time and resulted in eventual delay in the project.
l Change Management: Resistance to incorporate massive functional and institutional change in such a
cutting edge project is to be expected.
l Scope Changes: Constant change of goal posts became additional source of difficulties and delays.
These changes were mainly in overall scope of sites and functional changes due to either changes in
government policies or user resistance to accept the designed systems. Some key highlights are given
below:
l The PC 1 was approved for implementing 24 sites, Staff Appraisal Report mentioned 21 sites;
eventual scope was increased to implementing 52 sites.
l Change in government structure also forced us to re-align our implementation strategy. There were
two government types (federal and provincial) in the project design, this structure has been changed
after the devolution and we have 5 governments now (federal, provincial, district, Tehsil, union
council); we are implementing district government through an enhancement in the scope of the
project.
l Federal Government and almost all provincial government structures have changed during the

- 49 -
 implementation, leading to merger or bifurcation of various ministries and departments with scope
impacts in the implementation.
l Rapid advancements in technology have necessitated a major design change from decentralized to
centralized technology architecture. This led to further delays in the ICB procurement of new
hardware suitable for this architecture.
l GFS Standards underwent a revision in 2001 due to which the entire approved new Chart of
Accounts had to be changed in order to make it GFS 2001 compliant.
l MOF came up with 106 additional business requirements and 150 reports that were implemented in
the system as design changes.
l Site Preparation: Site preparation was one of the key activities of the project. This involved work
which was of peculiar nature. It was not big enough to attract big contractors who didn’t have liquidity
problems. However, small contractors took the job only to find out later that they had severe liquidity
problems due to which they could not deliver on time.
l Capacity and retention: Our offices had limited capacity in project management. The officials who
had the requisite skills, knowledge and experience did not have motivation to stay in the project as
project work involves sheer hard-work without any substantial incentives.
7. Implementation Arrangements

Detailed comments on the implementation arrangements of FABS have been provided in the design
and formulation part of this document.

For Audit component, there is a view that implementation through one big consultancy was a major
risk area. Dispute in one area would hamper progress on all other deliverables. The consultancy should have
been broken down into smaller engagements like Development of Financial Audit Manual and Internal Audit
Manual and Selection of software could have been better managed through three consultants.

Change Management workshop and Seminar part should have been implemented through hiring
individual consultant or in-house effort, as was actually done in the later part of the project.

8. Consultant Recruitment and Procurement

All consultants’ recruitment and procurement was strictly done according to World Bank
guidelines with active consultation of World Bank Missions and Resident Management. A brief on their
procurement mode and comparison of planned vs actual costs is given below.

No Consultants/Contractor Procurement Planned Cost Actual Cost

Mode

1 Design consultants for ICB $ 3.7 m $6m

Accounting standards and
implementation contractors

2 Local consultants for coding $ 1.7 m $7m

of accounting application

- 50 -
 3 Audit Consultants ICB $ 1.5 m

4 HRM/CM Consultants ICB $ .3 m

5 Training Consultants ICB $ .6 m

6 Design Consultants for Site NCB $

Preparation

7 Hardware for ICB ICB $ 4.5 m

9. Performance of Consultants, Contractors, and Suppliers

Seven main contracts, as highlighted above, involving Consultants, Contractors and Suppliers were
executed during the project, in addition to small contracts for the supply of hardware and construction of
sites.

Most the consultants/contractors/suppliers had disputes with the Project on various issues. Some of
key reasons of these disputes are analyzed below:

l Inexperience of consultant teams to understand the scope and complexity of the engagement and
give adequate estimates of efforts and costs. When they face the complexity of the real situation,
scope disputes arise.
l Eagerness to win contracts in anticipation that the scope issues will be managed through
negotiation with the management during contract execution in favor of consultants.
l Inadequate formulation of Terms of Reference to clearly delineate the scope in sufficient detail.

A summary of these disputes is given below.

No Consultants/Contractor Nature of Key areas

dispute

1 Design consultants for Scope disputes l Deficient deliverables in quality

Accounting standards and l Not all deliverables of TOR
implementation contractors completed.

2 Implementation contractors Scope dispute l SAP Licenses

for implementation of l Hiring of key personnel
accounting application Contract l Server prices
interpretation l Scope of support/design changes

3 Audit Consultants Scope dispute l Consultants understanding of

Financial Audit manual did not
reflect the requirements of AGP.
4 HRM/CM Consultants Quality of l Lack of quality
services l Inconsistency of consultant team

- 51 -
 5 Training Consultants Nil Nil

6 Design Consultants for Site Quality of l Some Personnel did not perform
Preparation services satisfactory work and resorted to
unprofessional correspondence
with PIFRA.
7 Hardware for ICB Contract Submission of performance
effectiveness guarantee

Conspicuously, Training Consultants had no major dispute and their performance was by far the
best both in terms of quality of the services and professional conduct.

Some of the big consultants and contractors did not match quality of their services and professional
conduct with high reputations they had. However, almost all the disputes were resolved through
negotiations.

10. Performance of the Borrower and the Executing Agency

Some of the conspicuous areas in this regard are summarized below.

l 13 Project Directors were changed during the lifecycle of the project.

l High initial ownership in the form of Auditor General himself being the Project Director could
not be sustained in the middle of the project. However, in the later part of the project, Auditor
General was nominated Chairman of the Steering Committee and took keen personal interest
and cobbled a team that considerably improved the speed of implementation and successfully
overcame critical political and technical obstacles, reaffirming the lesson that without the
personal commitment of the top management, the success of the project can not be assured.
l The capacity constraints of the executing agency at the beginning of the Project are commented
in the ‘Design and Formulation’ section.
l Some of the TORs did not delineate the scope at the required level of clarity and detail, leading
partly to the disputes with the consultants. Some contracts were not scrutinized appropriately
at the draft stage, giving opportunity to the consultants to have the contracts lopsided in their
favor.
l Some of the deliverables of the consultants were not followed up adequately, leading to
situations where that phase of the project was closed without some key deliverables. As per
contract, the services of the consultants would be deemed to have been delivered if deficiencies
were not pointed out within 90 days of the close of that phase. This clause was used by the
consultants to deny submission of the missing deliverables when pointed out in the subsequent
phase.
l Some of the agreements reached during the negotiation but could not be complied over the
years are given below.
l ‘PAD would employ staff with qualifications satisfactory to IDA in the key posts of
Controller General (Accounts), Deputy Auditor General (Government Audit), General
Manager (MIS), Director General (HRM), Director General (Training)’;

- 52 -
 l ‘PAD would designate and maintain a Project Director satisfactory to IDA, with overall
responsibility for project implementation’;
l ‘PAD would employ within the Project Directorate a Project Coordinator and a Deputy
Project Coordinator, with qualifications satisfactory to the Association and other staff
necessary for the carrying out of the Directorate's functions, and that these key staff,
subject to satisfactory performance, would be retained in post for a minimum of three
years’.
l Foreign Trained officials, sponsored through project funds, could not, in majority of the cases,
be placed on relevant positions critical for ensuring the success of the project. Deployment of
these officials remained a concern during the course of the project.

11. Performance of World Bank

Conditions: World Bank conditionalties were in great measure conducive to achieving project objectives.
They proved as common milestones in terms of procurement, resource and implementation planning
towards which relevant stakeholders did exhibit commitment that would otherwise had been difficult to
obtain. In all, a total of 35 conditionalities were met over a period of two years before PIFRA II could be
submitted before World Bank board for approval.

Technical Advice: Technical advice of the World Bank on key issues like Technology Architecture,
Hardware ICB for Roll Out sites, Implementation of IPSAS standards in Pakistan, Audit Methodology,
Technology Landscape for Auditing and procurement of various consultancy services was valuable.
However, in some areas, a more rigid insistence on the part of the World Bank on following guidelines
without any regard for advice of PIFRA based on ground realities resulted in difficulties in selecting the
appropriate contractor for carrying out works.

Nonetheless, close and proactive involvement of World Bank local management in later part of the
project considerably facilitated success in achievement of objectives. This contribution was commendable.

Lessons Learned

l Responsibility for the success or failure of the project should be given to a formally designated senior
executive within the Government hierarchy as executive commitment is the most important aspect for
the successful implementation of the project of this size and complexity. This commitment should not
only be there but also be perceived to be there.
l Inability to adapt to the new processes is the most critical risk that can only be managed if executive
will to implement the changed processes is supporting the project team during the implementation
l Demonstration of success through rapid implementation considerably helps in overcoming resisting to
changes. It is therefore critical to emphasize rapid implementation in order to create demonstration
effect to overcome resistance.
l Rigorous and persistent involvement of the power users in every aspect of implementation (training,
operational acceptance testing, requirement analysis, data migration etc) to completely iron out their
observations and issues is critical to ensure their buy-in without which the implementation can never be
successful. Most of the sites have one or two power users who are the opinion leaders of that office

- 53 -
 irrespective of their hierarchical position. Unless these opinion leaders accept the system,
implementation cannot be successful. The lack of acceptance of these opinion leaders is due to various
reasons, key among them are following:
l Lack of adequate understanding of the new system.
l Lack of willingness or capacity to understand the new processes.
l Vested interest

These powers users can be won over through various techniques like
l Extensive involvement in the design and implementation process
l Training
l Executive Coercion (used very selectively)
l Alignment of their vested interest with the project (foreign training, hiring them at better package
and making them a part of the project team, assurance their influence will continue within their
office and they will not be replaced etc)
l Introduction of project culture (team work rather than hierarchy, risk taking, long working hours,
strong communication and interpersonal skills) is an important factor within the project for the
successful implementation.
l Retention of key staff through incentives and better working conditions will be key to continued
success. Adequately skilled personnel with project-oriented mindset can bring about a drastic impact on
the pace and the quality of project implementation.
l PD should have adequate authority and autonomy in terms of exercising his powers in areas like hiring,
procurement, incentives and project personnel trainings, both local and foreign. Experience of PIFRA
has shown that these issues sometimes become bottlenecks for the project.
l Implementation at the field offices through a core team for every office should form the key to
implementation strategy. This strategy was proposed in the project design as well. All implementation
and training should be carried out through the members of the core team who should be trained by the
Consultants. Exposure of the consultants to rank and file end users should be minimized.
Communication for support and training with the end users should be made through the intermediate
layer of core team between the Consultants and end users.
l Strong Project communication structure should be established early in the project headed by an
experienced communication expert to address the communication challenge such a project poses.
l It is imperative that consultants or implementation contractors should employ known best practices in
order to ameliorate or overcome an already difficult and complex set of factors.
Conclusion

PIFRA had all the difficulties associated with the project of this nature; high visibility, abstract
nature of output, severe resistance to change at every level and multiplicity of stakeholders, to name just a
few.

It went through various phases with varying degree of success in terms of satisfactory progress on
achievement of objectives.

The key factor that galvanized the project was a professional team having the confidence of the
Auditor General. This team brought about fundamental changes in the approach to the project

- 54 -
management; professionalism, project culture, engagement, risk taking, aggressive tracking of activities etc.
Complementing the team was a proactive World Bank Team, the Implementation team of the
consultants/contractors. Together, the achievement of the project could be made possible. Total number of
productive sites is 30. The Federal and all the four provincial governments have prepared their budgets on
the new system of IMF compliant classification, a historic shift from the old to the new. Similarly, a
historic step was taken when IT auditing was carried out for the first time in public sector through the
project. The project has made an impact by training around 22000 DDOs, 400 auditors, 2000 SAP end
users. One of the key success parameters is that the project has made its presence conspicuous to a point
where it is high on Government agenda. The momentum of success must be carried on to PIFRA II with all
lessons learned.

- 55 -
- 56 -