PROJECT EVALUATION

Presented by
JITHIN MIDHUN JOHN MIGDAD MITHUN MUKESH NASEEM SAJEEH SAJITH

Overview
What is it?
A systematic and objective assessment of an ongoing or completed project
Design Implementation Results

Involves gathering, analysing, interpreting and reporting information Should be based on credible data

Purpose
Learning and improvement Accountability provide useful feed back to stakeholders;
entrepreneurs, sponsors, donors, client-groups, administrators, staff, and other relevant constituencies

Types of Evaluation
BASED ON TIMING

Exante evaluation
Conducted before the implementation of a project as part of the planning Also referred to as appraisal or quality at entry

Ex-post evaluation
Conducted after the project is completed Used to assess sustainability of project effects, impacts Identifies factors of success to inform other projects

POSITION OF EVALUATOR
Internal or self assessment Internally guided reflective processes Initiated and controlled by the group for its own learning and improvement. Sometimes done by consultants who are outsiders to the project Need to clarify ownership of information before the review starts External evaluation Initiated and controlled by the donor as part of contractual agreement Conducted by independent people who are not involved in implementation Often guided by project staff

METHODOLOGY
1. Quantitative 2. Qualitative

Steps in Managing a Project Evaluation

1. Establishing the need for an Evaluation AGRIS project manager(s) need to clarify the purpose of evaluations. E.g.
- Donor requirement Accountability Innovation Learning and change Responding to changed circumstance

 Evaluations take up significant time and resources Need to ensure that the costs are appropriate for the anticipated benefits. Some considerations: Importance of the evaluation to AGRIS or the donor-does it need to take place? Ability and readiness of AGRIS and partners to engage in the evaluation-Is there a reason that the evaluation should be postponed? Cancelled? Size of the evaluation. Setting the focus and scope for the evaluation Resourcing the evaluation: Money? Technical expertise?

2. Assessing the ability and readiness to evaluate

 Defining scope and size Clarify if external or internal Level of effort and resources required / available Stakeholder groups to be involved and how. Full stakeholder desirable, but could be limited to the following: Deciding whether or not to evaluate. Defining the type of evaluation, its scope, and criteria. Defining the evaluation questions, what are the key issues to explore in the evaluation? Defining evaluation work plan. Evaluation activities must be scheduled and fit into the stakeholders' agendas. Deciding which recommendations to adopt and which to reject. Disseminating and gathering feedback on the results.

3.Developing Terms of Reference (TORs)

TOR are the key guide for an evaluation. They should
clarify reasons for the evaluation highlight issues that have become apparent indicate the general depth and scope required spell out any imperatives for the evaluators provide details about methodology, scheduling, cost and the qualifications of the members on the evaluating teams

CONTINUE...
The project manager is responsible for ensuring clear and focused TORs This is as far as the Manager is responsible for development of the TOR

Contents of Terms of Reference

1. Context for the evaluation 2. Rationale or purpose for the evaluation 3. Evaluation issues and questions 4. Evaluation stakeholders 5. Methodology 6. Qualifications of evaluators 7. Schedule 8. Outputs and Deliverables 9. Cost 10.Action Plan 11.Appendices - Evaluation Matrix, Evaluation Policy, LFA

4.Engaging the Evaluator or Evaluation Team

Evaluators can be selected by you, imposed by donors or jointly agreed to. Which ever it is some guide is useful here:
The appropriate level of technical expertise or evaluation expertise The previous experience or profile of the evaluator Suggested profile of a good evaluation team Using peers as evaluators Roles and responsibilities

5.Reviewing and Approving the Work plan

The evaluation work plan is developed by the evaluator and the evaluation team It should:
provide roadmap for conducting the evaluation (interprets TORs) include proposed methodology and means of analysis

A poor work plan leads to poor evaluation Important that the leadership of the project review and approve the evaluation work plan

Reviewing and Approving the Work plan

Suggested outline of a Work plan
Introduction - purpose and stakeholders Evaluation Questions (framework) Methodology (sources, methods) Schedule (Gantt chart) Resource Allocation and Budget Evaluation Team Outline of Evaluation Report

6.Implementing and Monitoring the Evaluation Work

Managers required to facilitate evaluators work by:
Supporting field data collection Making documents available Responding to regular evaluation reports and feedback Distributing draft reports for comments to appropriate partners Participating in donor and evaluator meetings when requested Reviewing drafts of findings and reports and providing feedback

Different Audiences may have Different Needs

Internal staff might need a verbal report and a memo with key points Donors and external stakeholders might need a full report Ministries might need an abstract Public at large might need an abstract of findings only Know your audience and match your reporting approach

7.Effective Communication of Evaluation Results

Captures the data in its conclusions Speaks in language of users Detached, non-possessive stance Objective - truth to power, but Is pragmatic - goes only as far as the key stakeholders will accept

Assessing the quality of an evaluation report and process

Meeting needs commissioning managers, stakeholders Relevant scope Suitable methods Reliable data Sound analysis Credible findings Impartial conclusions Clear reporting What to Evaluate-Outcomes, Processes

Steps In Evaluation
Planning Selecting object (setting objectives) Methodology
Deciding on standards Choice of measures Data collection Data analysis

Implementing evaluation Reporting