Evaluation

.
Definition of Evaluation?

A time-bound assessment that systematically and objectively

assesses the relevance, performance, success and effectiveness
of ongoing and completed programmes and projects.

It is a value judgement of the set objectives in the project.

2
Definition of Evaluation?

It can be done during the implementation of the project

keeping in view the improvement of performance and strategy. It
can also be done after the completion of the project in order to
derive the outcome;

Thus evaluation can be formative and summative;

When the cook tastes the soup, thats formative; when the
guests taste the soup, that is summative evaluation.

3
Evaluation
Purpose
The purpose of an evaluation is the reason why you are
doing it. It goes beyond what you want to know to why
you want to know it.
Evaluation
Examples of an evaluation purpose could be:
To provide the organization with information needed to
make decisions about the future of the project.
To assess whether the organization/project is having
the planned impact in order to decide whether or not to
replicate the model elsewhere.
To assess the programme in terms of effectiveness,
impact on the target group, efficiency and sustainability
in order to improve its functioning.
Programme Evaluation
Ex Ante (Before)
Can the programme achieve the results set out in the plan?

Mid Term (Mid way)

How are we doing and what changes are needed?

Ex Post (After)
What did we achieve vs plan?

6
Project Evaluation
Project evaluation is a structured process
comprising:
Collection of relevant information
Assessment of this information against agreed criteria
Reporting of the evaluation results
Incorporation of these results into future planning

7
What intended beneficiaries needed?

8
What intended beneficiaries asked for?

9
What the NGOs promised

10
What intended beneficiaries felt

11
What actually was delivered

12
 A Case for discussion

A Rural water supply project was initiated in 1990 on

community participatory basis. Project was to be completed
in 1994 with cost Rs. 100 million but it actually completed in
1993 with cost Rs. 80 million.
Was project a success story? OR
Do you need some additional information to make judgment?

13
 A Case for discussion

Project in question, was to provide safe drinking water to

1 Lac families living in 25 villages by 2008 but it was
revealed it was accessible to only 25,000 families by that
year;
Was the project a success story?

14
 A Case for discussion

Project in question, was claimed to reduce incidence of

water borne diseases by 40% (1990=70%) by 2004;
But in actual incidences increased to 90% by 2004

15
 A Case for discussion

A survey was carried out in 2005 to see how many water

supply schemes were still in operation and how many had
ceased to exist;
Only 10% were found operational;
Is project a success story?

16
The Result Chain

17
The Power of Measuring Results

If you do not measure results, you cannot tell success or

failure
If you can not see success, you cannot reward it
If you cannot reward success, you are probably rewarding
failure
If you cannot see success, you cannot learn from it
If you cannot recognize failure, you cannot correct it
If you can demonstrate results, you can win public
support
18
Monitoring vs Evaluation
Monitoring Evaluation

Who? Internal management Usually incorporates external inputs

responsibility all levels

When? Ongoing Periodic mid-term, completion, ex-

post

Why? Check progress, take remedial Learn broad lessons applicable to

action, update plans other programs/projects, policy
review, etc
Focus on Inputs, activities, outputs Results, purpose, overall objective

19
 Monitoring vs Evaluation

Monitoring Evaluation
Routine collection of Analyzing information
information Ex-post assessment of
Tracking implementation effectiveness and impact
progress Confirming project expectations
Measuring efficiency Measuring impacts

Is the project doing things Has the project done the right
right ? things?

20
Why Evaluate?
Reporting of results to:
Sponsors
Decision Makers
Stakeholders
Partner Organisations
Beneficiaries

21
Evaluation and Indicators

Relevance
Efficiency
Effectiveness
Impact
Sustainability

22
Relevance
Relevance is concerned with assessing whether the project was
the right development intervention chosen to address a specific
need of the people;
For example, a project for constructing a road can be judged for its
relevance by asking whether its objectives were rightly framed in
accordance to the state's development plans

23
Efficiency
Efficiency deals with the relationship between project inputs and
outputs;
a project may be called an efficient intervention, if it uses minimum
inputs at least available cost to obtain the desired result;
Often there is a trade-off involved with quality of the inputs versus
the costs involved;
least-cost methods often compromise on quality. It involves careful
decision making on part of the project executioners

24
 Effectiveness
Measuring the effectiveness of the project refers to how well it fulfilled
its objectives, by taking into account the effect of the project on the
beneficiaries. When assessing project effectiveness, the following
questions need to be asked:
How have the conditions of the intended beneficiaries group changed
since the beginning of the development intervention?
To what extent was the change in the conditions caused by factors
external to the project.

25
Impact
Impact is a measure of all significant effects of the development
intervention, positive or negative, expected or unforeseen, on its
beneficiaries and other affected parties;
For most projects, it gets increasingly difficult to attribute broad
effects to specific causes. This is primarily due to the fact that
various number of factors that can lead to that particular effect;
Hence in order to conduct an impact evaluation, which aims at
measuring the effect of an intervention, an estimation of a
counterfactual situation is required. This can be achieved by
comparing the two identical situations - one which has been
subjected to the intervention and one which has not

26
 Sustainability
A project may fulfill all the above criteria, but if its positive impacts are
not long-lasting, the project loses its worthiness;
Sustainability deals with asking questions such as the availability of
sufficient resources to maintain project results. It is often the most
rigorous evaluation measure as it is concerned with how well a project
is environmentally and financially sustainable;
For example, sustainability of a road project could be measured by the
likelihood of the road being maintained and its perceived usefulness in
the future, keeping in mind forthcoming developments surrounding the
constructed road.

27
Evaluation and Indicators
Socio-economic problems Impact
Needs
Objectives
Programme

Objectives Inputs Operations Outputs

Efficiency
Relevance
Effectiveness
Sustainability

Evaluation
28
 One for the road!
It is not necessarily a crime to be running late
..It is always a crime not to know!

You can only manage the work that remains to be done!

Getting people to think ahead

is 50% of the benefit of project planning

A successful project is one that where outputs are delivered on time,

within budget, of the required quality.
.and no one died of a heart attack

29