Impact Assessment Roundtable NOM

Date: Tuesday, 10 May 2011 Time: 5-6.30pm Location: Seminar Rm 1-1, Ground Floor, Lee Kong Chian School of Business, 50 Stamford Road, SMU Event Partner: National Volunteer & Philanthropy Centre. This session was co-facilitated by the Lien Centre and Patsian Low, NVPC.

About the Session Most charities in Singapore have moved beyond simple measurements of outputs, to more sophisticated outcomes-driven systems. Impact assessment represents a new horizon in helping such organisations attract donors through informed giving and strategic philanthropy. Charity analysis will play an integral role in helping charities to better quantify their work for such social investors.

This invite-only roundtable brought together actors from various sectors in the social space to discuss how Impact Assessment plays a role in the growth of the sector, from the viewpoint of policy-makers, funders, capacity-builders, and NPOs.

One of the initiatives for discussion was NVPCs pilot project, Independent Charity Analysis, which aims to promote informed giving. The analysis covers not just financial health but also leadership, governance, programmes, and the charitys relevance to the community. NVPC will work with the charities analysed to publish the reports, so as to highlight charities that do good well. The pilot is in progress and NVPC is currently fine-tuning the initiative based on pilot feedback.

NPOs & Impact Assessment o Initially, NPOs have regarded impact assessment as another administrative burden to contend with. However, once they go beyond the funders need to understand whether NPOs are effectively and efficiently using their resources, NPOs have found that such practices have proven useful in strategic planning and prioritisation in the longer run. o The imposition of impact assessment reporting measures provides capacity building for the NPO, as they reorganise their processes in order to fulfil the requirements. This gives funders the assurances needed to continue their funding. o However, one cannot ignore the power distance between funders and NPOs, which leads to NPOs feeling pressured by funders and government agencies to fulfil audits and conduct impact analysis. There is also a risk of funders asking for metrics that NPOs may find difficult or irrelevant. o Other considerations include the desire to not have a NPOs performance be compared against their peers, although it is acknowledged that funders may need to make comparative decisions. NPOs also need to stand firm against certain funders suggestions (which may be perceived as demands) if it runs the risk of introducing mission creep.

Funders & Impact Assessment o Funders need to have humility in their approach and understand the constraints faced by NPOs from their perspective, so as to design metrics that work for them as well. o The more progressive funders will fund small pilot programmes, and overhead costs including salaries (of staff and ED). o NCSS employs a course-correction approach in assessing impact, as they review and learn with the NPO about their desired impact along the journey of implementation. This also provides pilot projects with more flexibility. o IDRC funds on a project basis. They describe outcomes as the changes in behaviour that you can reasonably expect to deliver. They also play an advocacy role through policy influence. They have accountability through their board, and work on a 5-year planning cycle. They have an evaluation unit that looks at how a NPO has grappled with the issues and learn from them over a period. Impact, by their definition, is not something that is directly created by the funded entity but a broader change that occurs over time as a result of various outcomes. o Community Foundation Singapore has multiple funds, and their challenge is managing donor expectations on how their monies are allocated. Donors are groups into 3 levels transactional donors who just donate and stop engagement funds are provided, those who are slightly more engaged and want to learn, and those who are experienced. Pilot projects must also be controlled more carefully than ongoing projects to determine feasibility and continuity. o AP Ventures supported the setting up of Ashoka in Singapore; their funding focus is very specifically on capacity-building and not programme or project specific.

Thoughts on Impact Assessment Tools o Firstly, it is important to differentiate between metrics and methodology. It is often difficult to compare different organisations even when using the same methodology because assumptions, targets and circumstances vary.

Most agree that understanding the stage of development of a NPO is important, and determines the level of impact assessment needed. A start-up, for example, requires a more flexible approach.

We must consider that different NPOs aim for different outcomes even within the same cause or sector; for example one organisation place more importance on research impact, another on child development progress etc.

At the end of the day, any tool which is used is only one of many proxies to measure a NPOs performance. Some therefore question the significance of such tools, and pointed that competence of management is more important.

There is a difference between programme impact and organisational impact. So far most assessment protocols have only looked at programme/project impact/outcomes. This may result in lack of attention paid to capacity-building and overall organisational effectiveness in serving community needs.

While NVPCs initiative to bring more transparency to funders is a step in the right direction, it might not be wise to use it as a one-size fits all tool to evaluate all NPOs.

The reluctance in analyzing impact is due to the prevailing opinion that it is not needed to attract retail funders. It is therefore important that NPOs think of outcome management and impact analysis not only as ways to attract funders, but how it can impact the ability of the NPO to perform better.

Capacity Builders & Impact Asessment o Capacity building is a continual concern, and NPOs training in impact assessment is needed in order to effectively deliver substantive reports. o o Capacity builders are critical in bridging the dialogue between funders and grantees. Another view is that civil society really needs to drive such initiatives for itself, and not wait for funders to herd them into the impact assessment arena. o Knowledge management is typically overlooked when discussing best practices in impact assessment. Accenture, for example, requires every deliverable to be kx-ed or knowledge exchanged before the project is considered completed. o Post-script: there is potential for creating a community of practice to continue the dialogue on this topic.