|
|
BOND Guidance Notes Series 6 Monitoring and Evaluating Advocacy This is the fourth in a set of four advocacy guidance notes, which can be read as a series or separately. The BasicsMonitoring and evaluating advocacy work is relatively new territory for NGOs and donors. A great deal of time is spent modifying other evaluation systems to try and meet the unique and opportunistic characteristics of advocacy. The best attempts are likely to involve you collecting what ever information you can as you go along - orally, scribbling down notes and filing them, talking into a tape etc. so you have evidence to back up your arguments or hunches when you need to. The onus should be on doing it rather than worrying about doing it. Monitoring is the collection of information about a project over time. It seeks to understand what is happening. Evaluation is an assessment of the project at one point in time, including the successes and failures. It seeks to understand why what happened, happened. Why monitor and evaluate your advocacy work? The initiative to begin monitoring and evaluating your advocacy work may come from a variety of motivations. The reason why you are evaluating your work will determine the techniques you use.
An essential ingredient of any advocacy strategy is an action plan. This is likely to be a table showing your inputs and outputs; INPUTS. The resources you will be using - often simply in terms of staff and production costs OUTPUTS. The activities you will be undertaking. To build learning, reflection and flexibility into your plans it is useful to add two further columns. OUTCOMES. The impact you expect to achieve from each of your outputs INDICATORS. The evidence
you will collect to show the outcome has been achieved.
Example: Action Plan for the first 6 months of WaterAid's advocacy campaign on Vision 21 and the World Water Forum, In March 2000, governments, NGOs and research institutions in the water resources sector will gather in The Hague to agree a Vision for the 21st century and a Framework for Action. This will be the first time that targets for water resources will be agreed internationally. Governments are expected to pledge to these targets. WaterAid's campaign aims to ensure the inclusion of WaterAid's position in these agreements and to lead a NGO coalition that will commentate on and monitor the implementation of the delivery of Vision 21 and its Framework for Action.
(Case study developed by Belinda U. Calaguas, March 1999, as part of the BOND training What kind of monitoring indicators are there?The range of indicators is infinite.
Monitoring your
target Monitoring your relationships Monitoring the media Monitoring your reputation Monitoring public opinion What should I
be trying to evaluate?
To evaluate the impact of your project you need to be clear about the
model or process you are trying to follow and then decide on what
information is available to enable you to assess each part of the process.
Recent work by the New Economics Foundation suggests you think of the
process as an Impact Chain. Build awareness > Change policy > Impact on peoples'
lives In each phase there are policy and grass roots activities, both of
which need to be monitored. The relationship between these activities is
also important, the more integrated they are, the more successful the
project is likely to be. Grass roots activities are likely to involve
capacity building activities. For example: Group formation > Group activities > Group federation beyond
village level > Movement launched which takes on vested interests >
Groups of poor are involved in framing legislation and have control over
resources Policy activities focus more on raising awareness and changing
attitudes. For example: Heightened awareness about an issue > Contribution to debate >
Changed opinions > Changed policy > Policy change implemented >
Positive change in peoples' lives. Different stakeholder's will have different views on what success is, depending on where they are within the impact chain. To get an overview of how successful you were you need to solicit the views of a range of stakeholders i.e: ultimate beneficiaries, local people and their organisations, staff involved, target audience, journalists and outsiders. This can be done using a variety of methods. For example: Attribution is one of the hardest issues to face in evaluating advocacy work. It is very difficult to know precisely what causes policy changes and precisely what impact those changes have in reality. Many different forces are in play, and NGOs are often amongst the least powerful actors advocating in any situation. Questions you might want to ask to help you assess impact and attribution
(From a model developed by Rick Davies) Top Tips
Bibliography Baranyi, S., Kibble, S., Kohen, A. and O'Neill, K. (1997) Making solidarity effective: Northern voluntary organisations, policy advocacy and the promotion of peace in Angola and East Timor. London: Catholic Institute of International Relations. (One of the few efforts to grapple with effectiveness - in terms of relative/comparative effectiveness of different campaigns.) Covey, J. (1995) "Accountability and effectiveness in NGO policy alliances", pp.167-182 in Edwards, M. and Hulme, D. (eds.) Non-governmental organisations - performance and accountability: Beyond the magic bullet . London: Earthscan. (Early discussion of effectiveness - suggests some avenues for further progress.) Fowler, A. (1996) Demonstrating NGO performance: Problems and possibilities. Development in Practice, Vol.6, pp.58-65. (Discussion of some of the problems faced by NGOs in evaluating performance - multiple stakeholders, no bottom line etc.) Fowler, A. (1995) Participatory self-assessment of NGO capacity. INTRAC Occasional Papers, Vol.10. (User-friendly guide to evaluation. Contains some useful conceptual frameworks, and ideas about indicators.) Fox, J. and Brown, D. (1998) The struggle for accountability: The World Bank, NGOs, and grassroots movements. MIT Press. (Big book, with much information about World Bank - NGOs relations. Contains some discussion of evaluating impact and effectiveness.) Herman, R. and Renz, S. (1997) Multiple constituencies and the social construction of nonprofit organization effectiveness. Nonprofit and voluntary sector quarterly, Vol.26, pp.185-206. (Theoretical discussion of what effectiveness might mean for nonprofits - multiple stakeholders etc.) Rees, S. (1998) Effective nonprofit advocacy. (One of the few empirical attempts to assess effectiveness. Focus is on lobbying in US domestic politics, but is useful re operationalizing and studying effectiveness. Roche, C. and Bush, A. (1997) Assessing the impact of advocacy work. Appropriate Technology, Vol.24, pp.9-15. (Short article exclusively about assessing impact and effectiveness. Argues that evaluating effectiveness is very important and makes some suggestions for progress in this area.) Bibliography prepared by Dr. Alan Hudson, March 1999 Acknowledgements Information contained within these Guidance Notes was brought together by Megan Lloyd laney, Jane Scobie and Alastair Fraser for the purposes of a training course, 'An Introduction to Advocacy', run through March and April 1999. The course was designed as a participative and discursive process and so a huge debt is owed to all the participants and to the 'resource people' used during the course to present case-studies of innovative work and best practice. These notes draw heavily on unpublished materials from Chris Roche of OXFAM. Thanks to the author for permission to use the information. Disclaimer: BOND Guidance Notes aim to encourage good practice through practical advice, however, BOND cannot be held responsible for the outcome of any actions taken as a result of the information contained in the Guidance Notes series. |
|