Evaluating Advocacy – Craft or Science?

“Advocacy requires an approach and a way of thinking about success, failure, progress, and best practices that is very different from the way we approach traditional philanthropic projects such as delivering services or modeling social innovations. It is more subtle and uncertain, less linear, and because it is fundamentally about politics, depends on the outcomes of fights in which good ideas and sound evidence don’t always prevail”
This is the central premise of a brilliant paper, The Elusive Craft of Evaluating Advocacy, by two academics from the US, Steven Teles and Mark Schmitt.
Although the primary audience of the document is grant giving foundations, the paper has some interesting reflections on the way that we look to evaluate our advocacy built around the concept that evaluating advocacy is perhaps more of a craft than a science.
The paper introduces 9 words or concepts that we might want to bring into our campaigning vocabulary when considering crafting our advocacy evaluations.
1. ‘abeyance’ – The value of keeping the fires burning on an issue even when little visible progress is being made. Critical given situations can often change quickly and without warning.
2. ‘strategic capacity’ – The idea that we should be less interested in evaluating a linear logic model, but instead look at the capacity of an organisation to read the external environment, understand the opposition and implementing the appropriate adaption.
Later in the paper, the authors suggest that ‘What really distinguishes one group from another, however, is what cannot be captured in a logic model—the nimbleness and creativity an organization will display when faced with unexpected moves by its rivals or the decaying effectiveness of its key tools’.
3. ‘spillover’ – a sense that campaigns can often operate devoid of consideration of other unrelated issues, but success elsewhere can often change the political opportunities for our campaigns if they fit a broader narrative set by a government.
4. ‘declining effacy’ – Over time some tactics grow old or ineffective, so organisations need to respond to this decline. The recognition that the ‘no campaign plan survives first contact with the enemy‘ means that campaigns need to continue to reassess and change their tactics.
5. ‘disruptive innovators’ – A strategy or organisational form that does not follow known strategies and the importance of not immediately dismissing new tactics that don’t work, but instead recognising that it can take time for innovators to find the most effective application.
6. ‘spread betting’ – In the paper this is the notion that funders should invest in a portfolio of opportunities, and that funders should have an organisational culture that can accept some failure, as long as they are balanced with notable successes. I’m sure the same principles should apply to organisations.
7. ‘policy durability’ – The need to see if a policy change actually sticks or creates a platform for further change. Build on the sense that advocacy requires long time horizons and doesn’t end when a piece of legislation is passed.
8 – ‘unit of analysis’ – A suggestion that rather than evaluating advocacy (that is the activities) the focus should be on a different unit of analysis, evaluating advocates, and their long-term adaptability, strategic capacity and influence.
9. ‘movement public goods’ – we should asses the value that organisations add to others, do they contribute to broader movement building efforts.