Evaluating Advocacy – Craft or Science?

“Advocacy requires an approach and a way of thinking about success, failure, progress, and best practices that is very different from the way we approach traditional philanthropic projects such as delivering services or modeling social innovations. It is more subtle and uncertain, less linear, and because it is fundamentally about politics, depends on the outcomes of fights in which good ideas and sound evidence don’t always prevail”
This is the central premise of a brilliant paper, The Elusive Craft of Evaluating Advocacy, by two academics from the US, Steven Teles and Mark Schmitt.
Although the primary audience of the document is grant giving foundations, the paper has some interesting reflections on the way that we look to evaluate our advocacy built around the concept that evaluating advocacy is perhaps more of a craft than a science.
The paper introduces 9 words or concepts that we might want to bring into our campaigning vocabulary when considering crafting our advocacy evaluations.
1. ‘abeyance’ – The value of keeping the fires burning on an issue even when little visible progress is being made. Critical given situations can often change quickly and without warning.
2. ‘strategic capacity’ – The idea that we should be less interested in evaluating a linear logic model, but instead look at the capacity of an organisation to read the external environment, understand the opposition and implementing the appropriate adaption.
Later in the paper, the authors suggest that ‘What really distinguishes one group from another, however, is what cannot be captured in a logic model—the nimbleness and creativity an organization will display when faced with unexpected moves by its rivals or the decaying effectiveness of its key tools’.
3. ‘spillover’ – a sense that campaigns can often operate devoid of consideration of other unrelated issues, but success elsewhere can often change the political opportunities for our campaigns if they fit a broader narrative set by a government.
4. ‘declining effacy’ – Over time some tactics grow old or ineffective, so organisations need to respond to this decline. The recognition that the ‘no campaign plan survives first contact with the enemy‘ means that campaigns need to continue to reassess and change their tactics.
5. ‘disruptive innovators’ – A strategy or organisational form that does not follow known strategies and the importance of not immediately dismissing new tactics that don’t work, but instead recognising that it can take time for innovators to find the most effective application.
6. ‘spread betting’ – In the paper this is the notion that funders should invest in a portfolio of opportunities, and that funders should have an organisational culture that can accept some failure, as long as they are balanced with notable successes. I’m sure the same principles should apply to organisations.
7. ‘policy durability’ – The need to see if a policy change actually sticks or creates a platform for further change. Build on the sense that advocacy requires long time horizons and doesn’t end when a piece of legislation is passed.
8 – ‘unit of analysis’ – A suggestion that rather than evaluating advocacy (that is the activities) the focus should be on a different unit of analysis, evaluating advocates, and their long-term adaptability, strategic capacity and influence.
9. ‘movement public goods’ – we should asses the value that organisations add to others, do they contribute to broader movement building efforts.

3 thoughts on “Evaluating Advocacy – Craft or Science?”

  1. Hi Tom,
    This is a really interesting post and I think there are definitely some links that can be drawn between the grant-giving foundations that the paper was destined for and businesses operating in the marketplace today. Two of the points I wanted to pull out straight away were 4, ‘declining effacy’ and 9 ‘unit of analysis’.
    As an online analytics company, here at Onalytica we are already monitoring declining (and sometimes inclining) effacy using influence as our unit of analysis for a number of different companies and organisations.
    In this day and age more and more companies are turning to the online debate to quantify and track the decline or incline of campaigns (or brands and products on the whole), alongside calculating influence, (which, in our case, is based on a more scientific approach than tweets, retweets, likes and followers(!)). We stand by a comment recently made by Sir Martin Sorrell, Chief Executive of WPP: “People used to say that information is power. That’s no longer the case. It’s analysis of the data, use of the data, digging into it – that is the power” (Source: Insights with Sir Martin Sorrell – Think With Google http://www.thinkwithgoogle.com/quarterly/innovation/executive-insight.html)
    If you would like more information on Onalytica please have a look at our website: http://www.onalytica.com
    Or for some real life analyses of the online debate, have a perusal of our blog: http://www.onalytica.com/blog
    Thanks again to Enrique for pointing out this post!
    Sophie | Community Manager | Onalytica Ltd.

Leave a Reply

Your email address will not be published. Required fields are marked *