Let’s be honest. Evaluating campaigning is a subject that excites few people, but it’s a really important part of the campaigning cycle.
Over the summer, I’ve been trying to think and learn more about campaigning evaluation, looking at the reports of other organisations that are available, like this one on Oxfam GB’s climate campaign, and learning about the favoured tools of funding institutions.
As you’d expect there are a wealth of approaches, but I wanted to share three tools that I found especially interesting;
Basic efficiently resource analysis – I came across this tool in the Oxfam evaluation where it was used to compare the perceived resourcing of activities to their impact on policies, political agendas or legislation. It seeks to identify the most efficient activities, in terms of achieving political impacts with the least resources.
It seems to be a really excellent way of engaging an organisation in a discussion about what works. As I understand it the report used it by surveying 200 individuals, both within and outside Oxfam who were involved/targeted by the campaign to share their opinions, so it was able to draw on the reflections of various groups. More on how the approach was developed for Oxfam here and here.
Bellwether methodology determines where an issue is positioned in the policy agenda queue, how lawmakers and other influential are thinking and talking about it, and how likely they are to act on it.
Developed by the Harvard Family Research Project, “bellwethers” are influential people in the public and private sectors whose positions require that they be politically informed and that they track a broad range of policy issues.
Bellwethers are knowledgeable and innovative thought leaders whose opinions about policy issues carry substantial weight and predictive value. The approach uses semi-structured interviews and the data helps to data indicate where an issue stands on the policy agenda and how effectively advocates have leveraged their access to increase an issue’s visibility and sense of urgency. More here.
The intense-period debrief is another tool developed by the Harvard. I thought that this tool would be especially useful, as so often in campaigning the evaluation gets left until everything is done and dusted.
The tool recognises that many advocacy efforts experience periods of high-intensity activity. While those times represent critical opportunities for data collection and learning, advocates have little time to pause for interviews or reflection. The unfortunate consequence is that the evaluation is left with significant gaps. Using focus groups or interviews its able to capture key information that might otherwise be lost in an end-of-campaign evaluation. More here.
For those interested in more on evaluation methods and approaches, BOND has produced this great list of tools that can be used. I’d also recommend this document from the Innovation Network on some more unique methods and this paper from the Harvard Family Research Project.
Category: tools
Making the most of Freedom of Information
I spent time on Friday sharing some of my experiences of using Freedom of Information with members of the Campaign Forum.
It’s a group of campaigners from across the sector that get together every quarter to share learning from their campaigns and hear from outside speakers.
I’ve always believed that it’s an under utilised tool for our campaigning, and if used well can help us to access invaluable information that allows our campaigns to be more effective.
It was encouraging to hear that many of the organisations present had made us of Freedom of Information in their campaigning work.
One good recent example of this approach was by Scope who worked with Demos to put together this interactive map of cuts to disability services across England and Wales.
As well as sharing a brief introduction to using Freedom of Information, drawing on the materials from the Campaign for Freedom of Information, I also shared a few top tips for making the most from it;
My tips were;
Be specific – That it’s very easy to have requests rejected because they’re asking for too much information, and as a result fall foul of the set limits that authorities have for ‘checking whether it holds the information, finding and extracting the information’. To avoid this, you need to be specific with what you’re requesting, and make use of time scales, titles of specific organisations/campaigns or locations to help to refine your asks.
I also shared the advice that Chris Coltrane had shared at the Netroots conference, that if you find that your request is rejected because they’ve calculated that it’ll cost too much to find, ask them how they’ve come to that calculation.
Be patient – Under law you’re meant to get a response within 20 working days, but it often seems that the deadline slips. Make sure you keep a good record of what you’ve requested and when, and follow-up once the 20 days have passed.
Ask for advice – That ‘ve found that Freedom of Information officers have often been very helpful it helping to access the information I’ve been looking for, and that you can make it easy for them to get in touch with you by providing a phone number with your request.
It takes time – If you’re an organisations planning to use the information in a media report or similar, don’t expect to have it all together in under a month. Plan well ahead and realise that the process of requesting the data itself is as time-consuming as processing the data afterwards! Campaigners present shared how they’d found it to be a really good project to involve interns in.
Request the data format – This is a lesson that I’m learning the hard way, from not doing so in my last round of request. I’ve got the information back in a whole range of formats, including pdfs which are incredibly hard to extract data from. You can specify the format that you want the information in, and the authority is required to comply with your preference so long as that is reasonably practicable.
You don’t have to justify why you want the information – But I suggested if you are worried about how requesting the information might impact a relationship with an official you can always do so in a personal capacity or work with a colleague to request it.
And the useful advice from the group;
Test out your request – A couple of campaigners spoke of how when they’ve been using Freedom of Information for large-scale requests they’ve tested out the request they’re making with a few friendly FoI Officers first to check that its understandable.
Get the Information Commissioner involved – One campaign spoke of how they’d still not heard back from some local authorities after a number of months and had as a result got the Information Commissioner involved. It’s a good reminder that there is recourse available if you’re not happy with the initial response.
I’ll keep an eye out for good uses of Freedom of Information by campaigns. What tips would you add, or what questions do you have?
Three great campaign innovations from ONE
Three really nice campaigning innovations from the ONE Campaign that I’ve come across in the last few weeks.
1 – The ONE Campaign App
Only available in the US, the iPhone app allows those who download it to stay in touch with the , access key information, connect with staff and take action. If anything it looks like the app is asking users to do much, but I love the feature that allows you to phone your Representative or Senator (complete with a script if you’re not so confident), as well as the integration that makes it easy to share the latest petition with their friends.
I’m sure the challenge of any app is to ensure that users go back to it regularly, but given the massive growth in the use of smartphones over the last few years I’m surprised that more organisations haven’t invested in similar technology.
2 – The DATA Report as an eBook
Another first? I’m not aware of other organisations that have made their flagship policy report available like this, but it makes so much sense to develop it given the environmental benefits and also the cost of sending hundreds of copies to policy makers. Be great to see more policy reports available like this.
3 – TweetNumber10
I’ve highlighted this before, but its such a great website and a really simple user experience. Just a few clicks and you’ve sent a tweet to Number 10. While its interesting that they ‘only’ persuaded 5,000 people to take action, compared to the tens of thousands that normally sign their online actions, I’m sure we’ll see lots more organisations using a similar approach.
How great research reports can help a campaign
Some of the wonderful policy colleagues I work alongside are going to be spending time today considering what makes a good research report.
Which has got me thinking about how a great policy report can really help a campaign.
Clearly, the general public aren’t the primary audience for a policy report. They’re normally written to influence key decision makers or technical experts within a government department and as such rightly have an appropriate style and tone.
But I’ve seen how a good policy report can be a massive benefit to a campaign, providing evidence, facts, recommendation and information that is critical to engaging the public.
So here are my thoughts on how policy reports can really help a campaign:
1 – Tell some great stories – Sure, policy reports need to focus on the findings of field research and draw out overall trends and issues, but I’m convince that in the hard work of collecting this information most researchers come across some brilliant stories. Sharing a few of these in your report can help to make the recommendations come alive. They bring a human face to the recommendations, and my hunch is that even policy makers enjoy reading them.
2 – Use some killer facts – Duncan Green has written about this, but facts that stick can really help to bring the injustice of a situation in one memorable statistic. Invest time in thinking about what these are, perhaps brainstorm with some others to identify them.
3 – Invest time in the Executive Summary – Sure it’s the last thing that get’s written and when you write it your fed up with the research, but often the part of the report that gets read the most. For campaigning having an accessible executive summary can be something to share with those supporter who want to go further or need a little more convincing.
4 – Consider writing that seminal policy report – Especially useful at the start of a campaign is the report that helps to frame the issue and provide the campaigner. This is the type of report that gets referred to over and over again, and probably means that the campaigner will end up asking the researcher/policy officer so many questions! A great example is Oxfam Make Trade Fair report launched to coincide with the start of their campaign in 2002.
5 – Work in collaboration – It’s great to be able to quote in presentations to supporters that research by x NGO and y University has found that. If it’s possible work with other organisations, think tanks or institutes it’ll give the research even greater legitimacy.
6 – Tell us about it before you write it! We’ll be interested in it and perhaps we can help to provide some ideas about how to communicate your research beyond the usual suspects. Consider using an info-graphic to communicate the main message of your report.
Campaigner – Do you agree? What else would you add?
Researchers – What have I overlooked? Have I hideously over simplified the process?
What we can learn from the Robin Hood Tax campaign
The Robin Hood Tax campaign has used a range of new and innovative tactics that other campaign coalitions could learn from.
I suspect that you’ll be hearing a lot about the Robin Hood Tax in the coming week. The campaign has a great opportunity to succeed when G20 leaders meeting in France at the start of November, while the ongoing #OccupyLSE protests have increased the level of debate about what’s stopping the UK government implementing the tax.
I’ve highlighted before that the campaign has produced some fantastic campaign videos, but a few other tactics that I’ve seen the campaign use stand out.
Innovation Day – Back in the summer, the organisers behind the campaign opened up the planning process to anyone who wanted to come along and attend its Innovation Day. The day was hosted in London and facilitated by a team involved in the campaign. It’s great to see a campaign ‘open-source’ its planning process in this way, inviting supporters to help to shape the direction and contribute their ideas. The day ended up producing two ‘concepts’ that supporters were invited to provide further feedback on and get involved in. I’m not sure what happened next, but it’d be good to see more invite committed supporters to get involved like this.
Fundraising for adverts – The campaign isn’t the first to invite supporters to make small contributions towards advertising or other campaign activities, but by using it the Robin Hood Tax campaign has highlighted a growing trend by campaigning organisations – asking for small donations for specific items of spending.
You can see the strengths in the idea with PayPal it’s easy for people to donate, and micro donations of £1-£3 don’t feel like an enormous cost to an individual. My only concern with this approach is that is doesn’t help to communicate the less visible costs of a campaign (producing a policy report, paying for staff, etc). We need to ensure that we’re encouraging people to commit to long-term support for campaigning organisations as well.
Gone after the vested interest – Often in campaigning I think we’re guilty of not spending enough time thinking about the forces and organisations who don’t want our campaigns to succeed. We campaign under a belief that if our arguments are ‘right’ then we’ll succeed. The RHT campaign was clearly aware this wouldn’t be the case when it came to heavy resistances from the financial sector, and it’s been good to see how the Robin Hood Tax hasn’t been afraid to highlight the ideological and financial links between the current government and the bankers.
Exemplary use of a experts – Bill Nighy has been an inspired choice as a spokesperson for the campaign. He’s been active in the campaign for the last year, appears on TV to talk about why we need a RHT, writes op-ed pieces and turns up at events. Unlike other celebrities he appears committed for the long-haul of the campaign.
It’s also been good to see how the campaign has used Economists, like getting over 1,000 to sign a letter ahead to G20 financial minister and more recently Jeffery Sachs writing to the UK Chancellor, to counter the financial voices that say that a FTT can’t work.
What else has impressed you about the Robin Hood Tax campaign? What hasn’t worked so well?
Using the Law in Campaigning – are we missing an opportunity?
Ask a campaigner about the intricacies of Parliament, and the likelihood is that they’ll speak at length about the role the APPGs, PQs and EDMs can play in campaigning.
But move the topic of conversation to the role of the Law Courts in campaigning and the conversation is likely to be much shorter!
Despite the judiciary being at the heart of our political system, we seem to see far few campaigns using legal routes to achieving change.
But as Mark Lattimer writes in The Campaign Handbook ‘the law’s authority can be invoked to constrain government action, check abuse and generally make life difficult for those who govern’ so it could be a powerful tool to use.
Indeed in recent week, we’ve seen 38 Degrees turn to legal opinionto get another view on the proposed Health Bill that was going through Parliament, and use it as a tool to challenge some of the lines that the government was putting out in the media and inform its campaign communications.
While at the end of last year, the Fawcett Society challenged the Government’s emergency budget on the ground that it hadn’t been drawn up in accordance with the law, and in particular the legal requirement to consider the way in which different measures impact differently on men and women.
Both are excellent example of how campaigners can make use of the law, so what might be stopping campaigners making more use of legal routes?
Cost – It’s an area where specialist legal training is more often than not needed, and unlike other campaigning activities it’s hard to get far without having to draw on this advice. Most campaigners can learn ‘on the job’ about how to write a good e-action or organise a stunt but that option doesn’t exist, and because of the high cost of professional help that can be prohibitive for many campaigns, plus off-putting to a CEO counting the pennies!
Speed – In an environment when campaigns want to win victories quickly, the deliberative nature of the law can mean that decisions that do or don’t support a campaign can take months or years to come, for example the Fawcett Society’s case took 3 months to be heard by the High Court (and that was quick!).
Knowledge – As campaigners, were just not as familiar with the legal system. We might have an understanding of how to use Freedom of Information or certain Environmental Information Laws but that’s about it. Most campaigning courses don’t spend long looking at using the law, plus add to that the legal environment is often changing with new laws being passed and judgments being handed down and it can be hard to keep up.
But given the examples of 38 Degrees and the Fawcett Society what resources are available to those campaigns looking to consider this?
Mark Lattimer’s The Campaign Handbook has a whole chapter devoted to ‘Using the Law’ although I’d suspect that some of it is now a little date, while Campaign Central has a section devoted to ‘The Law and Campaigning’ although much of it focuses on what campaigning you can and can’t do because of charity law.
Specialist groups exist to help provide advice and undertake pro-bono work for charities and other groups. I’ve been able to make use of Advocate for International Development, who title themselves as ‘lawyers ending poverty’ and provide support for development NGOs.
What other resources exist for campaigning looking at using the law? Should we make more use of legal routes in our campaigning?
Great campaigning resources
Looks like it’s a day for lists of useful guides on campaigning.
Firstly I came across this list that Mark Parker, a campaign organiser in Southwark has put together of books to read on activism on his Southwark Organising blog.
Secondly, it’s the last day of The Right Ethos resource tweeting marathon! For the last month they’ve been sharing ‘a series of 30 daily guides covering campaigning, policy & parliamentary’ via their twitter account. It’s a really useful list full of some great resources, many of them come from the work of the NCVO Campaigning Effectiveness programme, but they’ve also linked to some interesting resources from the US and beyond.
The full Right Ethos list is below;
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”hl=en_US&hl=en_US&key=0ArsF-z0r3hFfdGhhQXQ4UU9PdzNnX2NTcDNTRlp6bEE&output=html&widget=true” width=”500″ height=”600″ /]
What resources, books or guides would you add?
Four exciting things happening in campaigning
I spent some time with some colleagues last week talking about future trends in campaigning. As part of it, I was asked to share the four things most exciting things happening in campaigning at the moment.
Here’s my list, what would you include?
Change.org – Combining e-activism and crowd sourcing, change.org seems to have hit upon a great campaigning tool.
Although it’s not had a big launch in the UK yet, it did manage to generate two significant actions to Home Office in the last 12 months. Change combines a platform to allow individuals to come up with their own campaign actions and a mechanism to push those out to a wider audience, including media and organising support.
I really like the way that they’re putting the campaigning tools in the hands of individuals who are interested in running campaigns and the creativity of some of the actions that are being generated. Lots of campaigning organisations could learn from the approach that change.org is taking.
Gates Foundation – I’m not only excited by the recognition from the Gates Foundation that they need to be engaging in
advocacy, and the clear theory of change they have which is to invest in research and analysis in the south, initiate a debate in the media and support public mobilisation.
I’m excited at the potential of other Foundation following them and providing a much-needed funding stream for advocacy. I also think we should be thankful for the work that these Foundations have been doing to help us measure to monitor and evaluate the impact of advocacy.
Finding Frames – Because it’s helped to spark a conversation about the language that we need to be using to win our campaigns and helping the sector to engage in the literature about frames and values. It’s sister report, Common Cause was the most recommended item of summer reading, it’s a great introduction to lots of fascinating and vital literature.
Citizens UK – Community organising seems to be making a (much-needed) comeback and this has
been spearheaded by the work of London Citizens. It’s engaging a new set of activists, empowering communities that haven’t been engaged in campaigning before and having real success in changing policy. They’re reminding others that campaigning is about community, identity and empowerment.
What would you include in your top 4 and why?
Five for Friday 24th June….
It’s Friday, so here are five great articles I’ve read in the last few weeks that are worth reading in your lunch break…..
1. Research from the US suggested that ‘LinkedIn Is An Untapped Treasure Trove For Political Campaigns‘ because it draws older, more educated citizens–voters who are far more reliable when it come to casting ballots than those on Facebook. (h/t @rechord)
2. The Guardian reports on a new report from the Constitution Unit that suggests most decisions in the government are reached through informal channels rather than formal coalition machinery. Alison Goldsworthy has some useful advice on the NCVO Forum about influencing the coalition.
3. Casper ter Kuile points to a great article on the New Organising Institute that reminds us of ‘What We Can’t Teach: Courage and Commitment in Campaigns‘.
4. A new e-book reviews the last 10 years of the Treatment Action Campaign in South Africa. (h/t @sullyserena)
5. Charles Secrett causes a stir by arguing that ‘Environmental activism needs its own revolution to regain its teeth‘ promoting a strong rebuttal from Greenpeace and Friends of the Earth and a further response from Secrett. My own thoughts on the original article are here.
What else have you read that you’d add?
Be the first to know about new posts by subscribing to the site using the box on the right, adding http://thoughtfulcampaigner.org/ to your RSS feed or following me on twitter (@mrtombaker)
1. Research from the US suggested that ‘LinkedIn Is An Untapped Treasure Trove For Political Campaigns‘ because it draws older, more educated citizens–voters who are far more reliable when it come to casting ballots than those on Facebook. (h/t @rechord)
2. The Guardian reports on a new report from the Constitution Unit that suggests most decisions in the government are reached through informal channels rather than formal coalition machinery. Alison Goldsworthy has some useful advice on the NCVO Forum about influencing the coalition.
3. Casper ter Kuile points to a great article on the New Organising Institute that reminds us of ‘What We Can’t Teach: Courage and Commitment in Campaigns‘.
4. A new e-book reviews the last 10 years of the Treatment Action Campaign in South Africa. (h/t @sullyserena)
5. Charles Secrett causes a stir by arguing that ‘Environmental activism needs its own revolution to regain its teeth‘ promoting a strong rebuttal from Greenpeace and Friends of the Earth and a further response from Secrett. My own thoughts on the original article are here.
What else have you read that you’d add?
Be the first to know about new posts by subscribing to the site using the box on the right, adding http://thoughtfulcampaigner.org/ to your RSS feed or following me on twitter (@mrtombaker)
From Across the Pond – US approaches to planning and evaluating campaigns
I’m in the US this week linking up with various advocacy organisations, so I’m going to use the posts this week to highlight some the interesting work that’s coming out of the US about planning and evaluating advocacy.
Much of it has been driven by a desire by some of the large foundations (who are major donors to the work of US charities) to develop an approach that helps them to asses the quality of the applications they receive from organisations.
The first resource I’d flag up is ‘Campaigning For Change; Learning from the United States’. I’ve read it twice now and I have to confess to still not understood it all, but that’s not because it’s badly written, but simply because of the complexity and breadth of the topic it’s trying to address. Proving if you’re campaign is making a difference.
But I’m going to persevere because I think so much of the insight and the models it shares will be invaluable to UK campaigning. I know that I’ve lost count of all the times that I’ve been asked the question ‘is your campaign making a difference’ and then struggled to answer it, and I think this tool could help to answer that. The report shares lessons about how US organisations have used a ‘Theory of Change’ model to inform their advocacy planning over a number of years.
A Theory of Change (TOC) can be defined as ‘laying out what specific changes a group wants to see in the world and how and why a group expects its actions to lead to those changes’
A TOC is built on the basics of the Logic Model, but encourages organisation to develop an understanding of what is required for change to take place and what strategies will be used along the way, and to think about the links about how the activities undertaken and the end goals based on insight from political and social thinking. The main elements are;
1 – Stating a clear aim
The TOC encourage you to start with a clear aim, which should be seen as the overall purpose of the campaign, the change a organisaiton wishes to see and the impact it wants to make.
2 – Mapping activities to achieve your campaign aim
Examining what activities will bring about the campaign aim, and being clear how they link to the end goal you’re looking to achieve. At this point it’s about considering activities, like a creating the political will for change or developing of alliances as opposed to considering the detailed tactics (holding a march, running an advertising campaign, etc).
3 – Outcomes and how to get there – Using ‘so that’ chains
This stage of the process is about being clear about how each of the activities will link together, it encourages the use of ‘So That’ chains to check the validity of a set of specific assumptions by looking at the logical links between the different steps of the campaign. This allows you to link specific activities with the expected effect or outcome in the journey towards the desired result, and provides a space to challenge the assumptions that you might make.
4 – Understanding how social change happens
Central to TOC is an understanding of what strategies bring about what types of social change. Ensuring that the types of activities being undertaken match the overall strategy being pursued is important. The paper draws out a number of different reasons for how change happens based on academic studies and approaches. This can be used for the basis of a diagrammatic strategy to show the journey that you believe your campaign will undertake.
5 – Capacity of the organisation to achieve change
By doing the above, a TOC can help to illustrate the elements that an organisation will need to ensure they have the capacity to carry out their strategy.
6 – Evaluation built into the model
By articulating the change desired and the anticipated process that you’ll undertake to achieve it, you’re able to evaluate throughout the process, and also provide a space to question if the assumptions that were made are correct or not.
It’s hard to summarise the model in a few hundred words, so I’d strongly encourage you to read the report, or look at the Brian Lamb’s presentation at the NCVO Campaigning effectiveness conference.
Part of me really likes this approach;
- It ensures that assumptions about the value of a specific tactic or approach are discussed and understood by all involved.
- Requires organisations to consider how they understand about what brings about social change, especially important when trying to communicate about the rationale behind the tactics that have been selected in a large organisation, where not everyone is an advocacy specialist.
- Challenges a ‘one size fits all’ approach to deciding on campaign tactics because they’ve always been used.
- It puts evaluation at the heart of the process, so it’s easier to monitor the ‘impact’ that the advocacy is having.
But I’ve also got a few reservations about it;
- The model isn’t the easiest to get your head around which might turn some away from looking to use it. I hope that NCVO are considering offering further training about implementing it.
- I’m not clear from the report what about the role and place for considering the external environment is. It seems that the model doesn’t have an obvious space for exploring what’s happening outside of the organisation/campaign. I don’t think we should be putting down the PEST charts yet!
- Will it work for national advocacy campaigns? Most of the models quoted in the report are based on statewide or local public awareness campaign, where the required outcomes and results don’t require the activities
Have you used this model in your advocacy planning? Does using such a tool appeal to you or does the complexity make you switch off?