Let’s be honest. Evaluating campaigning is a subject that excites few people, but it’s a really important part of the campaigning cycle.
Over the summer, I’ve been trying to think and learn more about campaigning evaluation, looking at the reports of other organisations that are available, like this one on Oxfam GB’s climate campaign, and learning about the favoured tools of funding institutions.
As you’d expect there are a wealth of approaches, but I wanted to share three tools that I found especially interesting;
Basic efficiently resource analysis – I came across this tool in the Oxfam evaluation where it was used to compare the perceived resourcing of activities to their impact on policies, political agendas or legislation. It seeks to identify the most efficient activities, in terms of achieving political impacts with the least resources.
It seems to be a really excellent way of engaging an organisation in a discussion about what works. As I understand it the report used it by surveying 200 individuals, both within and outside Oxfam who were involved/targeted by the campaign to share their opinions, so it was able to draw on the reflections of various groups. More on how the approach was developed for Oxfam here and here.
Bellwether methodology determines where an issue is positioned in the policy agenda queue, how lawmakers and other influential are thinking and talking about it, and how likely they are to act on it.
Developed by the Harvard Family Research Project, “bellwethers” are influential people in the public and private sectors whose positions require that they be politically informed and that they track a broad range of policy issues.
Bellwethers are knowledgeable and innovative thought leaders whose opinions about policy issues carry substantial weight and predictive value. The approach uses semi-structured interviews and the data helps to data indicate where an issue stands on the policy agenda and how effectively advocates have leveraged their access to increase an issue’s visibility and sense of urgency. More here.
The intense-period debrief is another tool developed by the Harvard. I thought that this tool would be especially useful, as so often in campaigning the evaluation gets left until everything is done and dusted.
The tool recognises that many advocacy efforts experience periods of high-intensity activity. While those times represent critical opportunities for data collection and learning, advocates have little time to pause for interviews or reflection. The unfortunate consequence is that the evaluation is left with significant gaps. Using focus groups or interviews its able to capture key information that might otherwise be lost in an end-of-campaign evaluation. More here.
For those interested in more on evaluation methods and approaches, BOND has produced this great list of tools that can be used. I’d also recommend this document from the Innovation Network on some more unique methods and this paper from the Harvard Family Research Project.
Author: mrtombaker
Happy Birthday to the Government e-petition site
The Government e-Petition site celebrated its first birthday last month, and the team at the Government Digital Services released figures about usage in its first year.
In short;
- 15,600 petition were opened, but a massive 47% of petitions submitted were rejected.
- 6.4 million signatures were collected from the 13 million unique visitors to the site.
- Only 10 petitions reached the 100,000 target that allows them to be discussed in Parliament, and all have been or are due to be debated by members of the House of Commons.
- 97.7% of e-petitions receive less than 1,000 signatures.
I had mixed feelings when the site was launched a year ago, so a year on has the site proven to be a good addition to the campaigning landscape?
Here are a few observations;
It’s got people signing petitions – I’ve not been able to find any figures for the number of individuals who’ve signed a petition and I suspect some significant duplication, but even with that included getting millions of people used to taking action has got to be a good thing.
If you’ve signed a petition here, I think it makes you much more likely to sign a petition sent to you from 38 Degrees or others. Anecdotally I’ve seen a number of the petitions shared on my social media channels beyond those who normally express an interest in campaigning.
It’s provided a clear outcome to those petitions that reach their target. Their was some concern at the launch of the site, that Parliament wouldn’t have time to debate all the petitions that reached the 100,000 signature target, but until the recent rejection of the petition launched by Virgin Trains against the loss the franchise to run the West Coast mainline, it has.
Whatever you think about the topics that have been debated as a result, its good that they’ve been debated by Parliament, and in the case of the petition to get ‘full disclosure of all government documents relating to 1989 Hillsborough disaster’ helped to push an important issue that had been largely forgotten by much of the media back into the public consciousness.
However as the Hansard Society point out ‘the system is controlled by government but the onus to respond is largely placed on the House of Commons’ and many people might be disappointed to learn that the majority of petitions are debated in Westminster Hall, where votes cannot take place and are therefore held on non-votable ‘take note’ motions. As the case of the Virgin Train petition shows, it’s an especially ineffective tool when Parliament isn’t sitting and a petition responding to a current issue gains traction quickly.
It’s disempowering for the majority who have signed a petition. A handful of petitions have been debated, but many more have fallen short. At present there are 12 petitions with between 30,000 – 80,000 signatures on them, perhaps a few of them will make the 100,000 target but most won’t. My Campaigns Totals research has shown that 50,000 actions is a significant number, but the e-Petition site doesn’t provide those who create the petition with many tools to keep those interested in the topic engaged.
They get to send an email at the close but that’s it. I’m concerned that for most they sign a petition hopeful that it’ll actually change something, but that when that doesn’t happen they’ll start to question if other forms of campaigning actually work. As the Hansard Society point out ‘if an e-petition does not achieve the signature threshold but still attracts considerable support (e.g. 99,999 signatures) there is no guarantee of any kind of response at all’. The rigidity of the system means that many are going to be disappointed.
Most campaigning NGOs haven’t launched petitions on the platform, looking through the petitions that have reached the 100,000 target they’ve appeared to have provided an opportunity for individuals or very small campaigns with fewer resources to generate support for their issue . This has often come on the back of an effective social media campaign but the number of ‘failed’ petitions should add a note of caution that this is a high-risk strategy for organisations with limited resources.
Other successful petitions have been those backed by media organisations, for example the ‘Make financial education a compulsory part of the school curriculum’16 was backed by Money Mail, a sister paper of the Daily Mail, and the ‘No to 70 million’ petition on immigration has been heavily mentioned in some parts of the media.
So what next?
The Hansard Society has some good recommendations about how the procedure of dealing with the petitions in Parliament could be improved, for example;
- The creation of a Petitions Committee with staff which would tasked with sifting petitions that secure lower levels of support to ensure that, where appropriate, relevant petitions are, for example, still tagged to debates, that MPs are made aware of their existence, and petitioners receive some form of feedback.
- The Petitions Committee and its staff should respond ambitiously and flexibly to petitions, embracing the full range of parliamentary processes for consideration of them.
- Using petitioner postcode registration data to develop heat maps on the website to help MPs and others identify issues of specific concern to a community.
I’d also suggest that while the site has seen some innovation to it since it was launched, for example the inclusion of a ‘trending petitions’ section on the homepage to help you identify those that have been most active in the last hour, lots of other changes that were suggested at the launch haven’t been included which would help to make the site more engaging for petitioner to use.
Finally, I think as a sector we need to be doing more to help provide those looking to take action with information on if this is the most effective tool to use, this is in part being done by organisations like Change.org and 38 Degrees who allow people to create their own petitions, but others can do more to help inform these decisions and support the many campaigns that don’t actually need 100,000 signatures to deliver change.
Are you signed up?
If you’re not already subscribed to these excellent updates on campaigning, I’d encourage you to do so now;
1 – NCVO Campaigning and Influence – a good monthly round-up of news, views and training opportunities.
2 – eCampaigning Forum – a busy list of over 100+ eCampaigners from around the world.
3 – Sheila McKechnie Foundation e-bulletin – a monthly newsletter promoting the work of the Foundation with useful links to the foundations blogs.
4 – MobLab Dispatches – excellent learning from the Greenpeace powered Mobilisation Lab every three weeks. A really good read.
5 – New Organizing Institute Tip of the Day – Simply brilliant.
Finally, if you’ve not already subscribed to this blog, why not have new posts delivered direct to you inbox by signing up using the box on the right!
Campaign Totals 2012 – DFID
Total number of actions received between May 1st 2011 and May 1st 2012: 172,207
Number of postcards/letters/petitions: 159,913
Number of emails: 12,294
Biggest campaign: CAFOD – Thirst for Change – 60,477 actions
Breakdown by topic and organisation:
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdDk4dGFBdmpmNnB4SGNOYS1UblpSTnc&single=true&gid=1&output=html&widget=true” width=”500″ height=”350″ /]
Scorecard based on figures from 2010 to 2012;
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdDk4dGFBdmpmNnB4SGNOYS1UblpSTnc&single=true&gid=2&output=html&widget=true” width=”250″ height=”150″ /]
Par score based on number of actions at 75th percentile, birdie score on the 85th percentile, and eagle score based on the 95th percentile
View the spreadsheet in google docs here. Information taken from Freedom of Information request returned in July 2012 and has been sorted by number of actions received and is presented as it was received from DFID. More about the ‘Campaigns Total’ project here.
Be first to get the information from other departments by subscribing to the site using the box on the right, adding http://thoughtfulcampaigner.org/ to your RSS feed or following me on twitter (@mrtombaker)
Making the most of Freedom of Information
I spent time on Friday sharing some of my experiences of using Freedom of Information with members of the Campaign Forum.
It’s a group of campaigners from across the sector that get together every quarter to share learning from their campaigns and hear from outside speakers.
I’ve always believed that it’s an under utilised tool for our campaigning, and if used well can help us to access invaluable information that allows our campaigns to be more effective.
It was encouraging to hear that many of the organisations present had made us of Freedom of Information in their campaigning work.
One good recent example of this approach was by Scope who worked with Demos to put together this interactive map of cuts to disability services across England and Wales.
As well as sharing a brief introduction to using Freedom of Information, drawing on the materials from the Campaign for Freedom of Information, I also shared a few top tips for making the most from it;
My tips were;
Be specific – That it’s very easy to have requests rejected because they’re asking for too much information, and as a result fall foul of the set limits that authorities have for ‘checking whether it holds the information, finding and extracting the information’. To avoid this, you need to be specific with what you’re requesting, and make use of time scales, titles of specific organisations/campaigns or locations to help to refine your asks.
I also shared the advice that Chris Coltrane had shared at the Netroots conference, that if you find that your request is rejected because they’ve calculated that it’ll cost too much to find, ask them how they’ve come to that calculation.
Be patient – Under law you’re meant to get a response within 20 working days, but it often seems that the deadline slips. Make sure you keep a good record of what you’ve requested and when, and follow-up once the 20 days have passed.
Ask for advice – That ‘ve found that Freedom of Information officers have often been very helpful it helping to access the information I’ve been looking for, and that you can make it easy for them to get in touch with you by providing a phone number with your request.
It takes time – If you’re an organisations planning to use the information in a media report or similar, don’t expect to have it all together in under a month. Plan well ahead and realise that the process of requesting the data itself is as time-consuming as processing the data afterwards! Campaigners present shared how they’d found it to be a really good project to involve interns in.
Request the data format – This is a lesson that I’m learning the hard way, from not doing so in my last round of request. I’ve got the information back in a whole range of formats, including pdfs which are incredibly hard to extract data from. You can specify the format that you want the information in, and the authority is required to comply with your preference so long as that is reasonably practicable.
You don’t have to justify why you want the information – But I suggested if you are worried about how requesting the information might impact a relationship with an official you can always do so in a personal capacity or work with a colleague to request it.
And the useful advice from the group;
Test out your request – A couple of campaigners spoke of how when they’ve been using Freedom of Information for large-scale requests they’ve tested out the request they’re making with a few friendly FoI Officers first to check that its understandable.
Get the Information Commissioner involved – One campaign spoke of how they’d still not heard back from some local authorities after a number of months and had as a result got the Information Commissioner involved. It’s a good reminder that there is recourse available if you’re not happy with the initial response.
I’ll keep an eye out for good uses of Freedom of Information by campaigns. What tips would you add, or what questions do you have?
Five for Friday…13th July
Here we go with this weeks ‘Five for Friday’. These are 5 great articles on campaigning that you should be reading this week.
1 – Not sure when to tweet? A useful visualisation of the best (and worst) times to do so.
2 – Grist asks Did 350.org’s Twitterstorm to stop fossil fuel subsidies work?
3 – What Jamie Oliver can teach activists.
4 – Mobilisation Lab shares its top ten tips for writing emails to change the world (and if you haven’t signed up for their brilliant monthly emails I’d encourage you to do so.)
5 – Some good reflections from NCVO on the recent charity tax campaign they ran. I wish more campaigns would run this bite-sized learning with others.
What would you add to the list?
Three great campaign innovations from ONE
Three really nice campaigning innovations from the ONE Campaign that I’ve come across in the last few weeks.
1 – The ONE Campaign App
Only available in the US, the iPhone app allows those who download it to stay in touch with the , access key information, connect with staff and take action. If anything it looks like the app is asking users to do much, but I love the feature that allows you to phone your Representative or Senator (complete with a script if you’re not so confident), as well as the integration that makes it easy to share the latest petition with their friends.
I’m sure the challenge of any app is to ensure that users go back to it regularly, but given the massive growth in the use of smartphones over the last few years I’m surprised that more organisations haven’t invested in similar technology.
2 – The DATA Report as an eBook
Another first? I’m not aware of other organisations that have made their flagship policy report available like this, but it makes so much sense to develop it given the environmental benefits and also the cost of sending hundreds of copies to policy makers. Be great to see more policy reports available like this.
3 – TweetNumber10
I’ve highlighted this before, but its such a great website and a really simple user experience. Just a few clicks and you’ve sent a tweet to Number 10. While its interesting that they ‘only’ persuaded 5,000 people to take action, compared to the tens of thousands that normally sign their online actions, I’m sure we’ll see lots more organisations using a similar approach.
Campaign Totals 2012 – Department of Education
Total number of actions received between May 1st 2011 and May 1st 2012: 3,957
Number of letters: 1,042
Number of emails: 2,915
Biggest campaign: People and Planet – Climate Change in the National Curriculum – 1,930 actions
Breakdown by topic and organisation:
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdFlmTy1wNExDY0hHTlcwd0FURFhBbVE&single=true&gid=1&output=html&widget=true” width=”500″ height=”300″ /]
Scorecard based on figures from 2010 to 2012;
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdFlmTy1wNExDY0hHTlcwd0FURFhBbVE&single=true&gid=2&output=html&widget=true” width=”300″ height=”150″ /]
Par score based on number of actions at 75th percentile, birdie score on the 85th percentile, and eagle score based on the 95th percentile
View the spreadsheet in google docs here. Information taken from Freedom of Information request returned on 26 May 2011 and is presented as it was received from Department of Education.
More about the ‘Campaigns Total’ project here.
Be first to get the information from other departments by subscribing to the site using the box on the right, adding http://thoughtfulcampaigner.org/ to your RSS feed or following me on twitter (@mrtombaker)
Campaign Totals 2012 – Home Office
Total number of actions received between May 1st 2011 and May 1st 2012: 33,465
Number of postcards: 12,892
Number of emails: 20,573
Biggest campaign:No weakening of Protection for Animals in UK Laboratories – 13,914 actions.
Breakdown by topic and organisation:
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdFpVV0l5V2lqUVBibUMwTlNtUmZManc&single=true&gid=1&output=html&widget=true” width=”400″ height=”200″ /]
Scorecard based on figures from 2010 to 2012;
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdFpVV0l5V2lqUVBibUMwTlNtUmZManc&single=true&gid=2&output=html&widget=true” width=”250″ height=”200″ /]
Par score based on number of actions at 75th percentile, birdie score on the 85th percentile, and eagle score based on the 95th percentile
View the spreadsheet in google docs here. Information taken from Freedom of Information request returned in June 2012 and is presented as it was received from Home Office.
More about the ‘Campaigns Total’ project here.
Be first to get the information from other departments by subscribing to the site using the box on the right, adding http://thoughtfulcampaigner.org/ to your RSS feed or following me on twitter (@mrtombaker)
Campaign Totals 2012 – Number 10
Total number of actions received between May 1st 2011 and May 1st 2012: 525,000
Number of postcards/petitions: 447,700
Number of emails: 49,900
Biggest campaign: Southern Daily Echo – Have a Heart campaign – 240,300
NB – Number 10 we’re keen to stress that the figures they held were averages to the nearest 100.
Breakdowns by topic and organisation:
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdEVpMlJ0elJINmxyQzN5bXVSYjRIc2c&single=true&gid=1&output=html&widget=true” width=”500″ height=”500″ /]
Scorecard based on figures from 2010 to 2012;
[googleapps domain=”docs” dir=”spreadsheet/pub” query=”key=0ArsF-z0r3hFfdEVpMlJ0elJINmxyQzN5bXVSYjRIc2c&single=true&gid=2&output=html&widget=true” width=”300″ height=”150″ /]
Par score based on number of actions at 75th percentile, birdie score on the 85th percentile, and eagle score based on the 95th percentile
Information taken from Freedom of Information request returned on June 202 and is adapted from information provided by Number 10. The spreadsheets of information can be downloaded for No 10 here.
More about the ‘Campaigns Total’ project here. Be first to get the information from other departments by subscribing to the site using the box on the right, adding http://thoughtfulcampaigner.org/ to your RSS feed or following me on twitter (@mrtombaker)