Must we measure?
In November, Emer Coleman asked: Social media – Must we measure? In a fascinating post, Emer discusses how tricky it is to present the benefits of social media in traditional business cases. Emer argues that:
Making a business case to use these channels is like making a business case to read the newspaper; it suggests that there’s a decision to be made about whether to engage or not.
Both the post and all the comments are well worth reading. I’ve included a couple here:
Ben Giddings:
At the moment, the argument for stats on social media is rather like measuring the success of a speech by seeing how loud the microphone is. What we need to do is work on getting the right people into the room (whether it be physical or virtual) and then talking to them in the right way – just like we do in the real world.
Emer’s reply to a comment by Ranjit Sidhu:
What social does add is the “softer”, human voice that compliments all of the quantitative work that government does. It helps us understand what might really matter to people rather than what we, as government, think matters to people. But these as I say will remain subjective opinions but the more collective voices we can hear the more likely we are to reach a more informed consensus.
I missed Emer’s original post in November, and only became aware of it through the UK GovCamp discussion board via a suggestion by Steph Gray:
So what *can* we measure?
I’ve been wondering about this for ages, and have looked at a bunch of monitoring and reporting tools (mostly Twitter related so far) which I blogged about previously here and here.
I also had a go at tracking Facebook usage in a local area to see if it bore any relation to national & international stats. Posts about that are here, here and here.
I wasn’t looking for evidence to support a business case – I’ve given up on that – but was at least trying to find some metrics.
In response to Steph’s question, I had a go at a list which I’ve included below:
| Platform / type | Measurable (probably) | Additional indicators |
| Followers, re-tweets, mentions | Sentiment, specific feedback, requests | |
| Likes, shares, followers | Comments, requests | |
| You Tube | Likes, views, shares | Comments |
| Followers, repins, likes | Comments | |
| Google+ | +1’s, Shares | Comments |
| Search | Mentions | How the search is phrased |
| Web site | Visits, views, likes | Comments |
| Flickr | Views, favourites | Comments, referrers |
| Blogs | Mentions, pingbacks | Comments |
| Open Data | Downloads, views | Re-use, apps developed (and their popularity) |
So, you can gather at least some ‘how manies’, ‘whens’, ‘likes’, ‘views’ etc, but so what?
So what?
- ‘Followers’: If we see follower growth, can that really be attributed to us?
- I looked at Facebook usage recently in Hampshire, and pretty much all age groups are showing growth, so we might do nothing and still see follower numbers increase*
- Many people have several Twitter profiles – I know one individual who has 12 – in theory you could be followed by one person, many times
- Bots: Aargh! They aren’t human, they’re generally spam, and they definitely aren’t of any value in metrics
- ‘Likes’ are probably a bit more helpful. Most people probably won’t say they like something when in fact they don’t
- ‘Favourite’ might be interpreted as ‘like’, but that’s not necessarily correct – it’s more of a bookmark, and could just as easily be used to remind the reader of something that annoys them
- ‘Re-tweets’ don’t necessarily mean approval – lots of people retweet as a way to share, but not endorse
- Appearance in Search and Blogs probably indicates a level of interest, but you need to look at the content and context to establish if that’s good or bad
- Downloads of open data is an interesting one. It probably doesn’t fit within the definition of social media, but I thought I’d mention it anyway
- In theory data could be downloaded just once and then be re-used, mashed-up, or whatever and become massively popular without the data publisher ever knowing what they’ve started.
- James Cattell got a great debate going about open data on the UK GovCamp discussion board by asking: “Why isn’t #opendata done yet?”
What does meaningful digital evaluation look like?
I’d love to know the answer!
I look forward to comparing notes with friends and colleagues at UK GovCamp on 19th January, and hopefully learning a bit more about a tricky subject.
Photo credits (with apologies)
Mountain Bluebird Wikimedia Commons: http://commons.wikimedia.org/wiki/File:Mountain_Bluebird.jpg
Discover more from Mark Braggins
Subscribe to get the latest posts sent to your email.
You’ve hit the proverbial nail with the notes on the ‘intent’ behind social actions. Ultimately, as always, you can’t measure anything unless you have objectives to measure against that can provide meaning. Those objectives will make the metrics you need to measure clear.
Those are probably things like demographics though, which you’ll probably find hard to get hold of. Got me thinking though – I use a tool called Rapportive which, when I’m emailing people, shows me associated social profiles. Imagine if you could take a user on one social network, find their other profiles and combine the data to build a profile. You might then be able to tell if they’re a 27 year old male living in a city centre, matching your target demographic.
Such profiling may be possible, depending on how much those people are sharing online…
Thanks very much for commenting Phil. I’ve not heard of Rapportive before, which sounds really interesting – I’ll definitely look in to that.
Great post! Really like the additional indicators column that you’ve included, it really adds value to any analysis we might make. Thanks for the ideas!
I’m delighted you found it useful. I’d be really interested if you’ve got any other ideas on measures or indicators. Cheers, mark
It’s an interesting one – I find that it’s the indicators that are most helpful rather than the measures. We do record measures too (and tools like Tweetreach are really useful in showing the power of Twitter in numbers), but these measures tell us very little about the relationships we’re developing or the feedback we’re getting.
This is terribly woolly, but I do have a ‘Happy Folder’, where I store examples and feedback of effective working. It’s capturing that sentiment that you mention above, and when it is coupled with numbers it can be quite powerful. I also really like Storify for showing the difference that social media can make.
I probably don’t have a huge amount to add in terms of measures and indicators, especially in an engagement context, but one measure I do include in our stats is a number of articles and news stories that I’ve sourced through social media. As someone who works on an information service it’s a really useful figure to have.
I agree, Tweetreach is a great tool – I use it frequently. Love the idea of a ‘Happy Folder’!
I forgot to include number of articles and news stories, thanks for that. Presumably you include some sort of sentiment indication on that (positive, neutral, negative etc). Cheers, mark
That’s it really – a short but sweet bit of text about the sentiment so it’s not too onerous. That seems to be the battle of these things – capturing enough data so we can show the effectiveness of what we’re doing, but not so much that it takes more time than doing the social media itself!
…just enough, not too much…perfect 🙂
For me Mark, the indicators aren’t about how many more people have liked or followed or clicked or whatever but are people doing more of the things that you want them to do? Whenever I work with charities the first thing I do is get them to think about what they want people to do online; sign up to newsletter, donate, book training, enquire about services etc. Those are the things that really matter, which of course you can backup with the indicators you talk about above.
Hi Louise. Thanks very much for leaving a comment.
Your point chimes somewhat with Dan’s post: http://danslee.wordpress.com/2013/01/14/channel-shift-a-future-for-public-sector-comms-in-2013/
I totally agree where ‘transactions’ are involved, but am not so sure that it’s all about transactions (that might not be the correct word as I don’t necessarily mean payment is involved)
I think we’re looking at measurement from different perspectives, but I obviously need to think about it some more so I can explain myself better. Look forward to continuing the conversation later. Cheers, mark
I think the real answer is that it’s a mixture of both and I wonder if our different sectors (me charity, you public) go some way to explaining our approaches? Great post Mark.
Nice summary Mark.
Worth repeating that for Facebook pages you get “Insight” stats which include a multitude of measures. Two fairly useful measures are:
reach – “The number of people who have seen any content associated with your Page. (Unique Users)”
people talking about this – The number of people sharing stories about your page. These stories include liking your Page, posting to your Page’s Wall, liking, commenting on or sharing one of your Page posts, answering a Question you posted, RSVPing to one of your events, mentioning your Page, phototagging your Page or checking in at your Place. (Unique Users)
The effect of changes to the Facebook algorithms on your page/post reach has been discussed elsewhere – http://www.comms2point0.co.uk/comms2point0/2012/11/11/facebook-pages-the-pool-partys-over.html.
Jury’s still out. Hoping for a good discussion about this at #ukgc13.
Hi Pete, thanks for leaving a comment, and for the useful tips – I will look in to those and update the table accordingly.
Thought-provoking, Mark. I’ve always been more drawn to looking at effect rather than measurement, myself. While it might be interesting to muse on why, say, the TwitterJokeTrial gathered so much attention on social media, or why Gangnam Style went viral (sadly), to me personally, it is just a bit akin to the mediaeval debate about how many angels can sit on the head of a pin. But what effect those viral phenomenon have, well, to me that’s where you find the important stuff. Rather like meteorology: there are things we can observe and measure, for sure, but we can’t stop them. The more important element in it is to look at what those phenomenon are going to do, when, where and to whom.
Like all analogies, that’s a bit imperfect, but I firmly believe time spent studying impending and actual effect is getting a lesser share of attention at times than social media bean-counting.
Just another point of view…. See you soon.
I agree Tom, it’s intriguing to understand just what it is that grabs people’s attention / imagination and why it’s practically impossible to reliably reproduce the effect.