Media Relations Measurement: An Introduction
14th January 2007 by Eric Bergman, ABC, APR
Enter the world of media relations measurement, and you enter a new dimension. Depending on the agency, practitioner or client in question, it’s often a virtual hodgepodge of approaches that includes column inches, advertising equivalencies, tone, exposure, awareness, attitudes and behavior.
But let’s face it. Anyone can count column inches, search blog hits, and add up airtime. Tone can be subjective, and there are countless versions of advertising equivalencies and the multipliers that accompany them.
Personally, I don’t believe that counting column inches or placing an article “above the fold” is good enough as a measure of strategic success in media relations. As my colleague Sue Johnston, ABC, recently commented when she and I discussed this series: “Placing an article above the fold is a measure of a journalist’s success, not a communicator’s.”
Column inches and advertising equivalencies are, at best, a starting point. They are rooted in the public information model of public relations. Loosely defined, the public information model says that if you throw enough information at the wall, some of it will stick, although we have no way of knowing how much actually did.
That’s fine, but as communicators, our business should be attitudes, opinions and behavior — which are more closely aligned with two-way asymmetrical and two-way symmetrical models of public relations practice.
Over the next few weeks, I’ll be posting a series of articles based on podcast interviews I’ve conducted with professionals in marketing and public relations. The purpose is to shed some light on the spectrum of media relations measurement, to hopefully outline some best practices, and to encourage us, as media relations practitioners, to strive for more.
I don’t think that counting column inches will disappear at any point in the near future, but I believe we cannot claim strategic success until we close the communication loop and focus on attitudinal and behavioral change as measures of how effective we’ve been.
You can click on the “subscribe” button to the right to automatically be informed when articles are uploaded. And, as this series unfolds, I welcome your comments, questions and suggestions for future inquiry.
By the end of the series, I hope to provide a range of ideas and tools you can apply to media relations measurement, and to inspire you to think beyond equivalencies and column inches to the challenge of positively influencing attitudes, opinions and behavior.
January 14th, 2007 at 12:58 pm
For more approaches to measuring PR, people in the Toronto area can go to an American Marketing Association session called PR MEasurement Wars:Who’s Winning, on Tuesday, January 17.
There’s details here:
http://www.email01.com/AMA/email/december20_2006.html
Among the speakers is Tracey Bochner, of Apex Public Reltions, who will talk about a measurement system invented by the Candian Public Relations Society, in assocaition with various other people and, if I remember correctly, IABC Toronto (not international) was invlivced, too.
$75 if you are not a AMA member. I learned about this because the AMA sells its seminars in part by putting advertisements into the newspaper, so a broader audience than its own members can learn more about marketing.
I’m on Eric’s side in questioning ad line multipliers — perhaps even more violently opposed and certainly more prone to insulting people stupid enough to fall for them.
On the other hand, I get my symetricals and asymetricals and two ways and one ways and three ways all mixed up. Once upon a time I thought I’d start a PR company called multilogue, because monologue and dialogue did not go far enough.
Those exposed to the minbd-numbing education of most MBA courses love numbers — but smart people know that it takes insight to measure PR success, not a calcultor. Y
ou had a purpose. After the PR program (pr is not rteally events, it;s programs, but that’s a whole otehr story)was put into action to meet this purpose, it’s usually easy enough to know if you succeeded. This assumes, of course, that there was nothing but PR invoilved. No new packages for packge goods, no discount coupons. No ads.
I was part of an advertising measurement project last week. A software company ran a big ad in The Toronto Star, and infomercials on late night TV. The week after I looked around the room, and there were more than 100 of us in the room learninghow to get rich buying and selling currencies. Pretty much the only reason any of us knew the seminar was on was because we saw the ads.
But it’s hard to find a PR situation similar, although one that comes close would be book publicity.
Book publicists work hard, and then radio and television stations, newspapers and magazines, and various web sites publish little stories, sometimes big stories, excerpts and sometimes, reviews, and the publishing industry has fairly good methods of measuring relative sales (best seller lists) and actual sales (royalty statements; mine seem to come in quarterly at under three dollars. Lowest? 17 cents>)
So the measurement is not the press clipping book, it’s the sales figures. Every Southam paper once published a photo of mine large or really large; would more copies fo the book have been sold if the shot ran two columns instead of six? And mostly, the book sold well …(it was Boom Bust and Echo 2000 and the photo was of David Foot and Danny Stoffman beside the one bus shelter where the book was advertised) … soold well for many combined reasons. David and Danny wrote a good book, news editors thought the book was worth writing about, reviewers caught on to demographics, my picture caught the attention of readers, who then read the accompany story and made a buying decision, the sales reps talked the book stores into good positioning, the publicist got David and Danny interviewed all over the place…
But if anyone tells you four minutes with Sheila Ridgers on the radio is worth $1256, they are blowing smoke.
I look forward to the series here. Is IABC publicizing it well?
BAK
January 16th, 2007 at 6:24 am
As Brian Kilgore mentioned (thank you!), the AMA event on Thursday, January 18 is focused on this topic and I expect it will be a lively discussion with panelists talking about a variety of measurement tools and systems.
One small point to note:
As part of the CPRS Measurement Committee I have had the opportunity to speak at many conferences and events over the past nine months since we launched Media Relations Rating Points (MRP), CPRS’ new standard for measuring media coverage.
At every event, I ask the group if they are using ad values and very few, if any, raise their hands. My learning is this: it is now safe to say that ad values (or PR values or AVE or whatever your preferred name for them is) are finally dead. Many of us stopped using them years ago, recognizing the inherent issues with this form of measurement, and others have stopped more recently, but the end result is the same–they are no longer considered by the majority (or even a solid minority) as a valid form of measurement.
I think it is time we stop referring to ad values as a starting point for media measurement–even a bad one–because it reinforces its use as a viable measurement tool. Eric, I’m in complete agreement–the best solution is to find ways to measure behaviour.
Dialogue on measurement is always a good idea–I look forward to following this series.
Tracey
January 16th, 2007 at 9:03 am
Tracey,
I spoke with David Jones about Media Relations Rating Points, and our conversation will be featured in the third segment of the series.
I agree with you that advertising equivalencies should be dead, but I judged media relations categories for Gold Quill and IABC/Toronto’s OVATION awards programs last year and, if I was tempted to add up the equivalencies in the entries I judged, I would be reminded of one of my son’s favorite movies:
“We generated a hundred million-billion-gazillion dollars” in advertising equivalency.
I hope to see you Thursday morning!
Cheers,
//eric
January 16th, 2007 at 12:10 pm
Eric and Tracey,
There is a big difference between measuring media coverage and measuring the effectiveness of a media campaign. The challenge is in tying the two together…from the start of the planning process.
True measurement isn’t taught in most (any?) pr planning courses. Practitioners are often left to their own devices to determine success and it’s rare to see a media report that points in any direction other than success. When you can “measure coverage” by clips, impressions, hits, AVEs, PR equivalencies, above the fold, etc., then why bother with real outcome, bottom-line, effectiveness measures?
We’ve made it extremely easy for practitioners to slide when it comes to measures and have dug ourselves a deep hole. Not it’s time to start digging out.
Wilma
Wilma
January 16th, 2007 at 2:46 pm
All,
While I absolutely agree that AVE’s are not the way to go, I’d like to offer an important distinction. An AVE as a stand alone metric (an absolute number) as a measure of a campaign’s success, should indeed, be kicked to the curb. There’s less arguement across the industry daily on that among those who measure for a living, clients and, increasingly, agencies.
An AVE can, however, be somewhat useful in two possible contexts. And I’ll give proper credit here as they aren’t my ideas. Angela Jeffries of VMS will tell you that AVE’s can be used not so much in absolute as in relative terms. In other words, an absolute AVE by itself is useless, but as a comparison between relative market value (or ammount of media attention paid to) one product to the next within one company or perhaps one competitive product compared to another company’s, it can be somewhat telling. But even then the metric isn’t that useful on its own. Angela also makes and interesting case for correlating share of media discussion (which is a measure of both the quantity and quality of coverage) with sales. Skeptical? I was, but 160 studies of entire brand categories such as consumer packaged goods looking at 10’s of thousands of articles have proved interesting. Without seeing behind the methodological curtian, it would appear that while advertising is held constant (she uses and three-year longtitudenal example) share of discussion does correlate with trends in sales. Exclusively repsonsible? Not likely. Directly attributable? Tough to prove direct effects and behaviour as decades upon decades of academic research tells us. But complex statistical analyses like this and others like market mix modeling are making some headway.
Second, and this thinking comes out of the Institute for Public Relations Research camp, there’s a new ‘return-on’ called return on earned media. Though as an organizaton we object to the way that some in the industry in some cases (mis)apropriated the term ROI and its variants, the model at least takes AVE a little further. Essentially, it’s a slightly more sophsiticated version of AVE that accounts, as any analysis of media coverage should, for message delivery, quality, frequency etc. So, an ad rate of $55,000 x 60% message inclusion / $25,000 PR spend yields a 132% return. So in this case ad value gets adjusted downward. But, again, what’s more telling than that absolute number is the changes in relative terms to that number (track it compaign to campaign) over time. Some may note that I’ve left out the x2 earned media multiplier. Multipliers are a source of significant debate across the industry just now. While it’s still early, there was some fascinating research presented at the Institute for PR’s measurement summitt in September indicating, at least in the one example tested, there is not significant difference between advertising and editorial. So, I suspect the extent to which we are debating AVE’s is the extent to which we’ll be debating multipliers down the road.
Our official position on AVEs is that, yes we have access to the data and will somewhat reluctantly provide it to clients (only those who exlplicity ask for it) as one fairly unimportant among dozens and dozens of far more meaningful metrics and indices (some qualitative, many quantitative), but with many cautions and caveats.
January 16th, 2007 at 4:13 pm
Glad that this debate is taking place, as always. My problem with most of the above discussion is that it assumes that the purpose of PR is sales and it is frequently not about sales at all but about the management of a reputation, or the education of you constituencies around an issue, or the prevention of disease .. you get my point. We are misleading our publics if we say that AVEs or MPRs or anything else that is solely focused on print media is an answer to measuring PR. It is measuring media, and traditional media at that. It is NOT measurement of social networking, or the conversations taking place in the blogosphere, or the reputations being managed by hard working communications professionals every day.
January 16th, 2007 at 7:09 pm
Alan,
When you use the term “message inclusion”, what does that mean?
//eric
January 16th, 2007 at 7:13 pm
Katie,
I agree with you and I believe you’ve hit on the source of the discomfort I’ve felt for some time with any form of line counting of any kind.
It really doesn’t take into consideration those other elements — conversations, networking and discussions — which ultimately lead to people developing attitudes and opinions (and perhaps expresses itself as behavior out the other end).
So how do we get to the point of examining outcomes, rather than counting inputs?
//eric
January 17th, 2007 at 7:12 am
Great point(s) Katie.
Eric, you asked about message inclusion. Some call it pull through or resonance. Often we recommend to clients that 3-5 of their overarching core reputational messages be tracked in their coverage. We look for those messages (or the spirit of them) and we look at whether they are present, present supported, present but countered (and by whom in each case). Fairly common among the other content analysis shops out there.
One point on moving toward outcomes (beyond the clump ‘o clips and the sales correlations). And agreed, PR is much more than about either. Beyond Katie’s assessment which is spot on, PR is about corporate reputation, trust, sponsorships and relationships–all of which can be measured.
While it’s not behaviour, it’s the middle ground of opinion. We’ve done some work correlating trends in media coverage and profile with public opinion polling data. Two challenges here. First, while the data often already exists, it resides with the clients and they can be reluctant to turn it over. Second, the media content analysis shops aren’t always brought in before the public opinion polling process begins. Ideally, the media content analysis shop would be at that consultative table so that, at the very least, certain medai consumption questions could be added to make the correlation work down the road easier and more relevant.
As many in the measurement business will say, and I recall hearing Katie mentioning this, or something akin, at the measurement summit, frequently there are all sorts of data that resides with the client that could be pulled together if loked at strategically and holistically. The challenge is which sources, getting access to them and what to do with it once you have it.
January 24th, 2007 at 5:53 am
My thanks to Alan for drawing my attention to this debate. The discussion is very much to be welcomed, and I look forward to following Eric’s series.
As an IABC member for some years and as Chairman of the Board for the international Association for the Measurement and Evaluation of Communication (www.amecorg.com) I would commend all those with an interest to benchmark the AMEC URL, look at the AMEC College and join us in our activities. AMEC comprises more than 23 research companies from Europe, Australia and the Americas with a burgeoning individual membership too and runs training courses and a Quality Assurance standard to help promote best practice.
In terms of the discussion so far, I’d like to simply underscore a few basic principles.
Of course, behaviours and outcomes are ultimately the most important goal, but that doesn’t mean that interim processes are without value, far from it. The purpose of media content analysis is rather less to do with computing ‘an evaluation’, and rather more to do with providing a robust data platform which can then be used for many different purposes including performance management, audit, resource planning and testing hypotheses as to cause and effect.
AVE, crude multipliers, unreviewed interpretations of favourability and measuring column cms are all of very limited value. But each project should be set up to use the tools that are most appropriate. There is no silver bullet, and those that lust after one are missing the point. Successful communication of ideas and even product endorsments is too complex to be susceptible to a single algorithm.
The point about whether a story is above or below the fold is not about the effort and skill of the communicator (or the journalist), it’s about the likeley outtake by the audience(s) for that piece…for that’s the raw material the communicator then has to shape further as the issue progresses.
January 24th, 2007 at 7:57 am
Nicholas,
Thank you for your input and bringing AMEC to our attention.
Like you, I believe any measurement is a start. But I think too often we get a bit complacent and don’t focus on what sets us apart.
Ultimately, anyone with a reasonably interesting story can “generate ink” (although I guess we should now call it “generating electrons”).
I was motivated to do this series because I judged a number of media relations awards programs and was somewhat disheartened by the number of our peers who simply aren’t getting it.
Hopefully this series, with input from professionals like you and others, will help us all raise the bar.
January 27th, 2007 at 8:11 am
Eric,
Thank you for your welcome.
Your last point about raising the bar is exactly on the money…and as professional communicators all of us need to develop still further the ‘political’ will to undertake good research and evaluation of the kind that enables us to understand genuinely just how successful (or otherwise), in exact terms, our campaigns and activities are.
To be blunt, there are still too many in the industry who simply don’t want to know… because properly audited results will not be flattering. If such people can persuade their client or manager to accept the proposition ‘if the campaign looks good, then it must be good’, then they will simply reply “why bother?” when the notion of proper research and evaluation is raised. In agency terms, the planners are at least as important as the creatives, if not more so. If a brilliant campaign fails to reach the required audience, all has been in vain.
If the industry truly welcomes the idea that payment should be by results achieved and not effort expended, then the quality and accuracy of work done can only get better and better. Clients and Executive Boards carry a lot of responsibility - and in their own interest too - to ensure (and provide budget for) robust evaluation solutions.
In closing, I will differ with you just slightly if I may on your point about generating ink and electrons! It seems to me there’s just the merest hint of a little complacency here… that any coverage is good coverage, and that anyone can do it!
Achieving exactly the right tone and content of coverage, in front of exactly the right audience(s) and at exactly the right time is far from easy and yet doing so is critical to the further steps of changing behaviour…and that’s exactly why detailed and accurate media analysis is indeed so important - as diagnostic, for quality assurance and in helping achieve the ultimate business result.
Your series looks to be admirable and I hope many will participate. Thank you for your excellent initiative!
Nicholas Grant
Chairman of the AMEC Board (www.amecorg.com)
CEO Mediatrack Research (www.mediatrack.com)
February 2nd, 2007 at 10:42 am
This is going to be one of the best, ongoing online discussions I’ve ever read, and I’m really looking forward to learning from all of you.
My measurement “program” (underline the quotation marks) is still an infant – but I’d like to know if you think my approach is off the mark or a decent start.
Short background: I’m at Goodwill Industries International, a well-known nonprofit organization. Bet you’re thinking it’s time to clean out your closet (please do!). My problem is that, according to a 2004 natinal (U.S.) public opinion survey, 88 percent of you don’t know that the revenue we earn from selling your donations in Goodwill stores funds job training programs that benefit hundreds of thousands of people each year. In other words, when you hear Goodwill, you know donations and thrift stores. But we want you to know that donations and Goodwill stores get turned into Goodwill programs that help the most vulnerable people in your community find jobs and earn a paycheck so they can support themselves and their families.
So, back to what I measure – As a media relations professional, my job is to get stories in the press (and, as a veteran journalist, to the person who said “above the fold is the journalist’s success,” I would say it’s the success of the communicator as well, because it’s got to be a solid, well-framed story to capture the journalist and the editor’s attention, and a lot of cleverness, patience, persistence and tenacity on the part of the communicator to shepherd the process). I do need to measure the things that show I’m getting stories in the paper. But I look at the totality of these things as indicators to see if I’m making baby steps in the right direction – that is, at some point will I be pushing the needle towards better awareness?
• Number of clips
• Tone
• Message inclusion
• “Mission” mention – i.e. articles about retail/donations that mention what they’re used for. Believe it or not, even reporters don’t care what we do with the stuff you give us! Or don’t think it’s important to share that information with you.
• Media markets (relevant to a specific outreach effort, for example, on a public policy issue, are we hitting markets of the legislators we want to reach?)
• Placement source – i.e. unsolicited reporter call, pitch from my office, etc.
There are also specific efforts where I can see audience reaction – for example, two years ago we fought against a proposal to limit your tax deduction for clothing donations. Aside from the positive legislative outcome, we also saw hundreds of random people log onto our web site and use a web form to send letters of support to their congressional representatives. I’d love to have the resources for a PR campaign which would involve something like a special 800-number so I would know everyone calling is responding to my campaign.
Other efforts – cause-marketing partnerships, advertisting, etc – will help impact mission awareness as well, but I see the above as a start in measuring the earned media piece of the pie.
What do you think, off the mark or a decent start? I welcome your comments!
Christine Nyirjesy Bragale, ABC
February 5th, 2007 at 7:25 am
Christine,
I think the short answer is that you’re off to a wonderful start, so congratulations. What I find really encouraging is that you’re clearly looking on communications research not strictly as post-campaign evaluative, but also pre-strategy formative. Indeed PR can and should be a management science, not just a dark art tactical afterthought.
The indicators that you’ve listed could be tracked as part of an on-going media content analysis project. You might consider reaching out to a vendor in that business. (In the interest of total transparency, we are in that business, largely in Canada, but there are a number of other excellent providers in the U.S.)
Content analysis suppliers can track literally hundreds of variables for you and they will work with you to find those most meaningful for your organization. You might also want to look at variables such as hot button issues, executive profile, competative/peer organizations, consumer mention/sentiment and a sort of framing variable that looks to categorize how the story mentioning your organization is framed/presented.
You hit the nail on the head with reference to ‘moving the needle’ and ‘awareness.’ You noted that you;d undertaken some awareness polling in 2004. Communications and marketing people in your organization may want to consider updating that research to see what progress you’ve made. Costs could be shared among all departments that would find such data useful. You might want to consider adding some basic media consumption questions to the survey such that, come results time, media content analysis results could be correlated with awareness results. To do that, it’s best to have a media content analysis supplier play some consultative role with you and your polling provider at the design stage.
Of course, all this begs the question of direct proof and causality and things like accounting for PR’s influence versus marketing, among others. We know that PR does not (nor should it) occur in a vacuum. And decades of academic research on the topic tells us that people just aren’t as easily and diretly influenced by communications efforts exclusively. That said, there is progress being made on statistical modelling employed and, at worst (or best depending on how you look at it), you may find it another useful analytical tool to add to your list of indicators. Not the end all be all, but another tool among many.
This would a be a bit trickier, but you might also consider looking at correlating donation activity during peak campaign times to the media content analysis. But, again the same uncertainty and debate as it relates to causality remains.
Food for thought.
Cheers
Alan Chumley
VP, Cormex Research
http://www.cormex.com
February 5th, 2007 at 7:33 am
Christine,
Congrats on wanting to make your measurement program a solid one.
I think we often confuse various measurements when it comes to media.
First, we can measure or evaluate the process of our media relations efforts: are we operating in the most efficient way possible; do we have the tools/software we need to run an effective media relations shop; is our approval process getting in the way of our media plan…those are a few process measures.
We can also measure/evaluate output: are we sending out the right information to the right media in the right way at the right time for the right reason(s). Too often, as you know, there are requests (demands?) to “get some media coverage” on something that just isn’t suited for media. Surveying editors (as part of an omnibus research effort) will help answer questions about both your output and your relationship with media representatives.
We can finely dice measuring output as it relates to outcome: did those news releases and pitches result in media coverage — where, when, how, above the fold, etc.
And then we can measure bottom line outcome: did people write their congressional representative? Did more donations come in to Goodwill? Did more people bring items to Goodwill because they know how the money is used? Did the toll-free telephone number light up after a certain media push?
I understand the pressure(s) that media relations professionals have (or believe they have) and I believe that we rely too much on formulas and quantity to help alleviate that pressure.
You’re already pushing the needle towards awareness and will need good polling before and after a media campaign to know if you’ve increased the awareness on your topic. Until then, keep learning and asking…this is a community of individuals who understand where you are and want to help.
Best wishes,
Wilma Mathews
P.S. Thanks for your support on the tax issues related to non-cash donations. Some minor restrictions have been put in place, as I noted when working on my material yesterday to send to my accountant. As usual, the new language is just ambiguous enough to cause problems.
February 8th, 2007 at 11:08 am
Alan, Wilma - many thanks to you both for your great ideas. I will keep you posted (and ask for your good counsel) as I move forward!
Christine
Christine Nyirjesy Bragale, ABC
P.S. to Wilma: we have a valuation guide at http://www.goodwillpromo.org that might help!
March 1st, 2007 at 9:01 am
Best thread in a long time…
My latest measurement is “slices.”
We did a TV show about bagels - Since some of the people in this thread are Canadians, Breakfast Television, CITY TV, a remote with Jennifer Valentyne — and the next day learned that the shop owner ran out of Montreal smoked meat that afternoon, after Jennifer ate a smoked meat sandwich on camera around 8:15 in the morning.
Or cream cheese. That almost disappeared, too.
Plus, the client gave me a blogo’d cap, and over the next few days several stangers told me they’d seen the company on television, and then gone to buy bagels.
Or dog treats — we got talking to a store owner (sells pet stuff and we were in to buy dog treats) about publicity and she mentioned she’d seen a TV show that was so compelling her husband drove 20 miles to buy stuff to take to his family. “Our” bagels.
Or, as measurement, reduced leftovers. The client takes the leftover bagels to several organizations that feed less fortunate folks, and the numbers of free bagels dropped so much that he started baking just for them.
Or, as measurement, the cash register.
The good part about some publicity is that you can isolate the effect, as per a retailer on television. Harder when there’s a coinciding advertising program, a big public exhibition, free tastings at Main Street and First Avenue, and 20 percent off coupons in the stores.
BAK