IABC Measurement Commons

Back in late December, I wrote a post for my blog about the disagreement between Katie Paine and John Wagner over the value of measuring communication. For some unknown reason, the Ragan Report just covered this in its latest issue. Go figure. But Delahaye President Mark Weiner thought it was interesting enough to comment on, and his comment was inspiring enough that I thought it was worth reproducing here (with his permission):

First: The IPR’s position on AVEs is actually relatively moderate as compared to what this discussion suggests. Read the white paper (PDF file) written by Bruce Jeffries-Fox. While I believe that AVEs are an incomplete and inadequate measure most of the time, I’ve seen them correlate to sales when no other PR measure could. Even if they are not the preferred measure and even if I believe that AVEs should be highly qualifed in their use if they are used at all, I’d rather be partially right than totally in the dark. AVEs do provide some gauge to the amount of coverage (minutes/column inches) and the reach/prestige of the medium. Even an incomplete measure can be abused and I think this is what so many of us find offensive.

Second, the irony is that AVEs greatly undervalue PR’s unique role within the marketing mix. As the research provider who worked with P&G and helped to create PRevaluate, and after having done similar marketing mix analysis in dozens of categories for scores of companies over the years, we’ve discovered that PR consistantly delivers the best ROI of any marketing agent. On average, mass marketing advertising delivers about $1.25 on a dollar; price promotions only $.75 on the dollar and PR between $3.00 and $8.00 on the dollar. The trouble is that most companies don’t use marketing mix modeling, and even those who do prefer to track only those marketing elements on which they spend the most money (TV and promotions) rather on those elements which deliver the best ROI (PR). When viewed within the context of the complete marketing mix, PR is the only marketing option that provides a lift to all other forms of marketing (meaning that when the PR is good, the advertising is more effective, the direct marketing more efficient, etc). It’s what every PR person hopes is true, and maybe in their gut believes is true…now the truth is being revealed time and time again.

The question to ask is not “what will marketers do with this information,” but rather “what will the PR profession do with this information?” Will this be another lost opportunity for PR or will the leaders of our profession and the professional associations which represent us break free from conventional wisdom? Will they take the lead in promoting PR not because it “generates buzz” or “delivers a high equivelant ad value” but because PR delivers on meaningful business objectives? The story of P&G, Wachovia and Miller Brewing broke in the advertising and marketing trades rather than the PR trades. The evidence of PR’s unique power to drive businesses forward should be our call to action: Instead of squabbling among ourselves and finger-pointing, we should unite to promote PR simply because it works in meaningful and measurable ways.

Social connectedness leads to employee engagement and satisfaction. Our communication strategies can increase social awareness and create platforms for relationship building. Relationships enable change, retention and action. These business results can be measured.

Leading the Social Side of Change By: Carol Kinsey Goman, Ph.D.

Community engagement more important to happiness and productivity than material wealth Presented By:John F. Helliwell

How do you bring people together and how do you measure it?

As communicators we do some fantastic work and when it is measured we demonstrates the impact we have on success. Southwest Airlines has been successful in a number of ways and they are one of my favorite case studies. They are also winners of last years Golden Ruler. The Institute of Public Relations awards the Golden Ruler and shares the winning case study for other communicators to learn from. Southwest Airlines study of press releases and search engines details innovation and is a great case for measuring communication results.

Southwest Airlines with SEO-PR, “You Are Now Free to Link PR and Sales”

Professions can mirror each others experience within organizations. With this in mind I like to keep track of what CIO magazine researches about our friends in the IT field. They have captured the Chief Information Officer Title, and communicators should examine how they became the information advisors to our organizations. CIO just released a study that describes the ups and downs of the IT profession in the last year.

State of the CIO 2006

I couldn’t let St Patrick’s Day go by without finding out how it has impacted our behaviour.

10 Percent of Workers Admit to Drinking on Company Time on St. Patrick’s Day

Cheers!

A lot of communicators ask me where to start with their measurements, how to connect what we do to the outcomes our organizations are trying to achieve, and how to take the right amount of credit for communication’s impact versus other things the organization is doing to achieve the same outcome.

To link communication impact to a business result, I work backwards from outcomes in developing the communication approach:
1. Identify a measurable business outcome that communication can impact.
2. Break out specific behaviors/actions for different key stakeholders that need to change in order to achieve the business outcome. (This is the step most communication plans skip over.)
3. Use qualitative research with the affected stakeholders to identify what knowledge and attitude changes are required before they would change their behaviors, and which communication channels would be preferred for knowledge and for attitude inputs.
4. Develop your communication tactical plan with specific messages and channels you will use to influence #3 to change #2 that will result in #1.

Then you switch to quantitative measurement mode. You plug into the measurements others in your organization are already doing to track business results (#1) and key audience behaviors (#2) against your own inputs. You as a communicator measure changes in knowledge and attitudes and actual exposure to the preferred channels (#3), and track the actual volume and frequency of content of the needed messages you’re sending out through the preferred channels (#4).

You can isolate the impact communication has versus other organizational inputs in several ways:
1. Do a pilot test where communication is the only variable and all the other inputs are the same for all groups.
2. Track the timing of your communication inputs versus the timing of improvements in knowledge and attitudes and behaviors. A pattern of spikes and valleys that matches the inputs and outcomes becomes a compelling argument for the amount of impact communication has.
3. Ask the people whose behavior changed to what extent they credit the communications they were exposed to for their behavior change.

On a regular basis I get questions about how we are going to ensure security of survey data. The majority of our work involves employee surveys, and with these surveys I would say don’t use a password to enter the survey.

1. Candid feedback is anonymous and a password entrance leaves the impression that participants may be tracked.
2. We ask questions about trust of senior leadership all the time and using passwords undermines this message.
3. Every step to participate in a survey lowers the response rate.
4. We can track the responses coming in and track if “ballot stuffing” has occurred without using passwords.
5. We use a simple cookie to discourage multiple submissions.

Offering rewards for participation encourages multiple submissions. The value transaction for employee surveys should be the impact the results will have them. The key to validity is to encourage ownership of the process and the results.

Passwords may need to be used

Public internet surveys may need different levels of security. I have a good friend that put together a survey on a local chamber of commerce website. It was an issue survey. To publicize the survey he sent out emails to his contact list. His hope was to offer support to his cause from the business community. Unfortunately for him the results were unrealistically supportive and completely useless. I reminded him to call me the next time he thought of doing a survey.

In his case, they needed the chamber to run a secure survey to just the Chamber members. In this case a password would be important to maintain audience control. In community relations issues political activism can cause your data to be at risk.

How we protect our data is situation dependent, but we must also look at how we do the research communicates to our audience. What has been your experience around securing employee survey data?

Completely unrelated.
I came across a great study on linking employee moral with stock price.

In a previous post, Angie Jeffrey is understandably overwhelmed by measurement options. She is right when she says we have to do some serious thinking to determine what our goals are. And if we do, we’ll discover that our goals are aligned with, or the same as, our organization’s goals. And that is almost always related in some way to bringing in revenue, adding value or increasing profitability. In other words, money.

We have been using soft measures, like awareness, too long, according to the Association of National Advertisers. We either need to be contributing to short-term cash flows or long-term shareholder value, and we need to measure our return on investment (ROI).

As I mention in an article in the measurement issue of CW Online, when you know ROI of your marketing efforts, you also have access to data that will help you understand a great deal more. You can:

Provide a solid rationale for upcoming budgets.
Compare actual budgets to costs-to-date.
Compare ROI of each component of your marketing communications mix to see what works best.
Make adjustments to your existing program to get the greatest impact for your communications dollar.

In short, you can manage your budget as the investment it really is, and you can achieve more than justification of your marketing efforts–you can increase the return on your marketing investment.

This requires a certain amount of courage, as Ryan Williams points out in a previous post. When we determine ROI, we’ll expose the components of our marketing mix that are not producing as well as others. But only then will we know what we need to change. And only then can we improve.

I am a survey vendor and a consultant. Measurement is my gig. I see all the pros of doing good research. Codifying data, organizing it, making information into knowledge and applying it to my projects is what I was taught when I started my career. If I started in a corporation would I feel the same way? Done poorly measurement can be risky.

Is measurement too risky in today’s competitive environment? If I survey and the results are negative what can I do? I may embarrass my boss or be on the hook for the outcome.

So why don’t we measure more? Is it budget, competencies, time or the risk of accountability?

Do we need to be courageous to measure?

First, hello to all! I confess to being a blogging newbie, so have been avidly reading your posts and comments in an effort to get into the swing. Good questions and comments to date.

Second, I’ve just got to share my new view of PR’s measurement challenge since spending what felt like months reviewing white papers, speeches, articles and the web in preparation for an upcoming measurement speech at Bulldog’s Media Relations 2006. Bulldog wanted an industry scan of the very latest and greatest tools and techniques, so I ended up (exhausted!) with a seven-page spreadsheet of options.

Here’s the thing: we now have SO much information, SO many tools, SO many vendors and SO many approaches, that I am at a loss to understand why anyone is NOT measuring today. From web-based surveys to website tracking/analysis services; from sponsorship research to loyalty and brand-equity services; from CGM media analysis to traditional monitoring; and from 24/7 measurement platforms to cheap DIY tools … there are answers out there for just about any situation — including many FREE offerings (particularly for e-surveys and website measurement).

I’ve concluded that the biggest callenge facing us (besides getting through the long lists of options!) is thinking through the mixing and matching of output, outgrowth and outcome measures so that you end up with a truly integrated or linked set of results. Is this a left-brain or right-brain function? Test yourself sometime over a glass of wine, with a few different types of objectives … and see what you come up with. I found one glass of wine wasn’t enough …

I hate to agree with a heretic, as Shel Holtz refers to John Wagner, but John is right about one thing. Big brand name companies are not generating meaningful metrics, at least not when it comes to B2B integrated marketing communications. I’m continually stunned by this, but it’s true. Even the really big companies that profess to be all over metrics aren’t doing it.

Measurement guru Pat LaPointe says it best: “Most marketing organizations spend far too much time and precious resources answering questions that don’t generate any significant insights into the business.” Why is that? Well, there are a jillion reasons, not the least of which is defining what meaningful metrics are.

But we can agree that ROI is a pretty good number, right? In fact, for many CEOs, it’s the only number they care about. (When it comes to marketing, they should care about more than that, but that’s another blog.)

Yet here’s some sobering stuff. Forrester Research conducted a study of marketers, 66.5% of whom said measuring marketing ROI is important or very important, yet 59.5% reported being dissatisfied with their company’s ability to measure marketing ROI. And 51.1% said they were dissatisfied with their company’s ability to agree on a definition of marketing ROI. Yikes! No wonder we can’t measure it if we don’t know what it is! (Hint: It’s not counting click-throughs to your website.)

There’s no silver bullet, but we can achieve ROI accountability for the most part. What we can’t do is continue on as we have, inspiring comments like this in a recent Wall Street Journal article (by John Quelch, October 12, 2005, European edition):

Many marketing managers are failing their employers. They are often creative right-brain thinkers who can dream up campaigns to drive top-line sales, but they show little interest in the balance sheet impact of their promotional programs. Such marketers lack the quantitiative, analytical skills necessary to drive marketing productivity…

Anybody have a boss who doesn’t read the Wall Street Journal?

Judy Jones, from New York IABC, posted the following as a comment in the Employee Communications Commons, but it seemed appropriate to make it a full-blown post here:

Okay, I’m going to bring up a really dreary subject. (I can hear the virtual groans already.) Let’s talk about measuring Employee Communications or Internal Communications. Here’s the problems I’ve got: There are different productivity measures around the Company, so I can’t seem to consistently link employee communications to productivity. Margin, growth - great metrics but they’re influenced by a number of factors and I’m not successfully selling the case that employee communications can take credit; I have sold the idea that an initiative (based on multiple factors, like training, corporate communications, employee communications, etc.) can take credit. I don’t have a talent management metric to hang my hat on. HELP.