IABC Measurement Commons

Social measurement is based around being able to codify behaviors, opinions and descriptors of our reality. Our new social media is presenting a great deal of challenges to the codifying of relevant information. The variety of sources and our ability to define the relevant from irrelevant is subjective and best. My advice would be to remember that all measurement has flaws and our goal is to gather the best evidence available given our resources and the nature of the decisions we need to make. Frank Ovaitt President and CEO Institute for Public Relations reminded us where to start, “Like any communications effort, you need to start with clear objectives to have something to measure.” Recently Katie Paine wrote a paper that reminds us of the fundamentals of communication research and applies this wisdom to social media. Some of the highlights that I thought were of particular value to our communication strategies and measurement plans:

• We can not control what other people say and blogs are the best example yet of how futile it is to try and manage by control.
• We may not yet know how the social media is impacting our work but we need to actively listen to be able to adapt to the changing uses of the internet.
• Measurement is still as simple as defining outputs, outtakes and outcomes and then gathering the information to support decision making.
• Our communication efforts are based around influencing attitudes and behaviors.
• Our ability to understand why certain voices on the web have credibility and influence will determine our future communication strategies.

Chime on in. Are you monitoring and measuring the impact of web conversations on your organizations and on your communication strategies?

So let me ask you: is it better to generate a high volume of media coverage in a variety of sources, or to focus on fewer clips in highly targeted media? Is it better to have negative publicity than no publicity? Is corporate reputation driven more by good products and performance (even if no one hears about them?), or by news coverage of those facts? In other words, if a tree falls in the forest and no one hears it, does it still make a sound?

These are some of the questions Dr. Don Stacks, Dr. David Michaelson and I wrestle with in our new white paper, “Exploring the Link Between Volume of Media Coverage & Business Outcomes,“ on the IPR Commission on PR Measurement & Evaluation website. This seems such an elementary concept, but apparently little is available in current literature with hard data. So, we looked at the effects of volume alone; tonality-qualified volume, and message-qualified volume in three case studies. Of course, there are many other variables, but these seemed to offer a good beginning point.

At the end of the day, it appears that “more IS better” if it is at least neutral-to-positive in tone, and more so if key messages truly resound with the targeted audience. On the other hand, negative news is NOT preferable to NO news; scoring a few key placements may NOT be enough; and “getting all five key messages out there” may not matter.

This is interesting, since there’s been a lot of focus in recent years on forgoing the old ‘thud factor’ in favor of fewer targeted placements, and to consider a campaign a success if your report card shows you scored a high percentage of stories with “all five key messages.” Certainly, our job in media relations is easier if we can get away with doing less for our clients, and if we can utilize measurement reports with soft “proof of performance” scores like — “gee whiz, 90% of my coverage uses all the key messages.” It looks good on paper, but do you know if that coverage really correlates to outcomes like sales, and if so, do you know which message really worked?

Don, David and I are tackling “share” of volume next in a follow-up to this paper, which should be even more enlightening. Anyway, I’d love to hear your thoughts. Be gentle!

If a news story about an organization is copied and posted on that organization’s website, is it still news or is it marketing?

Okay, we’re not allowed to flog our own wares on this blog, but I trust it’s okay to flog someone else’s! Since so many of us wax on about the challenges of figuring ROI on communications efforts, I would be remiss not to share a cool new tool that one of our fellow bloggers, Merry Elrick, has created to make this a whole lot easier. Merry kindly walked me through her system a month or so ago, and I found it fascinating, and very different from anything I’ve seen in the PR measurement space thus far.

Merry’s web-based system, called DataDriven MarCom™, is a one-stop portal in which to store all costs related to each tactic in your marketing communications arsenal, along with (most importantly) details of every resulting lead and/or sale from each of those efforts. In other words, for every campaign, you (the communicator) would log-in costs for every ad, every direct mail piece, every media campaign, and so on. Then, as leads roll-in, you work in tandem with your sales department to insert contact information for leads generated by each campaign. The sales person assigned to each lead works it, and logs in the appropriate sales revenues if/when a sale results. (If your company is already working with a CRM package, I believe there is a way to link databases.)

A suite of report options allows you to quickly see which campaigns and tactics are working, and which are not, by comparing the costs of each effort against the numbers of leads and revenues resulting. Since the tracking is done in ‘real time,’ the communicator is able to change strategy and/or tactics quickly when necessary.

Anyway, it’s darned clever! I have to congratulate Merry on her innovativeness, (and bravery in expanding her career as a counselor into software entrepreneurship), and hope others on this blog will give it a whirl. The thinking behind the drill-downs is an exercise in good measurement, in my opinion.

I went to see a wonderful speaker today, Ryan Walter ex NHL player and leadership thought leader. His topic was the hunger spirit. He talked about 7 key aspects of leadership that we can all use personally and with our teams. The one point I would like to highlight is creating the frame.

The challenge in most ventures is not what to ask or include. The challenge is what we should leave out. The frame is where our leadership comes in. We are all creating a picture of what is relevant to our destination. We create steps towards a destination or away from it. This is the context that we should use to plan our measurements.

Surveys should only ask about the important issues. They act as a communication and must be seen that way. You are engaging your audience in what is important to you. If you don’t frame that discussion, it will be confusing and could offer ambiguous or non actionable data. Frame your research well and you start a discussion around what is most important. Framing will lead towards your destination and involve those most important to your success.

Great book on the topic

A view from Brisbane, Australia. Newsletters with case studies and good content.

To have successful change we need leadership. The one common element that defines leaders is that they have followers. To foster an organizational culture that maintains sustainable leadership employees have to be willing followers. Some of the key feelings employees need to sustain leadership is trust, ownership of decisions and a sense of self control. Traditional models of social action research have been used to achieve these ends. Practitioners have used qualitative participation, through workshops and focus groups. Consultants have used these practices and achieved limited results. The challenge with this model is the limited scalability. The amount of human and monetary resources needed to have broad-based participation is rarely practical, and as a result this model focuses on manager participation and perceived thought leaders. In our communities we would not accept this as full participation, and in our organizations we don’t have to accept this either.

Using employee surveys strategically we can achieve a much fuller participation. The researcher may still use a mixed methodology where workshops and focus groups inform the survey design. Planning is supported by survey results. The important principal to remember is how can we in a practical way involve all employees throughout our key decision making processes? This does not suggest asking every employee about everything, but rather developing a strategy where all employees are asked about important business issues once or twice a year. Equal access and participation will improve the key dynamics of leadership throughout the organizations, and communicators can offer leadership in advocating for these new channels of participation.

The social research models this concept is based on: Action Research
Action Science

In an earlier blog, I suggested we needed to be brave to conduct research. This article, “A major mistake that managers make” provides insight into why we may not want to gather possibly negative feedback. It also highlights the risks our organizations take when we don’t seek criticism of our decisions.
Ackoffcenter Blog Link

“But the only kind of mistake their organizations take into account are errors of commission. Then, to avoid censure one must try to minimize such errors. This is accomplished when managers do as little as possible. This is seldom a decision made consciously; rather it is a culturally imposed disposition of which most are unconscious.”
(Russell L. Ackoff, p.7)

In the past week I reviewed an opinion piece by Tudor Williams that compares customized to standardized employee surveys. Tudor’s conclusions are consistent with the need to customize surveys to meet individual situations. He may be biased as I am, we both design customized research strategies. We come by our bias naturally, after creating a standardized employee retention survey in the late 90’s.

In partnership with some wonderful professionals we developed our own employee retention tool. We never marketed it. Every time we met with a client, they had unique challenges: labour markets, professional demand, competencies, financial resources, size, locations, etc. Each situation needed a customized approach. While people do have some universal needs, we all have unique circumstances.

We start our research assignments by asking about our client’s strategic plans, priorities and what their big challenges are. To design our research/measurement strategies we seek to inform challenges and evaluate theories about client communication practices.

The nature of measurement informs this process.
“In research, measurement takes on a somewhat different meaning: Measurement is limiting the data of any phenomenon - substantial or insubstantial - so that those data may be interpreted and, ultimately, compared to an acceptable qualitative or quantitative standard.”
Paul D.Leedy and Jeanne Ellis Ormrod

In the market today we see,
Fortune’s 100 best companies to work for (By State)
In Canada, the Macleans 100 best companies to work for
In my home province of British Columbia, the BC Business 50 best to work for

Why do companies participate in these studies? As communicators we run the risk of having our organizations building incentives to score high on a survey to move up a list and not address our organizational issues.

Do these generalized measurements really offer a competitive advantage? Or do these assessments build apathy with our employees?

Respond with your experiences. Did you use an off the shelf tool or customized approach? What have been the positives or negatives of your approach?

Here’s something really new: measuring the ABSENCE of core messages to gauge the effect of public relations! Let me share some highlights from a new white paper, “A New Model For Media Content Analysis,” by David Michaelson, David Michaelson & Company, LLC, and Toni Griffin of MetLife, that is available on the IPR Commission for PR Measurement and Evaluation website. While it isn’t necessarily a practical approach for every-day use, it is very compelling intellectually, and should be seriously considered - especially for complex subject matter.

David’s and Toni’s hypothesis is that measurement isn’t funded because of a “lack of interest, lack of knowledge, lack of budget or a generalized fear of measurement; rather, it is a lack of usefulness of these basic research measures.”

To prove their hypothesis, they first analyze the eight commonly-used methods for media measurement today, which include clip counting, circulation and readership analysis, AVE, simple content analysis, message analysis, tonality analysis, prominence anlaysis, quality analysis (single-score indices) and competitive analysis, showing inherent weaknesses in every approach. Most interesting to me is that even scoring messages for tonality fails to offer a “proper situation diagnosis or prescription for a solution that is tied directly to communications objectives.”

Why? They cite two fatal flaws: 1) the absence of an analytic structure that determines the accuracy of coverage overall and the messages contained; and 2) the failure to link analysis to communications objectives and PR messages.

“Message accuracy” is:

” … based on an analysis of four basic elements: the inclusion of basic facts, the inclusion of misstatements about these basic facts, the inclusion of incomplete, deceptive or misleading information that biases the interpretation about basic facts and the ommission of basic facts.”

So, to analyze a story for these factors, you have to study the context of the article, and see if anything critical is missing that may have rendered the story a non-starter.

As for not linking messages to communications objectives, they challenge us to figure out where, exactly, in the Communications Lifecycle our target audience (and the media) may be. You all know this Cycle includes awareness, knowledge, interest and intent. If we shoot messages out there without knowing this, we will miss the target entirely … and thus, our objectives.

Finally, David and Toni share a couple mind-blowing case studies for MetLife that proved their hypothesis … and that lead to a revamping of PR strategy and tactics at MetLife that included in-depth education of the media (which didn’t understand half of MetLife’s subject matter) and a much better targeting of messages. In one of the cases, they found:

” … between 60 and 85% of the articles on the key issues of concern to MetLife included an error in reporting, a misrepresentation of key information or an omission of basic information that should have appeared in the contexts of the articles in question. By concentrating media relations efforts on those reporters and publications where these errors and ommission in reporting occurred, the eventual result was a significant decline in the proportion of articles with either errors or omissions, as well as an overall increase in the number of articles by 45% on the issue at hand.”

Of course, the problem with all of this is you need really smart people to analyze your coverage this way … and therefore, it is more expensive. However, David and Toni challenge this by saying that the return on such an investment is a significantly improved quality of media relations, and greater chance of success against objectives.

Yeah, baby!

In a recent BtoB article by Kate Maddox, “Measurement calls for more resources,” she cites a study that indicates marketers place a high priority on marketing measurement and analytics.

And last week the Association of National Advertisers released a study that found accountability is the No. 1 priority for senior marketing executives.

But if measurement is so important, then who can explain these astonishing stats in Maddox’s article:

In a recent survey of marketing executives, 47% said increasing brand value is a priority, but only 19% have a process in place to track this.

In the same survey, 48% of respondents are expected to measure ROI on individual campaigns but don’t have a process for doing this, compared with only 9% of companies that are expected to measure ROI on individual campaigns and have a process to do this. The remaining 43% are not expected to measure ROI on campaigns.

There may be a lot of resources available for those who want to measure, as Angie Jeffrey said in an earlier blog, but there’s a big gap between those who say they want to use them and those who actually do.

What’s up with that?

The extent that we benefit from measurement has a strong correlation to defining what our problems are. When structuring your measurement plan ask what are the emerging challenges. Your goal for measurement should address the premise of your challenge. To create a goal restates your challenge into a solution statement. Then define the issues that are measurable. The issues become your objectives.

The next steps in the research process:
Develop a methodology that will deliver evidence for your decision making.
Gather your data.
Analysis your data.
Contextualize the data.
Make recommendations for action.

Now test your assumptions and enjoy finding solutions.

Corporate reputation

A good corporate reputation is a challenge that many organizations expect communicators to manage. Ipsos has just released a series of articles on the global practice of corporate reputation.