Data does NOT Equal People

We marketers love data. We treat it like a holy grail: a thing to be worshipped. But we’re praying at the wrong altar. Or, at the very least, we’re praying at a misleading altar.

Data is the digital residue of behavior. It is the contrails of customer intent — a thin, wispy proxy for the rich bandwidth of the real world. It does have a purpose, but it should be just one tool in a marketer’s toolbox. Unfortunately, we tend to use it as a Swiss army knife, thinking it’s the only tool we need.

The problem is that data is seductive. It’s pliable and reliable, luring us into manipulation because it’s so easy to do. It can be twisted and molded with algorithms and spreadsheets.

But it’s also sterile. There is a reason people don’t fit nicely into spreadsheets. There are simply not enough dimensions and nuances to accommodate real human behavior.

Data is great for answering the questions “what,” “who,” “when” and “where.” But they are all glimpses of what has happened. Stopping here is like navigating through the rear-view mirror.

Data seldom yields the answer to “why.” But it’s why that makes the magic happen, that gives us an empathetic understanding that helps us reliably predict future behaviors.

Uncovering the what, who, when and where makes us good marketers. But it’s “why” that makes us great. It’s knowing why that allows us to connect the distal dots, hacking out the hypotheses that can take us forward in the leaps required by truly great marketing. As Tom Goodwin, the author of “Digital Darwinism,” said in a recent post, “What digital has done well is have enough of a data trail to claim, not create, success.”

We as marketers have to resist stopping at the data. We have to keep pursuing why.

Here’s one example from my own experience. Some years ago, my agency did an eye-tracking study that looked at gender differences in how we navigate websites.

For me, the most interesting finding to fall out of the data was that females spent a lot more time than males looking at a website’s “hero” shot, especially if it was a picture that had faces in it. Males quickly scanned the picture, but then immediately moved their eyes up to the navigation menu and started scanning the options there. Females lingered on the graphic and then moved on to scan text immediately adjacent to it.

Now, I could have stopped at “who” and “what,” which in itself would have been a pretty interesting finding. But I wanted to know “why.” And that’s where things started to get messy.

To start to understand why, you have to rely on feelings and intuition. You also have to accept that you probably won’t arrive at a definitive answer. “Why” lives in the realm of “wicked” problems, which I defined in a previous column as “questions that can’t be answered by yes or no — the answer always seems to be maybe.  There is no linear path to solve them. You just keep going in loops, hopefully getting closer to an answer but never quite arriving at one. Usually, the optimal solution to a wicked problem is ‘good enough – for now.’”

The answer to why males scan a website differently than females is buried in a maze of evolutionary biology, social norms and cognitive heuristics. It probably has something to do with wayfinding strategies and hardwired biases. It won’t just “fall out” of data because it’s not in the data to begin with.

Even half-right “why” answers often take months or even years of diligent pursuit to reveal themselves. Given that, I understand why it’s easier to just focus on the data. It will get you to “good,” and maybe that’s enough.

Unless, of course, you’re aiming to “put a ding in the universe,” as Steve Jobs said in an inspirational commencement speech at Stanford University. Then you have to shoot for great.

What the Hell is “Time Spent” with Advertising Anyway?

Over at MediaPost’s Research Intelligencer, Joe Mandese is running a series of columns that are digging into a couple of questions:

  • How much time are consumers spending with advertising; and,
  • How much is that time worth.

The quick answers are 1.84 hours daily and about $3.40 per hour.

Although Joe readily admits that these are ‘back of the envelope” calculations, regular Mediapost reader and commentator Ed Papazian points out a gaping hole in the logic of these questions: an hour of being exposed to ads does not equal an hour spent with those ads and it certainly doesn’t mean an hour being aware of the ads.

Ignoring this fundamental glitch is symptomatic of the conceit of the advertising business in general. They believe there is a value exchange possible where paying consumers to watch advertising is related to the effectiveness of that advertising. The oversimplification required to rationalize this exchange is staggering. It essentially ignores the fields of cognitive psychology and neuroscience. It assumes that audience attention is a simple door that can be opened if only the price is right.

It just isn’t that simple.

Let’s go back to the concept of time spent with media. There are many studies done that quantify this. But the simple truth is that media is too big a catchall category to make this quantification meaningful. We’re not even attempting to compare apples and oranges. We’re comparing an apple, a jigsaw and a meteor. The cognitive variations alone in how we consume media are immense.

And while I’m on a rant, let’s nuke the term “consumption” all together, shall we? It’s probably the most misleading word ever coined to define our relationship with media. We don’t consume media any more than we consume our physical environment. It is an informational context within which we function. We interact with aspects of it with varying degrees of intention. Trying to measure all these interactions with a single yardstick is the same as trying to measure our physical interactions with water, oxygen, gravity and an apple tree by the same criterion.

Even trying to dig into this question has a major methodological flaw – we almost never think about advertising. It is usually forced on our consciousness. So to use a research tools like a survey – requiring respondents to actively consider their response – to explore our subconscious relationship with advertising is like using a banana to drive a nail. It’s the wrong tool for the job. It’s the same as me asking you how much you would pay per hour to have access to gravity.

This current fervor all comes from a prediction from Publicis Groupe Chief Growth Officer Rishad Tobaccowala that the supply of consumer attention would erode by 20% to 30% in the next five years. Tobaccowala – by putting a number to attention – led to the mistaken belief that it’s something that could be managed by the industry. The attention of your audience isn’t slipping away because advertising and media buying was mismanaged. It’s slipping away because your audience now has choices, and some of those choices don’t include advertising. Let’s just admit the obvious. People don’t want advertising. We only put up with advertising when we have no choice.

“But wait,” the ad industry is quick to protest, “In surveys people say they are willing to have ads in return for free access to media. In fact, almost 80% of respondents in a recent survey said that they prefer the ad-supported model!”

Again, we have the methodological fly in the ointment. We’re asking people to stop and think about something they never stop and think about. You’re not going to get the right answer. A better answer would be to think about what happens when you get the pop up when you go to a news site with your ad-blocker on. “Hey,” it says, “We notice you’re using an ad-blocker.” If you have the option of turning the ad-blocker off to see the article or just clicking a link that let’s you see it anyway, which are you going to choose? That’s what I thought. And you’re probably in the ad business. It pays your mortgage.

Look, I get that the ad business is in crisis. And I also understand why the industry is motivated to find an answer. But the complexity of the issue in front of us is staggering and no one is served well by oversimplifying it down to putting a price tag on our attention. We have to understand that we’re in an industry where – given the choice – people would rather not have anything to do with us. Unless we do that, we’ll just be making the same mistakes over and over again.



The Rise of the Audience Marketplace

Far be it from me to let a theme go before it has been thoroughly beaten to the ground. This column has hosted a lot of speculation on the future of advertising and media buying and today, I’ll continue in that theme.

First, let’s return to a column I wrote almost a month ago about the future of advertising. This was a spin-off on a column penned by Gary Milner – The End of Advertising as We Know It. In it, Gary made a prediction: “I see the rise of a global media hub, like a stock exchange, which will become responsible for transacting all digital programmatic buys.”

Gary talked about the possible reversal of fragmentation of markets by channel and geographic area due to the potential centralization of digital media purchasing. But I see it a little differently than Gary. I don’t see the creation of a media hub – or, at least – that wouldn’t be the end goal. Media would simply be the means to the end. I do see the creation of an audience market based on available data. Actually, even an audience would only be the means to an end. Ultimately, we’re buying one thing – attention. Then it’s our job to create engagement.

The Advertising Research Foundation has been struggling with measuring engagement for a long time now. But it’s because they were trying to measure engagement on a channel-by-channel basis and that’s just not how the world works anymore. Take search, for example. Search is highly effective at advertising, but it’s not engaging. It’s a connecting medium. It enables engagement, but it doesn’t deliver it.

We talk multi-channel a lot, but we talk about it like the holy grail. The grail in this cause is an audience that is likely to give us their attention and once they do that – is likely to become engaged with our message. The multi-channel path to this audience is really inconsequential. We only talk about multi-channel now because we’re stopping short of the real goal, connecting with that audience. What advertising needs to do is give us accurate indicators of those two likelihoods: how likely are they to give us their attention and what is their potential proclivity towards our offer. The future of advertising is in assembling audiences – no matter what the channel – that are at a point where they are interested in the message we have to deliver.

This is where the digitization of media becomes interesting. It’s not because it’s aggregating into a single potential buying point – it’s because it’s allowing us to parallel a single prospect along a path of persuasion, getting important feedback data along the way. In this definition, audience isn’t a static snapshot in time. It becomes an evolving, iterative entity. We have always looked at advertising on an exposure-by-exposure basis. But if we start thinking about persuading an audience that paradigm needs to be shifted. We have to think about having the right conversation, regardless of the channel that happens to be in use at the time.

Our concept of media happens to carry a lot of baggage. In our minds, media is inextricably linked to channel. So when we think media, we are really thinking channels. And, if we believe Marshall McLuhan, the medium dictates the message. But while media has undergone intense fragmentation they’ve also become much more measurable and – thereby – more accountable. We know more than ever about who lies on the other side of a digital medium thanks to an ever increasing amount of shared data. That data is what will drive the advertising marketplace of the future. It’s not about media – it’s about audience.

In the market I envision, you would specify your audience requirements. The criteria used would not be so much our typical segmentations – demography or geography for example. These have always just been proxies for what we really care about; their beliefs about our product and predicted buying behaviors. I believe that thanks to ever increasing amounts of data we’re going to make great strides in understanding the psychology of consumerism. These  will be foundational in the audience marketplace of the future. Predictive marketing will become more and more accurate and allow for increasingly precise targeting on a number of behavioral criteria.

Individual channels will become as irrelevant as the manufacturer that supplies the shock absorbers and tie rods in your new BMW. They will simply be grist for the mill in the audience marketplace. Mar-tech and ever smarter algorithms will do the channel selection and media buying in the background. All you’ll care about is the audience you’re targeting, the recommended creative (again, based on the mar-tech running in the background) and the resulting behaviors. Once your audience has been targeted and engaged, the predicted path of persuasion is continually updated and new channels are engaged as required. You won’t care what channels they are – you’ll simply monitor the progression of persuasion.


The Coming Data Marketplace

The stakes are currently being placed in the ground. The next great commodity will be data and you can already sense the battle beginning the heat up.

Consumer data will be generated by connections. Those connections will fall into two categories: broad and deep. Both will generate data points that will become critical to businesses looking to augment their own internal data.

First, broad data is the domain of Google, Apple, Amazon, eBay and Facebook. Their play is it to stretch their online landscape as broadly as possible, generating thousands of new potential connections with the world at large. Google’s new “Buy” button is a perfect example of this. Adding to the reams of conversion data Google already collects, the “Buy” button means that Google will control even more transactional landscape. They’re packaging it with the promise of an improved mobile buying experience, but the truth is that purchases will be consummated on Google controlled territory, allowing them to harvest the rich data that will be generated from millions of individual transactions across every conceivable industry category. If Google can control a critical mass of connected touch points across the online landscape, they can get an end-to-end view of purchase behavior. The potential of that data is staggering.

In this market, data will be stripped of identity and aggregated to provide a macro but anonymous view of market behaviors. As the market evolves, we’ll be able to subscribe to data services that will provide real time views of emerging trends and broad market intelligence that can be sliced and diced in thousands of ways. Of course, Google (and their competitors) will have a free hand to use all this data to offer advertisers new ways to target ever more precisely.

This particular market is an online territory grab. It relies on a broad set of touch points with as many people across as many devices as possible. The more territory that is covered, the more comprehensive the data set.

The other data market will run deep. Consider the new health tracking devices like Fitbit, Garmin’s VivoActive and Apple’s iWatch. Focused purpose hardware and apps will rely on deep relationships with users. The more reliant you become on these devices, the more valuable the data collected will become. But this data comes with a caveat – unlike the broad data market, this data should not be striped of its identity. The value of the data comes from its connection with an individual. Therefore, that individual has to be an active participant in any potential data marketplaces. The data collector will act more as a data middleman – brokering matches between potential customers and vendors. If the customer agrees, they can choose to release the data to the vendor (or at least, a relevant subset of the data) in order to individualize the potential transaction.

As the data marketplace evolves, expect an extensive commercial eco-system to emerge. Soon, there will be a host of services that will take raw data and add value through interpretation, aggregation and filtering. Right now, the onus for data refinement falls on the company who is attempting to embrace Big Data marketing. As we move forward, expect an entire Big Data value chain to emerge. But it will all rely on players like Google, Amazon and Apple who have the front line access to the data itself. Just as natural resources provided the grist that drove the last industrial revolution, expect data to be the resource that fuels the next one.

The Persona is Dead, Long Live the Person

First, let me go on record as saying up to this point, I’ve been a fan of personas. In my past marketing and usability work, I used personas extensively as a tool. But I’m definitely aware that not everyone is equally enamored with personas. And I also understand why.

Personas, like any tool, can be used both correctly and incorrectly. When used correctly, they can help bridge the gap between the left brain and the right brain. They live in the middle ground between instinct and intellectualism. They provide a human face to raw data.

But it’s just this bridging quality that tends to lead to abuse. On the instinct side, personas are often used as a short cut to avoid quantitative rigor. Data driven people typically hate personas for this reason. Often, personas end up as fluffy documents and life sized cardboard cutouts with no real purpose. It seems like a sloppy way to run things.

On the intellectual side, because quant people distrust personas, they also leave themselves squarely on data side of the marketing divide. They can understand numbers – people not so much. This is where personas can shine. At their best, they give you a conceptual container with a human face to put data into. It provides a richer but less precise context that allows you to identify, understand and play out potential behaviors that data alone may not pinpoint.

As I said, because personas are intended as a bridging tool, they often remain stranded in no man’s land. To use them effectively, the practitioner should feel comfortable living in this gap between quant and qual. Too far one way or the other and it’s a pretty safe bet that personas will either be used incorrectly or be discarded entirely.

Because of this potential for abuse, maybe it’s time we threw personas in the trash bin. I suspect they may be doing more harm than good to the practice of marketing. Even at their best, personas were meant as a more empathetic tool to allow you to thing through interactions with a real live person in mind. But in order to make personas play nice with real data, you have to be very diligent about continually refining your personas based on that data. Personas were never intended to be placed on a shelf. But all too often, this is exactly what happens. Usually, personas are a poor and artificial proxy for real human behaviors. And this is why they typically do more harm than good.

The holy grail of marketing would be to somehow give real time data a human face. If we could find a way to bridge left brain logic and right brain empathy in real time to discover insights that were grounded in data but centered in the context of a real person’s behaviors, marketing would take a huge leap forward. The technology is getting tantalizingly close to this now. It’s certainly close enough that it’s preferable to the much abused persona. If – and this is a huge if – personas were used absolutely correctly they can still add value. But I suspect that too much effort is spent on personas that end up as documents on a shelf and pretty graphics. Perhaps that effort would be better spent trying to find the sweet spot between data and human insights.

Why Cognitive Computing is a Big Deal When it comes to Big Data


Watson beating it’s human opponents at Jeopardy

When IBM’s Watson won against humans playing Jeopardy, most of the world considered it just another man against machine novelty act – going back to Deep Blue’s defeat of chess champion Garry Kasporov in 1997. But it’s much more than that. As Josh Dreller reminded us a few Search Insider Summits ago, when Watson trounced Ken Jennings and Brad Rutter in 2011, it ushered in the era of cognitive computing. Unlike chess, where solutions can be determined solely with massive amounts of number crunching, winning Jeopardy requires a very nuanced understanding of the English language as well as an encyclopedic span of knowledge. Computers are naturally suited to chess. They’re also very good at storing knowledge. In both cases, it’s not surprising that they would eventually best humans. But parsing language is another matter. For a machine to best a man here requires something quite extraordinary. It requires a machine that can learn.

The most remarkable thing about Watson is that no human programmer wrote the program that made it a Jeopardy champion. Watson learned as it went. It evolved the winning strategy. And this marks a watershed development in the history of artificial intelligence. Now, computers have mastered some of the key rudiments of human cognition. Cognition is the ability to gather information, judge it, make decisions and problem solve. These are all things that Watson can do.


Peter Pirolli - PARC

Peter Pirolli – PARC

Peter Pirolli, one of the senior researchers at Xerox’s PARC campus in Palo Alto, has been doing a lot of work in this area. One of the things that has been difficult for machines has been to “make sense” of situations and adapt accordingly. Remember, a few columns ago where I talked about narratives and Big Data, this is where Monitor360 uses a combination of humans and computers – computers to do the data crunching and humans to make sense of the results. But as Watson showed us, computers do have to potential to make sense as well. True, computers have not yet matched humans in the ability to sense make in an unlimited variety of environmental contexts. We humans excel at quick and dirty sense making no matter what the situation. We’re not always correct in our conclusions but we’re far more flexible than machines. But computers are constantly narrowing the gap and as Watson showed, when a computer can grasp a cognitive context, it will usually outperform a human.

Part of the problem machines face when making sense of a new context is that the contextual information needs to be in a format that can be parsed by the computer. Again, this is an area where humans have a natural advantage. We’ve evolved to be very flexible in parsing environmental information to act as inputs for our sense making. But this flexibility has required a trade-off. We humans can go broad with our environmental parsing, but we can’t go very deep. We do a surface scan of our environment to pick up cues and then quickly pattern match against past experiences to make sense of our options. We don’t have the bandwidth to either gather more information or to compute this information. This is Herbert Simon’s Bounded Rationality.

But this is where Big Data comes in. Data is already native to computers, so parsing is not an issue. That handles the breadth issue. But the nature of data is also changing. The Internet of Things will generate a mind-numbing amount of environmental data. This “ambient” data has no schema or context to aid in sense making, especially when several different data sources are combined. It requires an evolutionary cognitive approach to separate potential signal from noise. Given the sheer volume of data involved, humans won’t be a match for this task. We can’t go deep into the data. And traditional computing lacks the flexibility required. But cognitive computing may be able to both handle the volume of environmental Big Data and make sense of it.

If artificial intelligence can crack the code on going both broad and deep into the coming storm of data, amazing things will certainly result from it.

The Human Stories that Lie Within Big Data

storytelling-boardIf I wanted to impress upon you the fact that texting and driving is dangerous, I could tell you this:

In 2011, at least 23% of auto collisions involved cell phones. That’s 1.3 million crashes, in which 3331 people were killed. Texting while driving makes it 23 times more likely that you’ll be in a car accident.

Or, I could tell you this:

In 2009, Ashley Zumbrunnen wanted to send her husband a message telling him “I love you, have a good day.” She was driving to work and as she was texting the message, she veered across the centerline into oncoming traffic. She overcorrected and lost control of her vehicle. The car flipped and Ashley broke her neck. She is now completely paralyzed.

After the accident, Zumbrunnen couldn’t sit up, dress herself or bath. She was completely helpless. Now a divorced single mom, she struggles to look after her young daughter, who recently said to her “I like to go play with your friends, because they have legs and can do things.”

The first example gave you a lot more information. But the second example probably had more impact. That’s because it’s a story.

We humans are built to respond to stories. Our brains can better grasp messages that are in a narrative arc. We do much less well with numbers. Numbers are an abstraction and so our brains struggle with numbers, especially big numbers.

One company, Monitor360, is bringing the power of narratives to the world of big data. I chatted with CEO Doug Randall recently about Monitor360’s use of narratives to make sense of Big Data.

“We all have filters through which we see the world. And those filters are formed by our experiences, by our values, by our viewpoints. Those are really narratives. Those are really stories that we tell ourselves.”

For example, I suspect the things that resonated with you with Ashley’s story were the reason for the text – telling her husband she loved him – the irony that the marriage eventually failed after her accident and the pain she undoubtedly felt when her daughter said she likes playing with other moms who can still walk. All of those things, while they don’t really add anything to our knowledge about the incidence rate of texting and driving accidents, are all things that strike us at a deeply emotional level because we can picture ourselves in Ashley’s situation. We empathize with her. And that’s what a story is, a vehicle to help us understand the experiences of another.

Monitor360 uses narratives to tap into these empathetic hooks that lie in the mountain of information being generating by things like social media. It goes beyond abstract data to try to identify our beliefs and values. And then it uses narratives to help us make sense of our market. Monitor360 does this with a unique combination of humans and machines.

“A computer can collect huge amounts of data and the compute can even sort that data. But “sense making” is still very, very difficult for computers to do. So human beings go through that information, synthesize that information and pull out what the underlying narrative is.”

Monitor360 detects common stories in the noisy buzz of Big Data. In the stories we tell, we indicate what we care about.

“This is what’s so wonderful about Big Data. The Data actually tells us, by volume, what’s interesting. We’re taking what are the most often talked about subjects…the data is actually telling us what those subjects are. We then go in and determine what the underlying belief system in that is.”

Monitor360’s realization that it’s the narratives that we care about is an interesting approach to Big Data. It’s also encouraging to know that they’re not trying to eliminate human judgment from the equation. Empathy is still something we can trump computers at.

At least for now.

The Bug in Google’s Flu Trend Data

First published March 20, 2014 in Mediapost’s Search Insider

Last year, Google Flu Trends blew it. Even Google admitted it. It over predicted the occurrence of flu by a factor of almost 2:1.  Which is a good thing for the health care system, because if Google’s predictions had have been right, we would have had the worst flu season in 10 years.

Here’s how Google Flu Trends works. It monitors a set of approximately 50 million flu related terms for query volume. It then compares this against data collected from health care providers where Influenza-like Illnesses (ILI) are mentioned during a doctor’s visit. Since the tracking service was first introduced there has been a remarkably close correlation between the two, with Google’s predictions typically coming within 1 to 2 percent of the number of doctor’s visits where the flu bug is actually mentioned. The advantage of Google Flu Trends is that it is available about 2 weeks prior to the ILI data, giving a much needed head start for responsiveness during the height of flu season.

FluBut last year, Google’s estimates overshot actual ILI data by a substantial margin, effectively doubling the size of the predicted flu season.

Correlation is not Causation

This highlights a typical trap with big data – we tend to start following the numbers without remembering what is generating the numbers. Google measures what’s on people’s minds. ILI data measures what people are actually going to the doctor about. The two are highly correlated, but one doesn’t not necessarily cause the other. In 2013, for instance, Google speculated that increased media coverage might be the cause for the overinflated predictions. More news coverage would have spiked interest, but not actual occurrences of the flu.

Allowing for the Human Variable

In the case of Google Flu Trends, because it’s using a human behavior as a signal – in this case online searching for information – it’s particularly susceptible to network effects and information cascades. The problem with this is that these social signals are difficult to rope into an algorithm. Once they reach a tipping point, they can break out on their own with no sign of a rational foundation. Because Google tracks the human generated network effect data and not the underlying foundational data, it is vulnerable to these weird variables in human behavior.

Predicting the Unexpected

A recent article in Scientific American pointed out another issue with an over reliance on data models –  Google Flu Trends completely missed the non-seasonal H1N1 pandemic in 2009. Why? Algorithmically, Google wasn’t expecting it. In trying to eliminate noise from the model, they actually eliminated signal coming during an unexpected time. Models don’t do very well at predicting the unexpected.

Big Data Hubris

The author of the Scientific American piece, associate editor Larry Greenemeier, nailed another common symptom of our emerging crush on data analytics – big data hubris. We somehow think the quantitative black box will eliminate the need for more mundane data collection – say – actually tracking doctor’s visits for the flu. As I mentioned before, the biggest problem with this is that the more we rely on data, which often takes the form of arm’s length correlated data, the further we get from exploring causality. We start focusing on “what” and forget to ask “why.”

We should absolutely use all the data we have available. The fact is, Google Flu Trends is a very valuable tool for health care management. It provides a lot of answers to very pertinent questions. We just have to remember that it’s not the only answer.

The Ill Defined Problem of Attribution

First published July 11, 2013 in Mediapost’s Search Insider

For the past few years, I’ve sat on the board of a company that audits audience for various publications. One of the challenges the entire audience measurement industry has faced is the explosion of channels traditional publishers have been forced to use. It’s one thing to tally up the audience of a single newspaper, magazine or radio station. It’s quite another to try to get an aggregate view of an audience of publishers that, in addition to their magazines, have a website, several blogs, various email newsletters, a full slate of webinars, a YouTube channel, multiple Twitter accounts, Facebook pages, other social destinations, digital versions of magazines and an ever-growing collection of tablet and smartphone apps. Consider, for instance, how you would estimate the size of MediaPost’s total audience.

The problem, one quickly realizes, is how you find a common denominator across all these various points of audience engagement. It’s the classic “apples and oranges” challenge, multiplied several times over.

This is the opposite side of the attribution problem. How do you attribute value, whether it’s in terms of persuading a single prospect, or the degree of engagement across an entire audience, when there are so many variables at play?

Usually, when you talk about attribution, someone in the room volunteers that the answer to the problem can be found by coming up with the right algorithm, with the usual caveat something like this: “I don’t know how to do it, but I’m sure someone far smarter than I could figure it out.” The assumption is that if the data is there, there should be a solution hiding in there somewhere.

No disrespect to these hypothetical “smart” data-crunchers out there, but I believe there is a fundamental flaw in that assumption. The problem behind that assumption is that we’re accepting the problem as a “well defined” one – when in fact it’s an “ill-defined” problem.

We would like to believe that this is a solvable problem that could be reduced to a simplified and predictable model. This is especially true for media buyers (who use the audience measurement services) and marketers (who would like to find a usable attribution model). The right model, driven by the right algorithm, would make everyone’s job much easier. So, let’s quit complaining and just hire one of those really smart people to figure it out!

However, if we’re talking about an ill-defined problem, as I believe we are, then we have a significantly bigger challenge. Ill-defined problems defy clear solutions because of their complexity and unpredictability. They usually involve human elements impossible to account for. They are nuanced and “grey” as opposed to clear-cut “blacks and white.” If you try to capture an ill-defined problem in a model, you are forced to make heuristic assumptions that may be based on extraneous noise rather than true signals. This can lead to “overfitting.”

Let me give you an example. Let’s take that essential human goal: finding a life partner. Our task is to build an attribution model for successful courtship. Let us assume that we met our own livelong love in a bar. We would assume, then, that bars should have a relatively generous attribution of value in the partnership “conversion” funnel. But we’re ignoring all the “ill-defined” variables that went into that single conversion event: our current availability, the availability of the prospect, our moods, our level of intoxication, the friends we were with, the song that happened to be playing, the time of night, the necessity to get up early the next morning to go to work, etc.

In any human activity, the list of variables that must be considered to truly “define” the problem quickly becomes impossible. If we assume that bars are good places to find a partner, we must simplify to the point of “over-fitting.”  It may turn out that a grocery store, ATM or dentist’s waiting room would have served the purpose equally well.

Of course, you could take a purely statistical view, based on backwards-looking data. For example, we could say that of all couples, 23.7% of them met in bars. That may give us some very high level indications of “what” is happening, but it does little to help us understand the “why” of those numbers. Why do bars act as a good meeting ground?

In the end, audience measurement and attribution, being ill-defined problems, may end up as rough approximations at best. And that’s OK. It’s better than nothing. But I feel it’s only fair to warn those who believe there’s a “smarter” whiz out there who can figure all this out: Human nature is notoriously tough to predict.

Seperating the Strategic Signal from the Tactical Noise in Marketing

First published April 4, 2013 in Mediapost’s Search Insider

It’s somewhat ironic that, as a die-hard Darwinist, I find myself in the position of defending strategy against the onslaught of Big Data. Since my initial column on this subject a few months ago, I’ve been diving deeper into this topic.

Here’s the irony.

Embracing Big Data is essentially embracing a Darwinist approach to marketing.  It resists taking a top-down approach (aka strategy) by using data feedback to enforce evolution of your marketing program. It makes marketing “antifragile,” in the words of Nassim Nicholas Taleb. In theory, it uses disorder, mistakes and unexpected events to continually improve marketing.

Embracing strategy — at least my suggested Bayesian approach to strategy — would be akin to embracing intelligent design. It defines what an expected outcome should be, then starts defining paths to get there. But it does this in the full realization that those paths will continually shift and change. In fact, it sets up the framework to enable this strategic fluidity. It still uses “Big Data,” but puts it in the context of “Big Testing” (courtesy Scott Brinker).

To remove the strategy from the equation, as some suggest, would be to leave your marketing subject to random chance. Undoubtedly, given perfect feedback and the ability to quickly adapt using that feedback, marketing could improve continually. After all, we evolved in just such an environment and we’re pretty complex organisms.  But it’s hard to argue that a designer would have designed such flaws as our pharynx, which is used both for eating and breathing, leading to a drastically higher risk of choking; our spinal column, which tends to become misaligned in a significant portion of the population; or the fact that our retinas are “inside out.”

Big Data also requires separating “signal” from “noise” in the data. But without a strategic framework, what is the signal and what is the noise? Which of the datum do you pay attention to, and which do you ignore?

Here’s an even bigger question. What constitutes success and failure in your marketing program? Who sets these criteria? In nature, it’s pretty simple. Success is defined by genetic propagation. But it’s not so clear-cut in marketing. Success needs to align to some commonly understood objectives, and these objectives should be enshrined in — you guessed it, your strategy.

I believe that if  “intelligent designers” are available, why not use them? And I would hope that most marketing executives should fit the bill. As long as strategy includes a rigorous testing methodology and honest feedback does not fall victim to egotistical opinions and “yes speak” (which is a huge caveat, and a topic too big to tackle here), a program infused with strategy should outperform one left to chance.

But what about Taleb’s “Black Swans”? He argues that by providing “top down” direction, leading to interventionism, you tend to make systems fragile. In trying to smooth out the ups and downs of the environment, you build in limitations and inflexibility. You lose the ability to deal with a Black Swan, that unexpected occurrence that falls outside of your predictive horizon.

It’s a valid point. I believe that Black Swans have to be expected, but should not dictate your strategy. By their very nature, they may never happen. And if they do, they will be infrequent. If your strategy meets a Black Swan head on, a Bayesian approach should come with the humility to realize that the rules have changed, necessitating a corresponding change in strategy. But it would be a mistake to abandon strategy completely based on a “what-if.”