Following up on my previous post about the problems with most market research, here’s a plea why we should keep trying to get it right.

At the recent London SMX show, I presented on the Ad Testing and Research panel. Like other times I’ve done this panel (this is probably the 3rd or 4th time) I hear about skillful practitioners employing various A/B and multivariate testing methodologies. Ad testing is a definite must do, but before my presentation, which came at the end of the session, I took a few minutes to provide an alternative point of view.

I asked the small crowd how many of them were doing regular campaign management, checking click through rates, conversion rates and optimizing their campaigns based on what they saw. Almost everyone put up their hand. Then I asked how many did A/B testing. This time, a little more than half put up their hands. Next, I asked how many were doing multivariate testing. This time, about one third of the crowd. Finally, I asked how many had actually sat, watched a customer interact with their site and then asked them questions. We dropped down to about 10% of the group, and most of these were in a fairly structured usability test, with limited or no opportunity for interaction with the user.

Now, campaign optimization, A/B and multivariate testing are all best practices and should be done religiously. But I urged the marketers in the room to step back from their data heavy, spreadsheet  bound view of the world and pick up a book on cognitive psychology, social science or simple usability. Better yet, spend some time just watching how real people interact with your site. Try, for a moment, to look at the world through your customer’s eyes.

The problem with the typical, quantitative methods are that they’re all lagging indicators. You don’t get an idea of what’s happening until after customers have interacted with your ads and your site. You generally get a good sense of what they did, but it’s very difficult to determine why they did it. To do that, you have to dig beyond the numbers. You have to try to get into that subconscious mind. And that’s not easy. Typical market research methodologies won’t cut it. To get some idea of what’s required, read Clotaire Rapaille’s The Culture Code, or Gerald Zaltman’s How Customer’s Think. Do some digging into the work of Herbert Simon.  It takes a deft combination of psychiatric know how and detective skills. But here’s why it’s worth it.

For the past Century, we’ve largely refined our marketing practices based on trial and error. Pretty much everything has been done through seeing what’s worked, changing something, and seeing if it worked better. That’s been okay, as long as the channels we used to reach customer’s were relatively limited. With limited channels and a certain amount of control inherent in the process, we could do this. But those days are over.

Now, rather than a few controlled channels that run pretty much straight from the advertiser to the customer, we have an explosion of information that turns the typical buying process into a Gordian knot of unbelievable complexity. We can’t control the variables anymore. When there are so many channels, so many interdependent factors and so much of it affects customers below the conscious level, trial and error is just not an effective testing methodology anymore. In fact, it was never an effective methodology, for all the reasons I stated in my previous post. It’s just the best we had.

Let me use another example. The way we did marketing was pretty much like jumping in a car, randomly making decisions whether to turn right or left, keeping track of our success rate in getting nearer to our destination, and using this method to eventually pick the right route. This method might eventually work okay in a town of a few thousand people, but try doing that to navigate through New York or Los Angeles. We don’t have enough time in our lives to leave this much to chance. A map (or better yet, a GPS) is a much better alternative.

But we’re just starting to put that map together. And it won’t come from market research. Market research, at least in it’s current incarnation, is hopelessly flawed. It will come from diving deep into the workings of our brains. And once we begin putting the map together, it will allow us to begin to measure leading indicators. It will keep us from the trap of relying on self reported rationalizations and dig deeper into all the activity that’s happening below the conscious surface of our minds. That’s where the answers will be found.

Here’s another reason. Our brains are not only complex, but they’re also highly adaptive. As we do new mental activities more often, and abandon previous ones, new routes are established through the neurons and old ones become overgrown and eventually, unused neurons are cut away. It’s called “pruning” and “neuroplasticity”. It’s probably why you’re much better at using a search engine now than you are doing the geometry you learned in grade 9. We’ve worn new paths in our brain.

This is also true of how we’re buying. The way we buy now is bearing little resemblance to the way we bought in 1975. As time goes on and we rely on the Internet more and more, the paths that we used to use for our consumer decisions will become overgrown and we’ll clear new ones. This will happen not only at the conscious level, but also the sub conscious level. We will literally rewire how our brains decide what to buy. So the body of market research that has laboriously been gathered over the past several decades will become obsolete. And to discover those again through trial and error will be an long and potentially impossibly task.

So, a word of advice. Step back from the spread sheet now and again. Take a break from looking at “what” and start to explore “why”. Dig into things like the triune brain, selective perception, bounded rationality, working memory and some other basic cognitive concepts. It will be time well spent.