Category Archives: Behavior & Psychology

The Whys of Buy: Impulse Buying

chickenebayThe other day, we were talking about what makes us buy (an appropriate topic for today, the biggest shopping day in the US) and Barb Newman, our General Manager, wondered what made us impulse buy? She was trying to figure out why she had dropped way more money than she intended on a purse. Being intrigued by the buying mechanism that seems to be locked in our skulls, I decided to do a little digging to find out what’s going on when we just seem to pick up something off the shelf on sheer whim.

Spool’s Impulse Buying Study

On doing some digging, I found a study done by Jared Spool, a usability consultant I have a tremendous amount of respect for. And Jared found that market research, as I posted about a few days ago, has its limits. As Jared’s starting point, he looked at survey’s conducted by The Yankee Group and Ernst and Young. Both surveys asked why respondents would make impulse purchases on the web. With the Yankee Group survey, 75% indicated because of price. Ernst and Young’s survey said that 88% of purchases were made due to price. Again, these were surveys where buyers were asked to rationalize their behavior. And saving money seems like a pretty rational reason.

But Jared wanted to see what real people shopping for real products did. As most usability people do, he wanted to observe real world behavior. So his group got 30 people, who had real things they wanted to buy, gave them some money and sent them on an online shopping trip to a few preselected sites that had what they were looking for.

What they found was significantly different that what the Yankee Group and Ernst and Young surveys showed. While many of the participants bought what they were looking for, a significant number, 34%, also added other items into their shopping cart that weren’t on their original lists. Was it because the prices were irrestible? No, in fact, only 8% of the impulse purchases were because of price.

Jared and his group purposely picked out shopping sites that had promotional offers and seasonal sales in prominent display positions, especially on the home pages. But very few of the purchases were of these sale items. The impulse buys were spread across 41% of the sites in the study, including everything from pet shops to computer accessory stores. Almost none of the items were on sale. They were just things that suddenly tweaked the shopper’s interest.

Here’s the other interesting thing about the study. Most of the impulse items were chosen while browsing through the category pages. They had chosen a category based on what they were shopping for and had found related items that struck their interest and were subsequently added to their cart.

The Nucleus Accumbens Made Me Do It

So, why do we impulse buy? I’m still not sure, but here are some hunches, based on some of the other research I’m doing in the mental mechanics of buying.

A study earlier this year by Carnegie Mellon, Stanford and the MIT Sloan School of Management might be able to shed a little light on Spool’s findings. Using fMRI imaging, they also gave participants money to go shopping. They then monitored activity in various parts of the brain.

They found that when we anticipate buying something, the pleasure center, the nucleus accumbens, is activated. We begin picturing ourselves in possession of a product and visualizing ourselves using it. We start to build neural pathways that reinforce what it would be like to have the product. But, if the price is excessive, the study found that the brain has a shut off mechanism.  A part of the brain known as the insula is activated and the part of the brain we use to balance gains versus losses, in other words, is the product worth the price, the medial prefrontal cortex, begins shutting down. We literally put the purchase out of our mind because the price is more than we’re willing to pay.

So, let’s go back to Jared Spool’s study. I suspect we get into shopping “modes” where the parts of the brain associated with acquisition of a product sustain some activity. We’re prepared to buy, so the nucleus accumbens kicks into gear and keeps firing. We’re in “buy” mode. And we’ve accepted that we have budget available. We start out looking for the product we intended to buy, but, on the way, if we see something we also decide we need, especially in a related category, our “buying” mechanism is already activated. We’re already primed to consider purchase. We’re not looking for a bargain (although finding one certainly wouldn’t hurt), but by the same token, an outrageous price would probably shut down the process by kicking in the insula. Think of the insula as the brain’s sprinkler system, snuffing out any impulsive sparks before we burn ourselves. As long as the price is reasonable, and doesn’t introduce significant “pain” we’re more likely to purchase.

The fMRI study also showed that once we flip into buy mode, we tend to stay in this groove. This is why it’s much more dangerous to shop with credit cards than cash. Credit cards allow us to put off the “pain” that might kick in the insula, letting the nucleus accumbens have its way. When cash runs out, it runs out. It forces us to pay more attention to the “pain”.

In Spool’s study, the pain had been effectively removed by giving the participants money to spend. And by browsing through categories where they already had interest, there was a greater likelihood to pick related products and purchase them through impulse. Bargain basement prices really had nothing to do with the process. It’s just that most of us don’t understand the mechanics of buying that happen at the subconscious level.

Back to Barb’s Purse

So let’s get back to Barb’s purse. Was it really a impulse buy? Well, not really. As I chatted more with Barb, she indicated that she had seen the purse earlier in a magazine, fallen in love with it, but the price was much higher than she wanted to pay. So Barb’s nucleus accumbens had gone into overdrive, but Barb, being a practical shopper, had quickly doused the flames when her insula kicked in. The pain was too great to make a purchase.

But, a few months later, she’s in the mall and sees that the store that carries the purse was having a 25% off everything in the store appreciation sale. Suddenly, the nucleus accumbens is reactivated, primed by all the visualization that Barb had done since first seeing the purse, thinking how great it would be to own it. The 25% off sale lowered the pain threshold enough to keep the insula from kicking in, and the next thing Barb knew, she had put down a deposit and put the purse on layaway. She didn’t know what hit her. Now, she knows it was a little bit of gray matter hiding deep in subcortal brain called the nucleus accumbens that’s to blame. But this wasn’t a true impulse buy. It was more like a delayed buy.

So, if you overspend today, remember, it was the nucleus accumbens that made you do it. Try explaining that one to your significant other.

Why We Have to Keep Doing Market Research

Following up on my previous post about the problems with most market research, here’s a plea why we should keep trying to get it right.

At the recent London SMX show, I presented on the Ad Testing and Research panel. Like other times I’ve done this panel (this is probably the 3rd or 4th time) I hear about skillful practitioners employing various A/B and multivariate testing methodologies. Ad testing is a definite must do, but before my presentation, which came at the end of the session, I took a few minutes to provide an alternative point of view.

I asked the small crowd how many of them were doing regular campaign management, checking click through rates, conversion rates and optimizing their campaigns based on what they saw. Almost everyone put up their hand. Then I asked how many did A/B testing. This time, a little more than half put up their hands. Next, I asked how many were doing multivariate testing. This time, about one third of the crowd. Finally, I asked how many had actually sat, watched a customer interact with their site and then asked them questions. We dropped down to about 10% of the group, and most of these were in a fairly structured usability test, with limited or no opportunity for interaction with the user.

Now, campaign optimization, A/B and multivariate testing are all best practices and should be done religiously. But I urged the marketers in the room to step back from their data heavy, spreadsheet  bound view of the world and pick up a book on cognitive psychology, social science or simple usability. Better yet, spend some time just watching how real people interact with your site. Try, for a moment, to look at the world through your customer’s eyes.

The problem with the typical, quantitative methods are that they’re all lagging indicators. You don’t get an idea of what’s happening until after customers have interacted with your ads and your site. You generally get a good sense of what they did, but it’s very difficult to determine why they did it. To do that, you have to dig beyond the numbers. You have to try to get into that subconscious mind. And that’s not easy. Typical market research methodologies won’t cut it. To get some idea of what’s required, read Clotaire Rapaille’s The Culture Code, or Gerald Zaltman’s How Customer’s Think. Do some digging into the work of Herbert Simon.  It takes a deft combination of psychiatric know how and detective skills. But here’s why it’s worth it.

For the past Century, we’ve largely refined our marketing practices based on trial and error. Pretty much everything has been done through seeing what’s worked, changing something, and seeing if it worked better. That’s been okay, as long as the channels we used to reach customer’s were relatively limited. With limited channels and a certain amount of control inherent in the process, we could do this. But those days are over.

Now, rather than a few controlled channels that run pretty much straight from the advertiser to the customer, we have an explosion of information that turns the typical buying process into a Gordian knot of unbelievable complexity. We can’t control the variables anymore. When there are so many channels, so many interdependent factors and so much of it affects customers below the conscious level, trial and error is just not an effective testing methodology anymore. In fact, it was never an effective methodology, for all the reasons I stated in my previous post. It’s just the best we had.

Let me use another example. The way we did marketing was pretty much like jumping in a car, randomly making decisions whether to turn right or left, keeping track of our success rate in getting nearer to our destination, and using this method to eventually pick the right route. This method might eventually work okay in a town of a few thousand people, but try doing that to navigate through New York or Los Angeles. We don’t have enough time in our lives to leave this much to chance. A map (or better yet, a GPS) is a much better alternative.

But we’re just starting to put that map together. And it won’t come from market research. Market research, at least in it’s current incarnation, is hopelessly flawed. It will come from diving deep into the workings of our brains. And once we begin putting the map together, it will allow us to begin to measure leading indicators. It will keep us from the trap of relying on self reported rationalizations and dig deeper into all the activity that’s happening below the conscious surface of our minds. That’s where the answers will be found.

Here’s another reason. Our brains are not only complex, but they’re also highly adaptive. As we do new mental activities more often, and abandon previous ones, new routes are established through the neurons and old ones become overgrown and eventually, unused neurons are cut away. It’s called “pruning” and “neuroplasticity”. It’s probably why you’re much better at using a search engine now than you are doing the geometry you learned in grade 9. We’ve worn new paths in our brain.

This is also true of how we’re buying. The way we buy now is bearing little resemblance to the way we bought in 1975. As time goes on and we rely on the Internet more and more, the paths that we used to use for our consumer decisions will become overgrown and we’ll clear new ones. This will happen not only at the conscious level, but also the sub conscious level. We will literally rewire how our brains decide what to buy. So the body of market research that has laboriously been gathered over the past several decades will become obsolete. And to discover those again through trial and error will be an long and potentially impossibly task.

So, a word of advice. Step back from the spread sheet now and again. Take a break from looking at “what” and start to explore “why”. Dig into things like the triune brain, selective perception, bounded rationality, working memory and some other basic cognitive concepts. It will be time well spent.

What’s Wrong with Market Research

sharingbrainWhen we first started doing research at Enquiro into how people used search, we found very quickly that what people say and what people do are very different things. It just happened that we were doing a survey and a focus group at roughly the same time. In the survey, where we got the results first, we asked if things like the position of a listing was important in whether people read it or not. We asked people to rank a number of factors on their relative importance, including position, relevancy and trust in brands and vendors shown. Almost without exception, in the survey, people indicated that relevancy was the key factor. They also indicated that they read listings pretty carefully and gave a fair amount of thought before selecting one. Finally, many said they would never click on a paid listing.

Then, we invited about 30 people into our labs and actually recorded their interactions with the search engines (before our eye tracking studies) and it quickly became obvious that how they said they used a search engine and how they actually did were two different things. The vast majority of clicks happened in the first few listings. Many who indicated they wouldn’t click on paid listings actually did, and perhaps, most interestingly, the average interaction was around 10 seconds or so. Subsequently, we’ve seen this type of behavior repeated in eye tracking after eye tracking study. Of course, the famous golden triangle study we did with Eyetools and Did It, and subsequent ones conducted by Enquiro, have shown over and over how quickly we interact with a search engine and how much of our scanning activity is “top loaded”. Also, we don’t really skip over sponsored listings, but in some circumstances (research based activity) we’re less likely to click on them. We’ve used this body of research to come up with a fairly consistent model of how people interact with search results. The results belie what people indicated in our very first survey. Well over 60% of the clicks happened in the first 4 or 5 listings, including the top sponsored ones. People generally spent just a few seconds on the page (around 10 to 12 seems to be the average) in which they scan (not read) 4 to 5 listings. There was almost no deliberation. People click quickly, and if they don’t like what they see, they click back. It would take the average person about 2 minutes to actually read all the results on the average search results page. Even if we just read the top 4 or 5, we’d be spending about 30 to 40 seconds on the page. It takes about 7 seconds to read one listing. But we don’t spend much longer than this covering 4 to 5 listings, about 2 seconds per listing. Obviously, we don’t give a lot of thought to the credibility of the search listings.

So, were all 1600 of our original survey respondents liars? Were they intentionally misleading us? No, they were just being human.

What we found was the systemic fault with almost all market research. And there’s a very good explanation for it. We’re generally not aware of 95% of what we do or why we do it. That’s because much or what we do is hidden in our subconscious. I’m currently reading How Customers Think by Gerald Zaltman and he pinpoints the problem with traditional market research. In almost every case, we ask people to tell us, either verbally or through writing, what they’re thinking. Just by doing this, we kick in the cortex, the rational seat of our intellect. But Zaltman tells us that at least 95% of every decision is made subconsciously. There, in the murky depths of our brains, predating the evolution of our cortex by many millions of years, thoughts are created through tremendously complex connections of memories, beliefs, instincts and intuition. In many cases, our decisions are made long before they bubble up to our conscious minds. The conscious mind exists to put a little polish on them and, in most cases, to rationalize a decision that was largely based on primal instincts. We may have done what we did because our flight or fight mechanism kicked in, or because our need to procreate surfaced. That’s why we chose the minivan, or the red convertible. It really had nothing to do with the Consumer Reports rating. But, being highly evolved humans, we convince ourselves that our choices are much more rational than those of a lizard (our basic brain core, which rules many of our decisions, is basically the same as a reptile’s brain).

In our case, our initial respondents indicated that they deliberated over which search result they chose. In actual fact, there was little risk in choosing a wrong link (it’s not like our lives, our family or our money is at stake), so we cut off the amount of deliberation we did and after a quick scan, picked the result that seemed to be most relevant to our intent. The lack of deliberation wasn’t lack of intelligence, it was a survival instinct bred into us by eons of evolutionary refinement. If there’s no immediate risk to us, why should we kick in our brains and spend unnecessary time and cortex processing power to come to the optimal decision. It’s not required. A simple scan and click will suffice. Our brains are simply doing what they’ve been programmed to do. And it’s not that the decisions are bad. As Malcolm Gladwell shows in Blink, often these decisions prove to be better than the ones that we endlessly deliberate over. Our brains, especially the 95% that remains under the surface, are amazingly adept at making good decisions.

But there’s a more fundamental issue here. If what we experienced in search is typical in all market research (which it is) how do we ever find out how people actually make purchase decisions?

This is a significant challenge, the extent of which might not be obvious at first glance. Let me use an analogy to further illustrate. Remember the tale of the shoemaker and the elves? Let me use that and adapt it slightly for my purposes. For those of you unfamiliar with the story, a poor shoemaker only has enough leather left for one pair of shoes. He cuts the leather and lays it out for stitching the next morning. He awakes, amazed to find the shoes made, and meticulously crafted at that. Elves apparently helped out during the night, soon to bring fame and fortune to the shoemaker.

But what if the elves didn’t exist. What if, instead, the shoemaker was actually making the shoes in his sleep? The idea is not so ridiculous. Rumor has it that Coleridge actually wrote Kubla Khan during a dream, and managed to scribble it down before it faded from his consciousness. As any psychiatrist will tell you, we’re closest to our subsconscious when we’re hovering between sleep and wakefulness. It’s about the only time we get a glimpse into those murky depths.

So let’s say our shoemaker actually makes the shoes in some bizarre bout of sleepwalking. He awakes every morning, to find the shoes nearly perfectly finished. All he needs to do is add the laces and a bit of polish. And the shoes are fair more carefully crafted then he could ever accomplish while awake.

The shoemaker really isn’t aware of where the shoes come from. In fact, as time goes on, and as he receives more and more recognition for the quality of his workmanship, he begins to believe that it’s solely due to the little bit of work he does while he’s awake, threading the laces and adding a little polish. He learns to ignore the 95% of the work that’s done while he’s asleep.

Now, imagine someone comes to ask him why his shoes are so exceptionally crafted. Would he admit the truth and say he doesn’t know? No, pride and genuine lack of knowledge would keep him from saying that. He has no idea what he does while he’s asleep. It’s almost as if someone else did the work for him. His conscious brain would kick in and come up with some perfectly rational but completely untrue explanation. Clotaire Rapaille, in his book The Culture Code, cites an example of this:

In a classic study, the nineteenth-century scientist Jean-Martin Charcot hypnotized a female patient, handed her an umbrella, and asked her to open it. After this, he slowly brought the woman out of her hypnotic state. When she came to, she was surprised by the object she held in her hand. Charcot then asked her why she was carrying an open umbrella indoors. The woman was utterly confused by the question. She of course had no idea of what she had been through and no memories of Charcot’s instructions. Baffled, she looked at the ceiling. Then she looked back at Charcot and said, “It was raining.”

This is what happens in almost every instance of market research. Our buying decisions are like the shoemaker’s shoes. They’re usually quite good, but we have little idea how they came into being.

For most of the history of marketing, we’ve been restrained by the limitations of market research. It’s only recently, through advancements in cognitive psychology and brain scanning technologies that we’re beginning to get a glimpse of what might actually be happening. My next post (tomorrow) why it’s important that we keep trying.

Satisficing, Bounded Rationality and Search

150px-HerbertSimonHerbert Simon came up with some pretty interesting concepts, among them satisficing, bounded rationality and chunking.

Before Simon, we commonly believed that humans came to optimal decisions in a rational manner, based on the information provided. We took all the data that was accessible, weighed pros and cons and used our cortexes to come to the best possible outcome.

Simon, in effect, said that this placed to high a load on us cognitively. In many cases, there was simply too much information available, so we had to make choices based more on heuristics, cutting the available information down to a more manageable level. He called this “satisficing”, a blend of satisfy and suffice. And Simon started saying this a half century ago. Imagine how this translates to the present time.

We have never had more information available. At the click of a mouse, we can access huge amounts of information. There’s simply no way we can process it all and come to rational decisions. And this brings us to another concept, that of bounded rationality. We’re more rational about some decisions than others. It depends on a number of factors, including risk, emotional enjoyment and brand self identification. Think of it as a chart with three axes. One axis is risk. We put more rational thought into decisions that expose us to greater risk. In consumer decisions, risk usually equates with cost, but in B to B decisions, it could also include professional reputation (related to but not always directly tied to cost). We’re going to put a lot more thought into the purchase of a car or house than that of a candy bar. Another axis is emotional enjoyment. This is a risk/reward mechanism to most decisions, and if the reward is one that is particularly appealing to us, we tend to be swayed more by emotion than rational decision. If we’re planning a holiday, we may make some irrational decisions (or at least, they might appear that way to an outsider) based on a sense of rewarding ourselves. We’ll treat ourselves to a few nights in a 5 star resort, when the 3 star resort would offer greater overall value. The final factor, and one that is usually buried somewhere in our subconscious, is how we use brands or products to define who we are. Now, no one usually admits to being defined by a brand, but we all are, to some extent. This touches on the cult-like devotees that some brands develop. Harley Davidson, Rolex, BMW, Apple and Nike all come to mind. Is a Rolex a rational choice? No. But a Rolex defines, to some extent, the person wearing it. It says something about the person.

Bounded rationality says that there are boundaries to the amount of rational thought that we can and we want to put into decisions. The amount we decide is sufficient depends on the three facts discussed.

Now, the use of Search tends to plot somewhere along this 3 dimensional chart. If risk is high and brand identification is low (buying software for the company), there is a high likelihood that search will be used extensively. If risk is low and brand identification is high (i.e. buying a soft drink or a beer) there is almost no likelihood that search will be used. In this case, the two factors usually work inversely to each other. Emotional enjoyment isn’t as directly tied to search activity. We will do as much (or as little) searching for a purchase that will give us great enjoyment as for those that won’t.

It’s interesting to watch how these factors impact search intent and behavior. Satisficing leads to a classic sort of search behavior, what I call I category search, where we use fairly generic, non branded queries that broadly define the category we’re looking at. Let me give you an example. Tomorrow my wife and I are headed to Europe for a week. We’re going to spend a few days in Portugal, then fly up to London for SMX (where I’ll be talking more about these ideas in some of my sessions). We’re flying into Lisbon, then renting a car and driving down to the Algarve region. I have GPS navigation software for my PDA, but only for North America. I wanted to get European software, but because of the limited use of it, I didn’t want to spend too much. The developer of my North American software didn’t make a EU version, so I turned to search to find a suitable candidate. Here there was no brand identification, some degree of risk (if it didn’t work in Europe, I’d be lost, literally) and no emotional enjoyment factor. My first search was what I call a “landmark” search. I wanted to find some sites to plot the landscape. Sites that listed and compared my alternatives would be ideal matches to my intent.

I searched for “pocket pc gps software”, knowing that “gps software” would be too broad. I soon found the sites were pretty much all about North American versions. Few of them offered or reviewed European versions. I spent several minutes on the TomTom site trying to order a European version from Canada but to no avail. Apparently TomTom doesn’t believe people in North America would ever choose to drive in Europe.

In classic “satisficing” behavior, I wanted to cut my research workload by setting some basic eligibility criteria: it had to work on a Pocket PC, it had to be reasonably priced (under $100 preferably) and it had to offer coverage for all of Europe (we’re going back to France and Italy next year and I’d like to use it then as well). My next search was for “pocket pc gps software europe”. This gave me what I needed to begin to create my satisficed list. Ideally, we want 3 or 4 alternatives to compare. I did find the TomTom choice, but I was already frustrated with this, and the price was over my threshold. Destinator also offered an alternative that seemed to be a little better match. It matched all the criteria, appeared to have some decent reviews and was available on eBay for about $75, including shipping. Sold! Was it the optimal choice? Maybe not. If I had spent hours more doing research, I could have probably found a better package or a better value. But it was good enough.

Chunking has to do with cognitive channel capacity, and the amount of information we can store in our heads, accessible for use. Again, we tend to maximize the available slots by creating chunks of information, grouping similar types of information together.

When you look at Simon’s work, even though the majority of it far preceded search engines, it sheds a lot of light on how we use search in a number of cases. If you want to tap into user intent, I would recommend finding out more about bounded rationality and satisficing. Chunking is probably worth a look as well.

Interfaces are only Skin Deep

Steve Haar had a great comment on my post about Ask breaking through in the search market share battle:

I agree about the interface being much better with Ask. But, what about the search results? I took a look at them compared to the others and, between sites for adsense and dead links, the results were so poor I was embarrassed for them. I wonder how many of the searches were from repeat users vs once and gone?

I think Steve points out a fundamental concept that we might tend to forget from time to time. The best interface on a piece of garbage just gives you nice looking garbage. Now, I’m not saying that Ask is garbage. But I’ve seen some cases (and heard anecdotally many more) of some issues with spam and I do think they have some work to do. Ultimately, it’s the quality of the results that will determine marketshare. In fact, a nice interface on top of poor results will kill Ask quicker than ever, as it draws more trial users (as Steve alludes to) and generates more negative word of mouth. This is exactly what Ask doesn’t want to happen.

I’m the first to speak up about the importance of the user experience, but it’s important to remember that the interface is only one small part of that. Ultimately, there needs to be enough under the hood to meet and exceed the user’s expectations. Steve (and others) are indicating that Ask might be falling short in the relevancy horsepower department.

The Other Long Tail of Search

smallcoverIn 2006 Wired Editor Chris Anderson released The Long Tail, and suddenly we were finding long tails in everything. The swoop of the power law distribution curve was burned on our consciousness, and search was no exception. Suddenly, the hot new strategy was to move into the long tail of search, those thousands of key phrases that individually may only bring a handful of visitors, but in aggregate, can bring more than the head phrases. At Enquiro, we were no exception. But lately, I haven’t heard as much about long tail campaigns. And I think it’s because our thinking was a little flawed.

One of the key elements of a long tail marketplace is that there can’t be a scarcity bottleneck. Shelf space has to be unlimited, production, distribution, etc. The economics of supporting a long tail have to make sense. There has to be little to no overhead in making a vast selection available to the market. The ideal example is digital entertainment. Once an MP3 is encoded, the only overhead for the distributor is a couple of megabytes of storage capacity.

That’s just not true in search management. It takes time to set up a campaign for a keyword. It takes time to organically optimize for it. There’s significant campaign management overhead associated with search. From a resource perspective, search marketing is expensive, and as anyone who’s tried to recruit a search marketer can tell you, there is no abundance mentality at play here. Sure there’s thousands of keyphrases that may potentially bring one or two visitors a month, and if you add them up, it would be hundreds of thousands of potential leads, but we just can’t move from the head to the tail because we don’t have the time.

Engines tried to open up the bottle neck by offering broad match, but results from broad match campaigns are dramatically less than spectacular. In many cases, we needed to go back to the more granular control that comes with exact match to keep performance levels where we needed them to be.

But in perusing some of our spreadsheets from past studies, and with Anderson’s Long Tail curve fresh in my mind, I found another long tail in search. In several studies, we’ve tracked click throughs by position. Look what happens when you take the most popular click through position, the number one organic spot, and then work down by position:

clickthrough tail

As you can see, we have another long tail. Now, due to the scale of the graph, it looks like the tail goes to zero. This isn’t the case, but by the time we get to the second page of results, we tend to hit percentages that are fractions of 1%. As long tails go, this is skinnier than most, showing the power of position on the first page of results.

But now let’s combine the two long tails. If you take the head being a #1 organic ranking for your most popular phrase, and work down from there, the results get quite dramatic. For the sake of this example, I’ll move down one position for each keyword. Here’s a table showing the numbers:

Keyword Volume Position X Click Through % Clicks
#1 46500 Org 1 26.35 12252
#2 38450 Org 2 13.05 5017
#3 32500 Org 3 10.2 3315
#4 23400 Spon 1 8.75 2047
#5 15750 Org 4 5.35 842
#6 12450 Spon 2 4.125 513
#7 8750 Org 5 3.875 339
#8 5250 Org 6 2.875 150
#9 4325 Side 1 2.5 108
#10 2750 Org 7 1.875 51

Now, let’s plot that on a graph:

clickthrough tail2

Given that search is a resource hog in terms of manpower, I argue that we’d be far better working our way up the tail rather than down it. And by moving up the tail, I mean the click through by position tail. By moving up the page, even a position or two, with relatively popular keywords, you can leverage the compounding effect of the two curves and dramatically improve your campaign performance.

Ask Beginning to Break Through

For quite some time, I’ve been wondering if my user “Spidey-Sense” was wonky. From everything I saw about the Ask 3-D interface, it should have been gaining marketshare. Also, for all my preaching about build a better user experience and you’ll reap the rewards, Ask’s reaping appeared to be a little on the grim side, lingering at about 3.5% of the market. But finally, according to a recent post by Bill Tancer over at Hitwise, my instincts seem to be back on track. Take a look at this graph:

ask

Ask is finally making a move. And their “share of search” has moved up from 3.49% of executed searches in August to 4.32% of searches in October, a bump of 23.7%. That’s huge. Bill wonders if it has anything to do with the ads Ask is running. I suspect it has a lot more to do with a great interface and some user generated buzz that’s beginning to catch some ears. Michael Ferguson and his team did exactly what they needed to do, shake things up by thinking about what users want.

Ask’s strategy has always been to be your first second choice. They don’t ever expect to knock Google out of the lead, but what they want to do is be the place you turn when you find Google just isn’t cutting it. So their move to 3D made a lot of sense. For certain types of searches, notably entertainment or discovery searches, users want something more than Google’s spartan, click and get out interface. They want a stickier, richer, more visual appearance. They want Ask 3D. We found the interface tested pretty well in our recent Search:2010 Eye Tracking study for entertainment based searches.

In fact, Marissa Mayer at Google paid Michael and his team the ultimate compliment when she mentioned the likelihood of Google moving to more of a portal, encyclopedia type format sometime in the future. So..that would make Google more like..Ask!

I will be watching with interest Ask’s marketshare numbers over the next 6 months. Again according to Hitwise, the jump regains all the marketshare they’ve lost in the last year, and puts them a lot closer to the current number 3, Microsoft, who is sitting just 3 and a half points ahead at 7.83%. Microsoft has been on a continuous slide for the past year, dropping 3 full points. Yahoo seems perpetually stuck between 22 and 23%. Google has captured most of the fallout, adding those 3 points to their marketshare numbers. But Google’s .5% drop in the last month seems to have gone directly to Ask, showing that the “First Second Choice” strategy might be paying off. Like Jim Lanzone said to me once, “Our goal is to take our 20 million users, who are currently using us twice a month, and bump that up to four times a month. That doubles our market share,” At the time Lanzone made the comment, Ask was sitting with about 2.5% marketshare. If you look at the table below, Ask has just about hit their goal.

 

Percentage of U.S. Searches Among Leading Search Engine Providers
Domain

Sept-07

Aug-07

Sept-06
http://www.google.com

63.55%

63.98%

60.93%
search.yahoo.com

22.55%

22.87%

22.29%
search.msn.com

7.83%*

7.98%*

10.87%*
http://www.ask.com

4.32%

3.49%

4.28%
Note: Data is based on four week rolling periods (ending 9/29/07, 9/01/07; 9/30/2006) from the Hitwise sample of 10 million US Internet users.

* – includes executed searches on Live.com and MSN Search.
Source: Hitwise

But I don’t think Ask is going to stop there. Within 6 months, you’re going to be reading stories all over the web about how Ask bumped Microsoft out of the #3 spot. It will be David vs Goliath, or in this case, Barry (Diller) vs Bill (Gates). Ask is on a roll, and thanks to Bill Tancer’s revisiting of the numbers, I have regained enough confidence to say, “mark my words”.

Search and the Digital CPG Shelf

First published October 25, 2007 in Mediapost’s Search Insider

Last April, James Lamberti from comScore, Randy Peterson from Procter and Gamble and I (representing SEMPO) grabbed a relatively quiet corner at the New York Hilton to talk about a potential research project. Here was our wish list:

–    A study that tied together online activity to offline purchase behavior
–    A study that identified the impact of search in a category not typically one that would be identified with search marketing
–    A study that would attempt to quantify the leveraged impact of search with brand advocates

Search and CPG: Are You Kidding?

As you can see, these were pretty lofty targets to shoot for. Choosing the product category was done at that table. What is the last category you would think of as generating significant search activity? Consumer packaged goods. After all, aren’t these either replenishment purchases, where we keep buying the same brand over and over, or a non-considered purchase, where we’re not really concerned with doing much research? Why would we need to turn to a search engine to figure out which toothpaste to buy, or which would be the right chips for Sunday’s ball game? We reasoned that CPG had the “Sinatra” Factor going for it: If search can make it here, it can make it anywhere.

To be honest, we really didn’t know what to expect, but comScore, together with a lot of help from Yahoo and Procter and Gamble, managed to come up with a workable study design. SEMPO jumped on board as a co-sponsor and we put the study out in the field. This week, with numbers crunched and charts drawn, the results were released in a study labeled The Digital Shelf. After several months of holding our collective breaths, we were about to see if people had already locked CPG brands into their brains, eliminating the need to search for product information.

Apparently not.

People went online for CPG information — in fact, to a significantly higher degree than even our most optimistic predictions.  Over a 3-month period, comScore recorded over 150 million visits to CPG websites in four categories: Food Products, Personal Care Products, Baby Products and Household Products. Those are numbers no marketer should ignore. But even more significantly, search drove significant portions of that traffic, from 23% of all visits in Household Products to 60% in Baby Products.

It’s not just automotive or travel that drive search traffic. We search for recipes, how to get the stains out of carpets, the most eco-friendly disposable diaper and yes, even the nutritional information for potato chips. We search, a lot!

And our searching sets us apart as a consumer segment. Searchers tend to be more interested in product information, comparing against competitors and what they need to make a purchase decision. Non-searchers are more interested in special offers and coupons.

Searchers spend more, about 20% more  — in all the categories in the study. In fact, in the Baby Care Category alone, people searching for information and eventually purchasing could result in almost $12 billion in sales.

Search = Opportunity

But perhaps the most compelling finding was this: People search because they’re comparing alternatives. This means they’re not locked into a brand. They could very well be your competitor’s customer right now. Non-searchers are more likely to go directly to a site because they do have a brand preference. They’re just looking for a bargain on that brand. The study found that 36% of searchers had recently switched their brand, compared to 29% of non-searchers. And, interestingly, searchers are less motivated by price. Only 27% of searchers switched because of price, compared to 38% of non-searchers.

So, the study delivered on our original wish list, and then some. It showed that search is a significant influencing factor in the most unlikely product category of all, the stuff on your pantry shelf or under the sink in your bathroom. In fact, I have yet to see a study done on any product category where search didn’t blow the doors off the competition in its effectiveness in connecting with customers. So perhaps the biggest question left unanswered by the study is this: Why are all those branding dollars still going everywhere but search?

The Wisdom of Consumer Crowds?

Following up on the theme of the rewiring of our brains, is the internet making us smarter consumers as well? There certainly seems to be evidence pointing in that direction.

A study by ScanAlert  found that the average online shopper in 2005 took 19 hours between first visiting a store and completing a transaction. In 2007, that jumped almost 79% to 34 hours. We’re taking longer to make up our minds. And we’re also doing our homework. Deloitte’s Consumer Products group recently released research saying 62 percent of consumers read consumer written product reviews on the Internet, and of those, more than 8 in 10 are directly influenced by the reviews.

In James Surowiecki’s Wisdom of Crowds, he believes that large groups, thinking independently with access to a diversity of information, will always make a better collective decision than the smartest individual in the group. Isn’t the Internet wiring this wisdom into more and more purchases? When we access these online reviews, we’re in fact coming to collective decisions about a product, built on hundreds or thousands of individual experiences. As the network expands, we benefit from the diversity of all those opinions and probably get a much more accurate picture of the quality of a product than we ever could from vendor supplied information alone. The marketplace votes for their choice, and the best product should theoretically emerge as the winner.

Of course, nothing works perfectly all of the time. As Surowiecki points out, communication can be an inexact and imperfect process, and information cascades based on faulty inputs can spread faster than ever online. But it’s also true that if a cascade leads to rapid adoption of an inferior product, we’ll discover we’ve been “had” faster and this news can also spread quicker. The connections of online make for a much faster dissemination of information based on experience than ever before, ensuring that the self correcting mechanisms of the marketplace kick into gear faster.

There’s a pass along effect happening here as well. For social networking buffs, you’ve probably heard of Granovetter’s “Weak Ties”. Social networks are made up of dense, highly connected clusters, i.e. families, close friends, co-workers. The social ties within these clusters are strong ties. But spanning the clusters are “weak ties” between more distant acquaintances. The ability for word to spread depends on these weak ties. What the internet does is exponentially increase the number of weak ties, wiring thousands of clusters together into much bigger networks than were ever possible before. This allows word of mouth to travel not only in the physical world but also in the virtual. I looked at a fascinating follow up study to Granovetter’s where Jonathan Frenzen and Kent Nakamoto also looked at the value of the information and the self interest of the individual and their “strong ties” within a cluster as a factor in how quickly word of mouth passes through a network.

Deloitte’s study graphically illustrates the weak tie/strong tie effect. 7 out of 10 of the consumers who read reviews share them with friends, family or colleagues, moving the information that comes through the weak ties of the internet into each cluster, where it spreads rapidly thanks to the efficiency of strong ties. This effect pumps up the power of word of mouth by several orders of magnitude.

But are we also becoming more socially aware in our shopping? The research by Deloitte also seems to indicate this. 4 out of 10 consumers said they were swayed by “better for you” ingredients or components, eco-friendly usage and sourcing, and eco-friendly production or packaging. The internet wires us into communities, so it’s not surprising that we become more sensitive to the collective health of those communities in the process.

What all these leads to is a better informed consumer, who’s not reliant on marketing messaging coming from the manufacturer or the retailer. And that should make us all smarter.

Are Our Brains being Rewired?

I have to start out by thanking Nico Brooks and Jess Gao. Without intending to, they both provided me more than enough fodder for a rather lengthy column in Seach Engine Land on Friday.

Nico is the Chief Search Strategist at Atlas. Jess is our intern at Enquiro, who’s currently working towards her doctorate, specializing in cognitive psychology. Through different paths, they both gave me some major brain melting ideas to chew over. I’m still digesting, but you can catch the thought process in action on my column.

But consider this. What if our brains are being rewired by the internet? Some of our behaviors are innate. They’re our OEM operating software, put there by the manufacturer. Flight or fight. The need to procreate. The appreciation of beauty. This stuff is hardwired.

But some of our behaviors are learned. We’ve developed them as we go. The things sit in our temporary memory caches, and we can adjust them if they’re no longer working. The thing that started all this was how we learn to navigate a physical environment. First we look for landmarks, then we memorize routes, then we put the two together to create a cognitive map. Nico’s suspicion (and Nico, I hope I’m capturing the essence of the idea accurately) is that our need to identify landmarks and even our ability to memorize routes is probably innate. It’s just how we are programmed to get around. But cognitive mapping, at least in the essentially rectangular grid pattern that is common in the Cartesian coordinate model, is a learned behavior. Rectangles have no place in the n dimensional space of online, so as we spend more time navigating online, will we change our mapping process?

Then, with Jess, we had a great chat about how we perceive things, especially ads. There’s a great introduction to selective perception that I would urge you to check out. In recent studies we’ve done at Enquiro, one of the interesting findings has been that the more intrusive the ad, the less it seems to work. It registers high in the first stage of perception, stimulation, and manages to succeed in the second, registration, but fails in the last two stages, organization and interpretation.

Other conversations I had this week, that didn’t make it into either of the columns. On Thursday I was in New York for Google’s B to B Summit and had a chance to chat with Mark Martel, who supports the B to B Tech Sales Vertical at Google. Mark has a healthy intellectual curiosity and I always enjoy chatting with him. We discussed schemas and how important they are in the process of perception. Then, on Friday, I was in Toronto chatting with the Yahoo Canada gang, including Maor Daniels and Adina Zointz (what a great name, literally covering everything from A to Z!) and we talked about how quickly we’re learning to judge the authenticity of content online. It’s as if our bullshit filters are more finely tuned than ever.

I’m definitely on a riff here, but there’s a lot of threads coming together. Even in someone of my ever upwards creeping years (I’m 46) I suspect my synapses are under construction. Old routes are being torn out and new ones are being built. And with my daughters, many of the paths are being built differently right from the start. The routes that were so important to me in grade school, times tables, rote memorization, etc, are becoming overgrown with weeds through lack of use. But new routes I never even thought of, like how to do homework, carry on an online chat and watch the TV with one eye, are being upgraded into major turnpikes. Multitasking is a major operational imperative now, and selective perception is kicking into overdrive.

Anyway, to further dive into some of the things on my mind, here’s some of the columns where I’m beginning to open up some of these ideas to the fresh, online air:

Infomediating a Broken Marketplace – a look at Hagel and Singers Infomediary model from their book Net Worth. Is Google aiming to be the ultimate match maker in the marketplace?

4000 Ads a Day, and Counting – Part One of the Infomediary Doubleheader, looking at the disconnect between customers who just want the facts, and advertisers that just want to control our buying habits

Some Big Ideas for a Friday – Some musings about how we perceive advertising, based on recent studies we conducted, and how we might be remapping the perception process

How We Navigate Our Online Landscape – The original exploration of landmark, route and survey knowledge and how it may map (or not) to how we navigate our online space

And please, do me a favor. This is all stuff I want to explore further in the book. If you think I’m full of bullshit, call me on it. Share your thoughts. Post a comment. Start a dialogue. I know it’s a pain in the ass posting comments on blogs because of spam, but PLEEASSSE take a few moments to do so. Or drop me an email.