Category Archives: User Behavior

The Rule of Three in Search

First published July 20, 2006 in Mediapost’s Search Insider

Once again, I find myself up to my earlobes in eye-tracking data. I have no one to blame, as I got myself into this mess when I made the well-intentioned but poorly thought out promise to have the first draft of a study done by the time I head out on vacation at the end of the month.

In wading through the sessions (about 420 of them) sometimes new insights rise to the top–and sometimes my eyeballs just roll back in my head as my hands jerk spasmodically on my keyboard and drool runs down my cheek. Luckily, this week it was the former.

In this study, we are looking at interactions with Google, compared to MSN and Yahoo. Recently, one finding in particular seemed to be screaming out to be noticed. Being a compassionate sort of researcher, I listened.

When we looked at interactions with the top sponsored ads, there was a notable difference between MSN, Yahoo and Google. On MSN and Google, the percentage of clicks happening on these top ads seemed to be in line with previous studies done both by us and by others. But the amount of activity on the Yahoo ads seemed to be substantially higher. We started out by looking at first fixations, or the first place people looked on the page, even for a split second. Here, the engines were all in the same ball park, with 83.7 percent of first fixations in top sponsored ads for Yahoo, compared to 86.7 percent for MSN and 80.6 percent for Google.

Then, we looked at where the first activity on listing happened; where on the page did people start actually scanning listings? Google held a good percentage of eyeballs, keeping 12.4 percent of the users, while MSN had a significant defection issue, losing 36.6 percent of the people who first fixated in the top sponsored ads. But Yahoo lost the fewest, with only 5.5 percent choosing to look elsewhere. And finally, Google had 25.8 percent click-throughs on these ads, and MSN had 16.7 percent (yes, this is low, but MSN was dealing with a number of issues at the time of the study). Yahoo led the pack with a 30.2 percent click-through rate. In fact, for the first time ever in our research, a sponsored link (the number one top sponsored) out-pulled the No. 1 organic link, at click-through rates of 25.6 percent vs. 14 percent. This was a complete reversal of the click-through ratios we saw on the other two engines.

For whatever reason, Yahoo’s top sponsored ads seemed to be locking searchers into their part of the results page to a much greater extent than Google and MSN.

Why? What the heck was going on? Better ads? Not really. If anything, Google’s ads seemed a touch more relevant.

Location, Location, Location

Part of it was real estate. Another interesting comparison we did was to look at the percentages of screen real estate devoted to various sections of the page. Yahoo has gone out of its way to make the top sponsored ads the dominant feature on a results page at 1024 by 768 screen resolution. At this size, the ads take up 23 percent of the real estate, compared to approximately 16 percent for Google and Yahoo. This pushes organic listings on Yahoo perilously close to the fold.

And there, as I stared at the screen shots of fully loaded (maximum ads and vertical results showing) Google, MSN and Yahoo results at standard resolution, a possible answer revealed itself. On Google, three top sponsored ads, three OneBox results, and three visible organic listings. On MSN, the same three:three:three presentation. But on Yahoo, there were four top sponsored ads, three vertical results, and just one and a half organic listings were visible.

The Rule of Three

Hmmm, three, three and three. There was something there, niggling in the back of my mind. Quickly, I did a search for the “Rule of Three” and sure enough, there it was. We humans tend to think in triplets. Three is a good number to wrap our mind around, and we see it in all kinds of instances. We tend to remember points best when given in groups of three, we scan visual elements best when they come in threes, and we like to have three options to consider. Think how often three comes up in our society: three little pigs, three strikes, three doors on “Let’s Make a Deal,” three competitive quotes. It’s a triordered world out there.

So is it coincidence that search results tend to be presented to us, neatly ordered in groups of three? I think not. It strikes me that this engrained human behavior would probably translate to the search engine results page as well.

The Ruler-breaker

MSN and Google tend to adhere to the rule of three in their layouts (depending on whether or not Google serves three top sponsored ads). Our choices are conveniently presented in neat trios, with logical divides between each.

Yahoo breaks the rule by tipping the balance in favor of the top sponsored ads. First, it provides four results, not three. Does this mean we need to spend a little more time up in these results, trying to fit one extra one into our limited memory slots? That appears to be the case, with people spending an average of 4.6 seconds in the Yahoo top sponsored results in our study, compared to 2.4 seconds for Google and 1.73 seconds for MSN.

Second, it only gives us one visible organic listing to consider. It breaks our natural desire to have three alternatives, thereby reducing the Promise of Interest for the organic listings. In effect, on the screen of results most people would see on Yahoo, we only have one alternative, the top sponsored ads.

An earth-shaking discovery? Perhaps not. But cut me some slack. I’ve been looking at eye-tracking data daily for three months now, spending about three hours each day looking at interactions with the three engines. I think it’s time I took the three other members of my family on a three-week vacation, during which we’ll be visiting three countries. Wait a minute! Do I sense a pattern developing?

Branded Terms in Search Results: Pre-Mapping in Action

First published July 6, 2006 in Mediapost’s Search Insider

Two separate occurrences in the last little while have lent credence to a behavioral occurrence we’ve seen in many of our studies.

First, I was sitting in on a meeting where an agency (not ours) was reporting on the performance of its sponsored search campaigns and was ecstatic with the performance of its branded term phrases, which were outperforming every other keyword bucket both in terms of click-throughs and conversions. While giddy with delight, company executives were at a bit of a loss to explain why.

On a similar track, a search marketing firm has recently released some results that looked at cannibalization of search campaigns when you are buying terms where you also hold top organic position. Again, they found this is most likely to happen when you’re buying branded terms.

While neither of these examples should be surprising to a seasoned search marketer, we’re all interested to know the reasons behind this interplay between organic and sponsored, particularly on branded terms. The answer, as it so often does, lies in looking more closely at what the search user is doing.

Pre-Mapping: A Theory

After looking at thousands of search sessions in detail, one thing is becoming clear. Searchers are incredibly adept at focusing in on just the portion of the results page that interests them. The time required to relocate to the prime real estate is literally a fraction of a second. Yet that real estate isn’t always the same spot. It varies depending on query and intent. It also varies by user, but even the same users will navigate the real estate of the listings in very different ways, depending on what they’re looking for.

Pre-Mapping supposes that we’ve interacted with search results pages enough to know the sections of real estate we typically deal with. We know where the top sponsored ads are and what they are. We know about where the top organic listings start. And in our minds, we already have a good idea of the type of site we’re looking for and approximately where we expect it to appear. Before the page ever loads, we’ve already mapped out the sections that would appear to hold the greatest promise to deliver on our intent. As the page loads, we do a split-second scan to get our bearings (orient in the top left corner, see how many top ads there are, see where organic starts) and then we go to the part of the map we’ve predetermined to be our best starting point.

Theory in Practice

Let’s run through a few examples. Imagine you’re looking for the possible side effects of a medication. The types of sites you would be looking for would be authoritative information sites, either the official site for the medication, a recognized health portal or possibly a government information site. In this case, you may be leaning more towards objective sites, rather than the pharmaceutical company’s own site. After launching the search (the name of the drug) you’ll quickly filter out, or thin slice, any commercially oriented sites. In this type of interaction, you’ve determined through pre-mapping that your area of greatest promise is not likely to be in the sponsored ads. You also expect the official site to rank No. 1 organically, so your area of greatest promise is probably in the No. 2 to 5 organic rankings, where you expect the types of sites you’re looking for to sit. In a split second, you’ve narrowed the real estate where you’ll start your active scanning to about 10 percent of the total real estate.

Now, let’s say you’re looking to renew your auto insurance. You’ve already checked out a few quotes online, but before you commit to any, you want to see how your current carrier compares. You’ve also pre-mapped the page in this case. Here, you expect your company to be bidding for the term ( “Brand Name auto insurance”) and because it’s a commercially oriented query, you assume that the sponsored listing would take you to a page where you could get a quote. Your area of greatest promise is the top sponsored ads. Again, you do your orientation scan to find your bearings in the upper left, but in this case, you would start right at the top sponsored link and work your way down the page until you find a link to the carrier in question that offers the promise of giving you a quote.

Theory Applied

Considering these two examples of user behavior, you can easily see what was happening in the two anecdotes I cited at the beginning of this piece. Brand terms will convert like gangbusters in the top sponsored location, because when a brand term is used, it’s very likely that the user has pre-mapped and is expecting to find that site in those top sponsored spots.

Similarly, you will find significant cannibalization because when users have pre-mapped, they start at the top and work down. They’ll hit the sponsored result before they hit any organic result that might appear. They’re looking for the quickest route, and in this case, the sponsored listing is giving it to them.

The likelihood to pre-map, and what this means for interaction for the page, lies in that deep dark place where all the answers to search engine success lie, the mind of your target prospect. Spend some time exploring it.

Usability and Asinine Comments from the Bay

Had a chance to catch one day of Jakob Nielsen’s Usability Week in San Francisco. Yesterday, I sat in on the eyetracking session and saw the results of the Nielsen Norman’s just completed study (numbers are still being crunched as we speak).

It was heartening to see that many of their findings mirrored our own, including F shaped scanning patterns, quick scans of pages and aversion of ads and large graphic blocks. It was in this last category that the asinine comments part comes (that’s why you’re really reading this, isn’t it?).

Jakob was demonstrating interaction with the home page of jcpenney.com. (The picture that I’ll be talking about has changed, but the basic page structure is the same). The heat map image showed clearly that the big block graphic, in this case a picture of a bed with a colorful spread, with some promotional text inset in the upper left and the lower right, received virtually no scanning. All the scanning was in the top navigation bar. The large block graphic “fenced in” the scan area, cutting users off from other promotional information that lay below the graphic. We saw the same thing occur with the Bombay Company site in a eyetracking study we did for MarketingSherpa (see below).

bombays

Some hot shot designer in the room decided to take exception with the proof in from of him, and called out some of the examples that Jakob has shown of large graphics that had received no scanning. He used words like apex composition and other regurgitated terms from a graphic design university text book to show that all the sites adhered to basic design principles and that the theoretical composition of the JCPenney picture was in fact spot on, drawing the eye from one promotional headline to the next. Jakob patiently pointed out the obvious, that the theory breaks down, because as the heat map clearly showed, no eyes were even being attracted, let alone drawn to any headlines. We settled back in our chairs, silently cheering the adroit handling of the blow hard in the back. Much to our amazement, the guy wouldn’t give up, continually going back to the point that the theory is right and works, despite evidence on a screen roughly 40 by 30 feet to the contrary. The mic finally had to be taken away from him.

A couple points here. Theories are theories, not fact. Heat maps are facts, at least for the sample of people in the study. And while you may argue that a sample of a couple hundred (the n of the NN/g study) isn’t representative, I would disagree. We’ve done enough to know that consistent behavior in eyetracking starts to emerge at about 10 people, then defines itself very clearly at 20 to 30 people. So designers, you just may have to forget everything you learned, because the way people interact with information is changing faster than new theories can be created. You have to keep an open mind.

Second of all, this guy was approaching this from a print paradigm, not an online one. His spouting of picture composition and eye flow comes from centuries of guessing about how we look at images. I remember talking to a university arts professor once who was really excited about eye tracking because we could finally find out if all the “crap that’s been spouted about how we look at paintings is even true or not”. I’m not saying century old principles are wrong, but you have to consider them in the appropriate context. Take our J.C. Penney picture. Mr. Design Dictionary is correct. The flow of the bed spread and the contours of the bed should hypothetically draw the eye from one headline to the other, if the eye entered the picture in the first place. In the print advertising world, photos act as an attractor. They grab the person who is reading adjacent, usually non relevant content, and pull them over to the ad. They are the entry point. If they do their job efficiently, you have altered the intent of the prospect. They have switched from reading a story to looking at your ad. The job of the photo is to channel this new intent to the right place.

With a website, you have the full intent of the user. That’s why they came to your site. A large block graphic gets in the way of that intent, and will be thin sliced out of the way. Worse, it could block the user from seeing the content on the site that they’re there to see. All the composition theory in the world won’t prevent that. Jakob’s point wasn’t that the picture was composed incorrectly; it was that the picture was a waste of valuable home page real estate.

Probably the most valuable thing I took from yesterday was a comment Jakob made as an aside. Branding online comes from the experience, not the exposure. This was in response to another comment somebody made about large graphics being present for branding purposes, and the seeming contradiction between the need for branding and best practices for usability. Online, a successful brand engagement and a successful user experience are the same thing. If you deliver efficiently on a user’s intent and make their online experience a pleasure, you will build more brand equity than you could ever build with gratuitous flash files, streaming media and huge graphics. The two aren’t mutually exclusive, but all too often online, the designers win at the user’s expense.

RSS Feeds vs E-mails: More Eyetracking Data from Jakob Nielsen

Jakob Nielsen’s Neilsen Norman Group just released an eyetracking study looking at scan patterns of e-mail newsletters vs RSS feeds. The summary results? People spend more time scanning newsletters, but are ruthless in scanning titles that pop up in their newsreaders. Again, both Jakob’s studies and ours seem to keep coming to the same conclusions, we’re evolving a very advanced form of “thin slicing” when we interact with information online. We have to, as there’s an overload of stimuli. I’m heading down to San Francisco next week to spend some time at Jakob’s usability summit, and hope to chat with him more about this.

Friday’s Fodder Folder Clear Out

After almost 2 months of blogging, I’m started to get a system. Usually, when I see items of interest come through my inbox or have interesting conversations, I file them away for a future blog post in a folder called Blog Fodder. Well, the folder is overflowing, and I don’t have time to do full posts, but I did want to pass them along, so I’m cleaning house today.

More Search Research

The Daves (Williams and Berkowitz) and the rest of the gang over at 360i and SearchIgnite released a study looking at the value of multiple clicks on a search ad. This is an interesting indicator of the complexity of the search interaction in a purchase life cycle, something that needs a lot more light shone upon it. I remember Greg Sterling and I talking at one point at a SES session about the messy and twisting nature of a consumer’s online path in a purchase cycle. I’m happy to say that research companies are starting to focus on this Gordian knot (and I’m pretty sure it wasn’t named after me).

ComScore is one of those jumping on board with a recently announced study to look at the influence of online research on offline purchase. The value here is huge, just never quantified that well (or at all) and the ComScore study should be a step in the right direction. I’m hoping to chat with VP James Lamberti more about the study next week. If I’m able, I’ll drop a few tidbits about what they’re looking at.

OMD and Yahoo also released a study looking at this, called the Long and Winding Road. Speaking of Greg Sterling, he’s got a look at the study on his blog, with links to the press release and a few columns. Not sure how publicly available the study is. If you’re interested, perhaps contact your friendly neighborhood Yahoo rep. Fascinating reading!

The Bulls of SEM

Sapna Satagopan from JupiterResearch is bullish on the future of SEM, saying as the number and size of companies moving into search continues to increase, it will drive SEM outsourcing. At first glance, this seems to contradict the findings from the annual SEMPO survey, which indicates that more companies are bringing this in house. Steven Rappaport, a writer who’s currently working on an online advertising field guide for ARF, asked about this in a conversation this week. I explained that the two seeming different viewpoints are two stages in the same cycle. As companies dedicate more attention and budget to search, they do want to gain control in-house, so they are looking for search expertise to bring on board. While these new “directors of search” oversee search activities, they look for experts in specific areas to outsource to. It’s not really efficient for companies to set up an entire search marketing division in-house, and many companies realize this after going down this road for awhile.

Long Tail and other Musings

Cory Treffiletti wrote a column on the Long Tail model of business that has been exploited expertly by Amazon, eMusic, iTunes and the king of long tails, eBay. This is an idea I’ll have to come back to, as it has fascinating implications for retail. But until then, consider, an internet etail model doesn’t have any of the physical limitations of a traditional store. With virtual inventory, provided by direct suppliers, the store, or site, simply acts as the connector. And with expert use of search, the primary connection vehicle, it becomes possible for an online story to carry everything, but with the inventory infinitely segmentable. This brings about the idea of a mega-online shopping site, which is close to what eBay and Amazon have become. Tie this in with smarter shopping search tools and the social networking WOM power of a MySpace, and you’ve got a convergence model that’s mind blowing in its implications.

Tom Hespos takes a stab at a favorite subject of mine, the transference of control over brand messaging from the advertiser to the consumer.

Relevancy Rules in Sponsored Search Ads

First published May 4, 2006 in Mediapost’s Search Insider

Let me quote some rather startling numbers to you from a recent eye tracking study we did. In the study, we examined where people first looked on a search results page, where they first scanned a listing, and where they eventually clicked.

First of all, we gave participants a number of different scenarios that involved looking to a search engine to help them make a purchase. We used Google, Yahoo and MSN in the study. In all cases, on all 3 engines, the vast majority of people first glanced at the top- sponsored listings. In eye tracking parlance, we call this a fixation, or a momentary pause of the eye. On Yahoo, 84 percent of the first fixations were on the top sponsored listings when they appeared, on Google it was 81 percent, and on MSN it was 87 percent. So, almost nine out of every 10 people start looking at the search results page by at least glancing at the top sponsored listings

The next thing we measured was active scanning. This is where participants started reading a listing. On Google and Yahoo, there was strong correlation with the first fixation point, with 79 percent of the first reading activity on top sponsored for Yahoo, and 71 percent for Google. MSN was another story. While 87 percent of participants first glanced at the top sponsored ads, only 55 percent started reading there. Almost 32 percent of our participants immediately relocated past the sponsored ads.

Finally, we recorded where the eventual clicks happened. In Google’s case, 26 percent of the clicks happened in the top sponsored ads, with Yahoo it was 30 percent, and MSN came in with 17 percent click through on top sponsored.

Here’s what we took from the numbers. On Google, although over 80 percent of searchers started in the top sponsored, only 26 percent found something relevant and compelling enough to click on, and remember, these were commercial, product oriented searches. On Yahoo, 84 percent started in top sponsored, but in Yahoo’s case, about 30 percent stuck around and clicked an ad. And with MSN, something entirely different was going on. It seems that MSN users have a bad case of banner blindness when it comes to top sponsored ads.

Scanning Follows Relevancy

The reason top sponsored ads are effective is because they’re placed in the highest traffic portion of the page. We orient ourselves in the page on the upper left. Our destination is the top organic ad. Top sponsored ads are placed in the middle of the most popular real estate on the SERP. This is shown by the high percentage of fixations that happen in this section.

But our interactions with the SERP are not all about position. We can, very quickly, determine if what’s there is relevant to what we’re looking for. We quickly scan titles to see if the ads presented match our intent. And when I say quickly, I’m talking fractions of a second. We start picking up relevancy without even having to read the listings by determining scent. If the listing has “scent” and it’s a good match, we’ll not only hang around and start scanning the listing, we may even click on it. Otherwise, we do what we intended to do in the first place and skip down to the organic listings. That’s what’s happening on Google and Yahoo. MSN is another story.

The MSN Two-Step

During the study period, MSN was in experimentation mode. It was in the process of dropping Yahoo ads from the top listing and substituting its own advertising, which in most cases wasn’t keyword-driven to the same extent that the Yahoo ads were. This usually meant that the “scent” or relevancy match wasn’t as great. When this happened, we saw almost immediate relocation down to organic results. Users could determine the existence, or in this case, absence of scent in a fraction of a second and relocated down. In effect, it was an example of banner blindness, where they were determining that the top sponsored results weren’t relevant.

The lesson from this for the search engines is that you can’t take position for granted. You have to deliver with relevancy and the greater the relevancy, or at least, the perceived relevancy, the better those top sponsored ads will perform.

Yahoo’s Relevancy Capitulation

Yahoo has learned this over time. In the beginning days of GoTo/Overture/Yahoo, position was determined solely by bidding. When Google came on the scene, it offered a blended approach, where click-through rates also helped determine position. The theory was, the higher the click-through rate, the greater the relevancy.

Yahoo has recently announced integrating click through rates and relevancy into the sponsored positioning algorithm as well. This is the beginning. Soon, message and landing page relevancy will also be factored into the position equation.

When it comes to capturing a searcher’s click, you have to deliver relevancy. It’s not all about position–and this fact will become more true in the future, not less.

Branding, Search and the Definition of Engagement

First published April 13, 2006 in Mediapost’s Search Insider

Currently, the Advertising Research Foundation has an initiative called MI4. Its task is to create a cross-channel measurement of advertising effectiveness that can foster more accountability and facilitate multichannel marketing measurement. They have decided on the concept of engagement. It is a noble endeavor, and one that is much needed in our new, highly fragmented marketing world. But I fear there may be a fundamental chasm that one metric will be unable to bridge.

Joe Plummer, ARF’s Chief Research Officer, offered the group’s first draft of a working definition, “Engagement is turning on a prospect to a brand idea enhanced by the surrounding context.”

The Two Sides of Engagement

The problem, from a search perspective, is that there are two very different forms of engagement seen with consumers, and brand plays a very different role in each.

In most marketing, brand engagement is essential. You have to form a relationship between a brand and the latent or expressed needs and desires that lie with the consumer. Engagement is essential, because you have to form an emotional bond that can rise to the surface and express itself as top-of-mind awareness when consumers are ready to actively consider their options. In this instance, engagement is emotional, intuitive and often subconscious. It is this level of engagement that I think ARF is trying to define by somehow quantifying this emotional bond, referred to in market speak as being “turned on.”

But there is another type of engagement: engagement with the actual act of purchasing. Here, the consumer is engaged with a product, but not necessarily a particular brand. This is the typical point when a consumer will interact with a search engine. And with ARF’s working definition of engagement, I don’t think search will do particularly well in a multichannel comparison.

Branding and Search

One of the issues with search has been its value as a brand-building channel. The prevailing wisdom is that search is not a particularly effective brand-building marketing medium. I believe this to be true, but it’s because we’re trying to apply the first definition of engagement, the idea of engaging with a brand, not a product.

Consider a typical brand engagement measurement. If I did a brand lift study with a typical page of search results, where I showed a consumer the page, some results with brand messaging included, and determined if brand lift occurred, the results would probably be less than stellar. First of all, the act of searching is done with the left brain. It is a rational, logical interaction, not an emotional one. That’s why text-based advertising does well, and graphic or rich media doesn’t. We’re intellectually engaged in a task, and we’re looking for information that will help us succeed in accomplishing that task. We’re not looking to be influenced by an emotionally charged message. In fact, we block anything that smacks of overt commercialism or looks like advertising out of our consideration. We “thin slice” it out of the way. We are not emotionally connected. We are not looking to be “turned on.” We are evaluating our alternatives with a rational view.

When a consumer is interacting with a search engine, the time for brand engagement is already long past. That job had better be done already. Here is how branding does work in search.

Engagement with Buying, Not Branding

When I use a search engine for consumer research, I’m thinking in terms of the specific thing I’m looking for, not a specific brand. Generally, when I start, I will not use a branded search term. I am building a consideration set. Yes, I likely have brands I have an affinity for, but I won’t explicitly include them in my query. I’m looking for the search engine to provide me some alternatives to consider. Typically, searchers will look at four to five results before making their selection. These are usually the top sponsored, and the top two or three organic, results. This represents the prime and very limited “shelf space” of the search results page. If a brand appears that the consumer has an existing affinity for, the chances are good that the site will capture a click-through. If the brand doesn’t appear, the company has likely lost the opportunity to connect with a consumer that will soon be ready to buy.

Search: The Consummation of a Consumer Relationship

So, for brand marketers, the question is not, “does search actively engage the consumer in my brand messaging” but rather, “am I prepared not to have my brand present when my target consumer is looking to buy (or at least, research to buy)?” To me, it’s as elemental as not stocking the store shelf with your product. The consumer is not looking at building a relationship with a brand, he’s looking to consummate that relationship. Wouldn’t you want to be around for that rather important event?

So, to go back to ARF’s working definition of engagement, I don’t think it works for search. That definition of engagement is about building a relationship with the brand for “some day,” implanting a brand message for the time when the prospect turns into a shopper. When the shopper turns to search, that brand message is already planted. But if the brand isn’t present on the search results page “store shelf,” the message will be forgotten as the consumer clicks on the link of the next alternative.

I applaud ARF’s effort to define one all-encompassing metric, but when you have real people interacting with products and messaging in two very different ways, I’m not sure engagement, at least the way it’s currently defined, will be able to bridge the gap and do the job.