You know, if I didn’t know better I’d say that Shari Thurow had issues with me and eye tracking. I ran across a column a couple of weeks ago where she was talking about the niches that SEO’s are carving out for themselves and she mentioned eye tracking specifically. In fact she devoted a whole section to eye tracking. Now, it’s pretty hard not to take it personally when Enquiro is the only search marketing company I know that does extensive eye tracking. We’re the only ones I’m aware of that have eye tracking equipment in-house. So when Shari singles out eye tracking and warns about using the results in isolation…

That brings me to my favorite group of SEO specialists: search usability professionals. As much as I read and admire their research, they, too, often don’t focus on the big picture.

…I’m not sure who else she might be talking about.

I’ve been meaning to post on this for awhile but I just didn’t get around to it. I’m on the road today and feeling a little cranky so what the heck. It’s time to respond in kind. First, here’s Shari’s take on on eye tracking and SEO.

Eye-tracking data is always fascinating to observe on a wide variety of Web pages, including SERPs (define). As a Web developer, I love eye-tracking data to let me know how well I’m drawing visitors’ attention to the appropriate calls to action for each page type.

Nonetheless, eye-tracking data can be deceiving. Most search marketers understand the SERP’s prime viewing area, which is in the shape of an “F.” Organic or natural search results are viewed far more often than search engine ads are, and (as expected) top, above-the-fold results are viewed more often than the lower, below-the-fold results. Viewing a top listing in a SERP isn’t the same as clicking that link and taking the Web site owner’s desired call to action.

Remember, usability testing isn’t the same as focus groups and eye tracking. Focus groups measure peoples’ opinions about a product or service. Eye-tracking data provide information about where people focus their visual attention. Usability testing is task-oriented. It measures whether participants complete a desired task. If the desired task isn’t completed, the tests often reveal the many roadblocks to task completion.

Eye-tracking tests used in conjunction with usability tests and Web analytics analysis can reveal a plethora of accurate information about search behavior. But eye-tracking tests used in isolation yield limited information, just as Web analytics and Web positioning data yield limited (and often erroneous) information.

Okay Shari, you didn’t mention me or Enquiro by name but again, who else would you be talking about?

Actually, Shari and I agree more than we disagree here. I agree that no single data source or research or testing approach provides all the answers, including eye tracking. However, eye tracking data adds an extraordinarily rich layer of data to common usability testing. When Shari says eye tracking is not the same as usability testing, she’s only half right. As Shari points out, eye tracking combines very well with usability testing but in many cases, can be overkill. Usability testing is task oriented. There’s no reason why eye tracking studies can’t be task oriented as well (most of ours are). The eye tracking equipment we use is very unobtrusive. It virtually like interacting with any computer in a usability lab. In usability testing you put someone in front of the computer with the task and asked them to complete the task. Typically you record the entire interaction with software such as TechSmith’s Morae. After you can replay the session and watch where the cursor goes. Eye tracking can capture all that, plus capture where the eyes went. It’s like taking a two dimensional test and suddenly making it three-dimensional. Everything you do in usability can also be done with eye tracking.

The fact is, the understanding we currently have of interaction with the search results would be impossible to know without eye tracking. I’d like to think that a lot of our current understanding of interaction with search results comes from the extensive eye tracking testing we’ve done on the search results page. The facts that Shari says are common knowledge among search marketers comes, in large part, from our work with eye tracking. And we’re not the only ones. Cornell and Microsoft have done their own eye tracking studies, as has Jakob Nielsen, and findings have been remarkably similar. I’ve actually talked to the groups responsible for these other eye tracking tests and we’ve all learned from each other.

When Enquiro produced our studies we took a deep dive into the data that we collected. I think we did an excellent job at not presenting just the top level findings but really tried to create an understanding of what the interaction with the search results page looks like. Over the course of the last two years I’ve talked to Google, Microsoft and Yahoo. I’ve shared the findings of our research and learned a little bit more about the findings of their own internal research. I think, on the whole, we know a lot more about how people interact with search than we did two years ago, thanks in large part to eye tracking technology. The big picture Shari keeps alluding to has broadened and been colored much more extensively thanks to those studies. And Enquiro has tried to share that information as much as possible. I don’t know of anyone else in the search marketing world who’s done more to help marketers understand how people interact with search. When we released our first study, Shari wrote a previous column that basically said, “Duh, who didn’t know this before?” Well, based on my discussions with hundreds, actually, thousands of people, almost everyone, save for a few usability people at each of the main engines.

There are some dangers with eye tracking. Perhaps the biggest danger is that heat maps are so compelling visually. People tend not to go any further. The Golden Triangle image has been displayed hundreds, if not thousands of times, since we first released it. It’s one aggregate snapshot of search activity. And perhaps this is what Shari’s referring to. If so, I agree with her completely. This one snapshot can be deceiving. You need to do a really deep dive into the data to understand all the variations that can take place. But it’s not the methodology of eye tracking that’s at fault here. It’s people’s unwillingness to roll up their sleeves and weed through the amount of data that comes with eye tracking, preferring instead to stop at those colorful heat maps and not go any further. Conclusions on limited data can be dangerous, no matter the methodology behind them. I actually said the same for an eye tracking study Microsoft did that had a few people drawing overly simplified conclusions. The same is true for usability testing, focus groups, quantitative analysis, you name it. I really don’t believe Enquiro is guilty of doing this. That’s why we released reports that are a couple hundred pages in length, trying to do justice to the data we collected.

Look, eye tracking is a tool, a very powerful one. And I don’t think there’s any other tool I’ve run across that can provide more insight into search experience, when it’s used with a well designed study. Personally, if you want to learn more about how people interact with engines, I don’t think there’s any better place to start than our reports. And it’s not just me saying so. I’ve heard as much from hundreds of people who have bought them, including representatives at every major search engine (they all have corporate licenses, as well as a few companies you might have heard of, IBM, HP, Xerox..to name a few). I know the results pages you see at each of the major engines look the way they do in part because of our studies.

Shari says we don’t focus on the big picture. Shari, you should know that you can’t see the big picture until you fill in the individual pieces of the puzzle. That’s what we’ve been trying to do. I only wish more people out there followed our example.