Matt Cutts: Personalization and the Future of SEO

mattcutts555I had the chance to interview Matt Cutts this week about personalization and it’s impact on the SEO industry. Excerpts from the interview and some additional commentary are in my Just Behave column on Search Engine Land today. As promised, here is the full transcript of the interview:

Gord: We’ve been awhile setting this up, and actually, this came from a discussion we had some time ago about geo-targeting of results in Canada, and we’re going to get to that a bit later. With this recent move by Google to move towards more personalization of the search results page, there’s some negative feedback and, to me, it seems to be coming from the SEO community. What’s your take on that?

Matt: I think that it’s natural that some people would be worried about change, but some of the best SEO’s are the SEO’s that are able to adapt, that are able to look down the road 4 or 5 years and say, “What are the big trends going to be?” and adjust for those trends in advance, so that when a search engine does make a change which you think is inevitable or will eventually happen, they’ll be in a good position. Personalization is one of those things where if you look down the road a few years, having a search engine that is willing to give you better results because it can know a little bit more about what your interests are, that’s a clear win for users, and so it’s something that SEO’s can probably predict that they’ll need to prepare for. At the same time, any time there’s a change, I understand that people need some time to adjust to that and need some time to think, “How is this going to affect me? How is this going to affect the industry? And what can I do to benefit from it?”

Gord: It seems to me, having a background in SEO, that the single biggest thing with personalization is the lack of a “test bed”, the lack of something to refer to when you’re doing your reverse engineering. You can’t look at a page of search results any more and say “that’s going to be the same page of test results that everyone’s seeing“. Given that, , more and more, we’re going to be seeing less of universal search results, is this the nail in the coffin for shady SEO tactics?

Matt: I wouldn’t say that it’s necessarily the nail in the coffin, but it’s clearly a call to action, where there’s a fork in the road and people can think hard about whether they’re optimizing for users or whether they’re optimizing primarily for search engines. And the sort of people who have been doing “new” SEO, or whatever you want to call it, that’s social media optimization, link bait, things that are interesting to people and attract word of mouth and buzz, those sorts of sites naturally attract visitors, attract repeat visitors, attract back links, attract lots of discussion, those sorts of sites are going to benefit as the world goes forward. At the same time, if you do choose to go to the other fork, towards the black hat side of things, you know you’re going to be working harder and the return is going to be a little less. And so over time, I think, the balance of what to work on does shift toward working for the user, taking these white hat techniques and looking for the sites and changes you can implement that will be to the most benefit to your user.

Gord: It would seem to be that there’s one sector of the industry that’s going to be hit harder by this, and I think it was Greg Boser or Todd Friesen who said, “You don’t take a knife to a gun fight.” So when you’re looking at the competitive categories, like the affiliates, where you don’t have that same site equity, you don’t have that same presence on the web to work with, that’s where it’s going to get hit, right?

Matt: I think one area that will change a lot, for example, is local stuff. Already, you don’t do a search for football and get the same results in the U.K. as you do in the U.S. So there are already a lot of things that return different search results based on country, and expect that trend to continue. It is, however, also the case that in highly commercial or highly spammed areas, if you are able to return more relevant, more personalized results, it gets a little harder to optimize, because the obstacles are such that you’re trying to show up on a lot of different searches rather than just one set of search engine result pages, so it does tilt the balance a little bit, yes.

Gord: I had a question about localization of search results, and I think being from Canada we’re perhaps a little bit more aware of it. How aware are American SEO’s that this is the case, that if  they’re targeting markets outside the U.S., they may not be seeing the same results that you’re seeing in the U.S.

Matt: I think that many SEO’s are relatively aware, but I’ve certainly talked to a few people who didn’t realize that if you do a search from the U.K., or from Canada, or from India, or from almost any country, you can get different results, instead of just the standard American results. And it’s definitely something that’s a huge benefit. If you’re in the United Kingdom and you type the query newspapers, you don’t want to get, necessarily, the L.A. Times or a local paper in Seattle, the Post-Intelligencer. Something like that. So I think it’s definitely started down that trend, and, over time, personalization will help a lot of people realize that it’s not just a generic set of results, or a vanilla set of results. You have to be thinking about how you’re going to show up in all of these markets, and personalization and localization complement each other in that regard.

Gord: Now one difference between localization and personalization is that personalization has the option of a toggle, you can toggle it on and off. Localization doesn’t have that same toggle, so as a Canadian, sometimes I may not want my results localized. Where does that put the user?

Matt: It’s interesting, because you have to gauge…and you talked to Marissa a couple times already, and from that you probably got a feel for the difficulty in making those decisions about just how much functionality to expose, in terms of toggles and advanced user preferences and stuff like that. So what we try to do is tackle the most common case and make that very simple. And a lot of the times, the functionality is such that you don’t even necessarily want someone that’s coming in from the U.K. to be able to search as if they’re coming in from Africa because it just makes things a lot more complicated. So, over time, I’d say we’re probably open to lots of different ways of allowing people to search. For example, you can select different countries for the advertisements. There’s a GL parameter I believe, where you can actually say, “now, show the ads as if I were searching from Canada. Okay, now I’m going to switch to Mexico”. Stuff like that. And that’s been very helpful, because if you giving Google money to buy ads, you want to be able to check and see what those ads would look like, in different regions. For search we haven’t historically made that as easy. It’s something that we’d probably be open to, but again, it’s one of those things where probably SEO’s are a lot more interested,you’re your regular user isn’t quite as interested.

Gord: And that gets to the ongoing problem. SEO’s have one perspective, users have another and arguably, yes, localization is good for the user, but for an SEO that deals with a lot of Canadian companies where the U.S. is their primary market. They’re looking at hitting that U.S. market. I guess this restricts them to making it look like their sites actually reside in the U.S. to get around it. So again, we’re trying to poke holes in the functionality, rather than live with it.

Matt: Well, one thing that should be possible is to indicate some sort of preference, or some sort of origin of location where you can indicate where you are. Historically Google has been ahead of the other search engines at the time by not just using the top level domain, so .ca, but also the I.P. address. So you can have .com hosted in Canada and that’s worked very well for many, many years. But we do continue to get feedback that people would like more flexibility, more options, so it’s a matter of deciding how many people that would help and just allocating resources on those types of things.

Gord: So we talked about personalization, we talked about localization. Are there other factors that are coloring the search results we should be aware of as we’re trying to consider all these aspects?

Matt: Once you’ve sort of “broken the mould” with different results for different countries, after that it’s good for people to move beyond the idea of a monolithic set of search results. If we had the ability to say someone is searching for Palo Alto or someone is searching for Kirkland or Redmond and give them local newspapers, truly local newspapers, that would be a good win for users as well.  So over time, I would expect search results to serve a broader and broader array of services.  The idea of a monolithic set of search results for a generic term will probably start to fade away, and you already see people expect that if I do a search and somebody else does the search, they can get slightly different answers. I expect that over time people will expect that more and more, and they’ll have that in the back of their heads.

Gord: Let’s take that “over time” and drill down a little more.  One of the things it was interesting for me when I was talking to Marissa with the fact that the Kaltix acquisition was made four years ago and it’s really taken four years for that technology to really show up in the search results.  Obviously a cautious approach to it.  And even with that we’re talking a couple of results being lifted into the top 10 and we’re talking one in five searches.  Also Marissa wasn’t exactly sure about this so I’ll clarify this with you.  She believed that it would never replace the number one organic result.

Matt: I believe that’s correct.  I’d have to double check to make sure.

Gord: So that’s a fairly tentative step in the direction of personalization, and you said over time we can expect this to continue to ship to be more of an individual experience.  Are we talking months, are we talking years, are we talking tomorrow?

Matt (chuckling): It’s usually not our policy to comment on exactly when stuff might roll out in the future, but personalization is an important trend and the ability to make search results better through personalization is really exciting to us here at Google.  I think if you look backwards over time, a lot of the reason why we might not have been able to personalize before was because Google was very much a “you come to the front page, you do a search, you get the results and you’re gone” type of model.  And there really weren’t that many opportunities to have a single sign on or some sort of Google account, where we could actually learn or know a little bit more about you to make your results more relevant.  So I think part of it involves getting all of the different ways of having an account together, so you can have personalized news, which rolled out a while ago, you could have a personalized homepage and those things give people a reason to sign in to Google.  Once you’re signed in to Google that helps us a lot more, by having your search history and the ability to offer personalization.  So at least looking backwards, I think some of the amount of time was just getting people ready to have a Google account and not just show up in Google, do a search and leave.

Gord: So part of it is that transition from a tool you use to more of a community you are engaged in.

Matt: Yes

Gord: That’s moving closer to your competition, notably Yahoo and Microsoft.  Google’s done very well as a tool.  Is this just the inevitable progression?

Matt: I think one nice thing is that Google adapts very well to what users want, and also the industry marketplace.  And so when our primary competition was a pure search engine, whether it be AltaVista or AlltheWeb or HotBot or Inktomi, then pure search mattered very much.  Search is still a part of everything we do.  It’s at the core of all the information that we organize and yet competing against sites like Yahoo and Microsoft involves a different set of strategies than competing against just a search engine for example.  So I think competition is very good for users, because it makes all of us work hard and it keeps us on our toes.  The one strength that Google has is that we do adapt and we look at the marketplace and we say, “What do we need to deliver next for our users to help them out and to encourage them to be more loyal to Google?”

Gord: So for your job, where you’re looking at the quality of the index and policing it, how does personalization change your job?

Matt: To some degree, it makes it easier, because it’s not one monolithic set of search results anymore.  But let me flip that around and say how we can make it easier for SEO’s as well.  I’m a computer graphics person, so if you go back to a concept called digital half toning, it’s this process where you have nothing but black and white, yet you are able to approximate different shades of gray. And if you look at the existing set of search results, a lot of people before had a very black or white mentality.  I’m ranking, or maybe I’m ranking number one or are not in the search results at all.  And that’s a very harsh step function, in terms of you not ranking where you think you should be, and maybe you’re not getting very much traffic at all.  If you are ranking number one, or very highly, you’re a very happy person.  And yet that monolithic set of search results may not serve users the best.  So now as we see that spread and soften, more people can show up at number one but for a smaller volume of queries.  And so individual users are happier because they’re getting more relevant search results and yet it’s not a winner take all mentality for SEOs anymore.  You can be the number one ranking set of results for your niche, whether it be a certain demographic or a certain locality, or something like that.  And I think that’s healthier overall, rather than having just a few people that are doing very well, you end up with a lot more SEO, and a lot more users who are happy and that’s softens the effect quite a bit.

Gord: What you’re talking about is a pretty fundamental shift in thinking on the part of a lot of SEOs…

Matt: yes

Gord: … a lot of SEOs are almost more engineers right now, where they’re looking at the algorithm and trying to figure out how to best it.  You’re asking them to become a lot of things, more marketing, PR, content developers, and know more about the user, more about user behavior online.  These are very different skill sets and often don’t reside in the same body.  What is this going to do to the SEO industry?

Matt: I think the SEO’s that adapt well to change an optimized for users are going to be in relatively good shape, because they’re trying to produce sites that are really pleasing and helpful to users.  It’s definitely the case that if all you care about is an algorithm than the situation grows more complicated for you with personalization.  But it’s also an opportunity for people to take a fresh look at how they do SEO.  So give you a quick example: we always say, don’t just chase after a trophy phrase.  There are so many people who think if I ranked number one for my trophy phase I win and my life will be good.  When, in fact, numerous people demonstrated that if you chase after the long tail and make a good site that can match many many different user’s queries you might end up with more traffic than if you had that trophy phrase.  So already the smart SEO, looking down the road, knows that it’s not just the head of the tail, it’s the long part of the tail and with personalization and the changes in how SEO will work, it will just push people further along the spectrum, towards looking at “it’s not just looking at a number one result for one query, how do we make it across a lot of queries.  What value do I deliver?  Am I looking at my server logs to find queries that I should be targeting?  And not just search engines, how do I target different parts of the search engine?  Like the local part of Google, the maps part of Google.  How do I target Google notebook and the other properties and how do I show up well across the entire portfolio of search properties?”  And that’s a healthy transition period that will push people towards delivering better value for their users and that’s better for everybody.

Gord: I get that and I’m an SEO.  My challenge comes in getting my clients, who in a lot of cases did their own SEO or worked with another SEO firm before they came to us and are used to that trophy phrase ranking.  How do we get them to get?  Because I see that being a challenge with a lot of SEOs. They will understand that, but getting the client to understand it could be a different matter

Matt: Sometimes I think you might have to do a demonstration like sign them into personalized search, do a query, sign them out, do query and show them, these are very different sets of results.  And sometimes the demonstration can be very visceral, you know, it can drive home the point that it’s not just going to be this one trophy phrase. People are going to have to think and look at the entire horizon of the space.

Gord: In Google there’s a very definite church versus state divide and traditionally the relationship with the advertiser was almost exclusively on one side of that divide.  But this could mark a fairly fundamental shift, and it will impact your advertisers, so as part of that community, will Google be doing anything to help those advertisers understand the organic part of their visibility on Google?  Will you be doing the same demonstration you just telling us we should be doing?

Matt: I think Google is always trying to communicate with the outside community, both with webmasters and advertisers.  So it’s really exciting to see some of the different techniques that we’ve used, everything from webinars to training materials to making videos available.  I would definitely say that every part of Google is going to keep their eyes open on how to best communicate how to stay on top of changes, because nobody wants anybody outside of Google to be unprepared for personalization or improvements in any of our technologies.

Katie (Katie Watson, Google PR representative who was sitting in on the interview) Something to actually cite there is that I know we recently just opened up our webmaster blog to outside comments, so that’s a good example of gradually moving forward to communicate even better.

Matt: You were couching the question in terms of advertisers, but if you look at the general story of webmaster communication and assume that that’s the leading edge, it’s pretty safe to assume that those smart ideas are percolating throughout the company and we’re trying to figure out all the different ways to communicate more.

Gord: So that’s the canary in the coal mine. Whatever’s happening in the webmaster community will act as a testbed for communication?

Matt: Exactly.

Gord: There is a debate raging right now about “is SEO rocket science”?  (Matt begins laughing) So what does personalization means for that debate?  Does it become more complicated?  You said it becomes easier in some ways and I countered that by saying that may be, but is also spreading out in a lot of different directions. Is there still a place for the pure SEO consultant out there?

Matt: I think there still is a place for you for a pure SEO consultant but it’s also true that over time those consultants have to keep adding to their skill set.  A few years ago no one would have even thought about the word Ajax and now people have to think about Ajax or Flash and how do I handle some of these new interfaces to still make sites crawlable?  So I definitely think there will still be places for consulting and improving crawlability of sites and advice on keywords and personalization will add some wrinkles to that, but I have faith that, over time we’ll see the benefit to users and if you make good site for your users, you will naturally benefit as a result.  Some people spend a lot of time looking at data centers and data center IP addresses and if people want to have that as a hobby they’re welcome to it but a lot of people don’t do that anymore and they’re just worried about making good results and yet, everything still comes out pretty well for them.

Gord: Some time ago I wrote a column along that line and said that, in many ways, the white hat SEO has helped clean up the Black hat side of the street because they enabled those good site to be crawled, to show up in the index and to assume their rightful place in the results.  It would seem to mean that personalization is just going to drive that process faster.

Matt: I think it will.  It’s making Black Hat tougher to do.  I think it’s interesting, it was designed primarily to improve the relevance for users but as a side effect, it definitely changes the game a lot more if you’re on the Black hat side of things then if you’re on the white hat side of things

Gord: Matt, I think that wraps things up for me..

Matt: Thanks, that was fun.

Webpronews Video: Who Said What?

I happened to be browsing through Webpronews on the weekend and saw one of their new video news updates. The clips are well produced, professional looking and even have their own attractive newscaster, Nicole Eggers. One I happened to pick, however, left a little to be desired on the accuracy front. As you’re probably aware, I just did a series of interviews with the top usability people at each of the three engines for Search Engine Land and a couple weeks ago I did a recap talking about the differences I saw between each of their philosophical approaches. The blurb on the video appeared to be on the same topic so I decided to give it a watch. If it, Webpronews indicated that search expert Danny Sullivan had talked to each of the three usability people at the engines and had come to the following conclusions:

  • That relevancy was almost a religion for Google
  • Yahoo had a heightened sensitivity to the needs of their advertising community
  • Microsoft was still finding their competitive niche

Huh? That’s exactly what my recap said. They even pulled a few quotes from it and attributed them to Danny. I quickly e-mailed Danny to see if we were doing some kind of weird Cyrano de Bergerac thing but Danny was apparently as out of the loop on this as I was. Anyway a quick e-mail to Webpronews seems to have got it straightened out. They’ve pulled the clip and apparently they’re redoing it.

Not that I mind being mistaken for Danny, but I just hate to be putting words in his mouth. By the way, does anyone else feel like they’re being scolded by Nicole? Again, not that I mind.

Marissa Mayer Interview on Personalization

marissa-mayer-7882_cnet100_620x433Below is the full transcript of the interview with Marissa Mayer on personalization of search results. For commentary, see the Just Behave column on Searchengineland.

Gord: It’s a little more than two weeks ago since Google made the announcement that personalization would become more of a default standard for more users on Google.  Why did you move towards making that call?

Marissa: We’ve had a very impressive suite of personalized products for awhile now: personalized homepage, search history, the personalized webpage and we haven’t had them integrated, which I think has made it somewhat confusing for users. A lot of people didn’t know if they had signed up for search history or personalized search; whether or not it was on.  What we really wanted to do was move to a signed in version of Google and a signed out version of Google.  So if you’re signed in you have access to the personalized home page, the personalized search results and search history.  You know all three of those are working for you when you’re signed in.  And if you’re signed out, meaning that you don’t see an email in the upper right hand corner that personalized search isn’t turned on.  If anything, it’s a cleaning up of the user model, to make it clearer to users what services they’re using them and when they’re using them.

Gord: But some of the criticism actually runs counter to that.  One of the criticisms is that it used to be clearer, as far as the user went, when you were signed in and when you are signed out.  There were more indicators on the Google results page whether you were getting personalized results or not.  Some of those have seemed to disappear, so personalized results have become more of a default now, rather than an option that’s available to the user.

Marissa: If you think about it as default-on when you’re signed in, I think that it’s still as clear on the search results page.  We removed the “turn off the personalized search results” link, but you still see very clearly up in the upper right-hand corner whether or not you’re signed in, your e-mail address appears, and that’s your clue Google has personalized you and that’s why that e-mail address is there.  I do think, based on our user studies and our own usage at Google, that we’ve made the model clearer.  We were actually ended up at the stage with our personalized product earlier this year where, at one point, Eric (Schmidt) asked “am I using personalized search?”  And the team’s answer as to whether or not he was currently using it was so complicated that even he couldn’t follow it.  You’d have to go to “my account”, see whether or not he was signed up for personalized search, make sure that your toggle hadn’t been turned off or on, and there was no way to just glance at the search results page and easily tell whether or not it was invoked.  So now it’s very easy, if you see your username and e-mail address up in the upper left-hand corner, you’re getting personalized results and if you don’t, you’re not.  So effectively there are two parallel universes of Google, per se.  One if you’re signed out where you see the classic homepage and the classic search results and one where you’re signed in, where you get the personalized home page and…you’ll be able to toggle back and forth, of course…and then the personalized search results page and the search history becomes coupled with all that because that’s how we personalize your search.

Gord: So, to sum up, it’s fair to say that really the search experience hasn’t changed that dramatically, it’s just cleaning up the user experience about whether you’re signed in or signed out and that’s been the primary change.

Marissa: That’s right.  Before you could be signed in and be using one of the three products or two of the three products but not all and, of course, because people like to experiment with a new product, they forget whether they signed up for personalized search.  Had they signed up for search history?  This just makes it cleaner.  If you’re signed in you’re using and/or have access to all three, if you’re signed out, you’re on the anonymous version of Google that doesn’t have personalization.

Gord: We can say that it cleans up the user experience because it makes it easier to you know when you’re signed in or signed out, but having done the eye tracking studies, we know that where the e-mail address shows is in a location that’s not prominently scanned as part of the page.  Do the changes mean that more people are going to be looking at personalized search results, just because we’ve made that more of a default opt in and we’ve moved the signals that you’re signed in a little bit out of the scanned area of the page.  Once people fixate on their task they are looking further down the page.  This should mean at a lot more people are looking at personalized search results than previously.

Marissa: Actually, I don’t think it will change the volume of personalized search all that much, not based on what we’ve seen on our logs and usage.  It makes it cleaner to understand whether or not you’re using it and I do think that over time, what it does is it pushes the envelope of search more such that you expect personalized results by default.  And we think that the search engines in the future will become better for a lot of different reasons, but one of the reasons will be that we understand the user better.  And so when we think about how we can advance towards that search engine of the future that we’re building, part of that will be personalization.  I do think that when we look five years out, 10 years out, users will have an expectation of better results.  One of the reasons that they have that expectation is that search engines will have become more personalized.  I think that in the future, working with the search engine that understands something about you will become the expectation.  But you’re right in that we believe that for users that are signed in, who find value in the personalized search results, over time as those users know they are signed in and that there search history is being kept track of, that their search results are being personalized, and they don’t need to look at every single search task to see whether or not they are signed in because that’s what their expectation is and they’re expecting personalized results.  So I do think we won’t see a drastic increase of volume right now of the use of personalized search but that it will hopefully change the user’s disposition over time to become more comfortable that personalization is a benefit for them and it’s something they come to expect.

Gord: There are a number of aspects of that question that I’d like to get into, and leave behind the question of whether you’re signed in or signed out of personalized search, but I have one question before we move on.  We’ve been talking a lot about existing users. The other change was where people were creating a new Google account and they got personalized search and search history by default.  The opt-out box is tucked into an area where most users would go right past it.  The placement of that opt-out box seems to indicate that Google would much rather have people opting into personalized search.

Marissa: I think that falls in with the philosophy that I just outlined. We believe that the search engines of the future will be personalized and that it will offer users better results.  And the way for us to get that benefit to our users is to try and have as many users signed up for personalized search as possible.  And so certainly we’re offering it to all of our users, and we’re going to be reasonably aggressive about getting them to try it out. Of course, we try to make sure they’re well-educated about how to turn it off if that’s what they prefer to do.

Gord: When this announcement came out I saw it as a pretty significant announcement for Google because it lays the foundation for the future.  I would think from Google’s perspective the challenge would be knowing what personalized search could be 5 to 10 years down the road,  what it would mean for the user experience and how do you start adding that incrementally to the user experience in the meantime?  From Google’s side, you have invested in algorithmic work to categorize content online. I would think the challenge would be just as significant to introduce the technology required to disambiguate intent and get to know more about users. You’re not going to hit that out of the park on the first pitch. That’s going to be a continuing trial and error process.  How do you maintain a fairly consistent user experience as you start to introduce personalization without negatively impacting that user experience?

Marissa: I will say that there are a lot of challenges there and a lot of this is something that’s going to be a pragmatic evolution for us.  You have to know that this is not a new development for us. We’ve been working on personalized search now for almost 4 years. It goes back to the Kaltix acquisition. So we’ve been working on it for awhile and our standards are really high.  We only want to offer personalized search if it offers a huge amount of end user benefit.  So we’re very comfortable and confident in the relevance seen from those technologies in order to offer them at all, let alone have them veered more towards the results, as we’re doing today.  We acquired a very talented team in March of 2003 from Kaltix.  It was a group of three students from Stanford doing their Ph.D, headed up by a guy named Sep Kamvar, who is the fellow who cosigned the post with me to the blog. Sep and his team did a lot of PageRank style work at Stanford.  Interestingly enough, one of the papers they produced was on how to compute PageRank faster.  They wrote this paper about how to compute page rank faster and it caused a huge media roil around the web because everyone said there are these students at Stanford who created an even faster version of Google.  Because the press obviously doesn’t understand search engines and thinks that we actually do the PageRank calculation on the fly on each query, as opposed to pre-computing it.  Their advance was actually significant not because it helps you prepare an index faster, which is what the press thought was significant.  Interestingly enough, the reason they were interested in building a faster version of PageRank was because what they wanted to do was be able to build a PageRank for each user.  So, based on seed data on which pages were important to you, and what pages you seemed to visit often, re-computing PageRank values based on that. PageRank as an algorithm is very sensitive to the seed pages.  And so, what they were doing, was that they had figured out a way to sort by host and as a result of sorting by host, be able to compute PageRank in a much more computationally efficient way to make it feasible to compute a PageRank per user, or as a vector of values that are different from the base PageRank.  The reason we were really interested in them was: one, because they really grasped and cogged all of Google’s technology really easily; and, two, because we really felt they were on the cutting edge of how personalization would be done on the web, and they were capable of looking at things like a searcher’s history and their past clicks, their past searches, the websites that matter to them, and ultimately building a vector of PageRank that can be used to enhance the search results.

We acquired them in 2003 and we’ve worked for some time since to outfit our production system to be capable of doing that computation and holding a vector for each user in parallel to the base computation.  We’ve been very responsible in the way that we’ve personalized Search Labs and we also did what we called Site Flavored Search on Labs where you can put a search box on your page and that is geared towards a page of interests that you’ve selected. So if you have a site about baseball you can say you want to base it on these three of your favorite baseball sites and have a search box that has a PageRank that’s veered in that direction for baseball queries.

So, the Kaltix team has been really successful at integrating all these Google technologies and taking this piece of theoretical research and ultimately bringing it to life on the Web.  And as it’s growing stronger and stronger and our confidence around the Kaltix technology grew, we’ve been putting it forward more and more.  We started off on Labs through a sign-up process, then we transitioned it over to and now we are in effect leaning towards a model where for people who use and have a Google account, they get personalized search basically by default.  If you look at the historical reviews of the Kaltix work it’s gotten pretty rave reviews.  The users that have noticed it and have been using it for a long time, like Danny (Sullivan), they’ll say that they think it’s one of the biggest advances to relevance that they’ve seen in the past three years.

Gord: So when you the Kaltix technology working over and above the base algorithm, obviously that’s going to be as good as the signals you’re picking up on the individual.  And right now the signals are past sites they visited, perhaps what they put on their personalized homepage and sites that they’ve bookmarked. But obviously the data that you can include to help create that on-the-fly, individual index improves as you get more signals to watch.  In our previous interview you said one thing that was really interesting to you was looking at the context of the task you are engaged in, for example, if you’re composing an e-mail in Gmail. So is contextual relevance another factor to look at.  Are those things that could potentially be rolled into this in the future?

Marissa: I think so.  I think that overall, we really feel that personalized search is something that holds a lot of promise, and we’re not exactly sure of the signals that will yield the best results.  We know that search history, your clicks and your searches together provide a really rich set of signals but it’s possible that some of the other data that Google gathers could also be useful. It’s a matter of understanding how.  There’s an interesting trade off around personalized search for the user which is, as you point out, the more signals that you have and the more data you have about the user, the better it gets.  It’s a hard sell sometimes, we’re asking them to sign up for a service where we begin to collect data in the form of search history yet they don’t see the benefits of that, at least in its fullest form, for some time.  It’s one of those things that we think about and struggle with. And that’s one reason why we’re trying to enter a model where search history and personalized search are, in fact, more expected.  And I should also note that as we look at reading some of the signals across different services we will obviously abide by the posted privacy policies.  So there are certain services where we’ve made it very clear we won’t cross correlate data. For example on Gmail, we’ve made it very clear that we won’t cross correlate that data with searches without being very, very explicit with the end user.  You don’t have to worry about things like that.

Gord: One of the points of concern seems to be how smart will that algorithm get and do we lose control?  For example, when we’re exploring new territory online and we’re trying to find answers we’ve refine our results based on our search experience.  So, at the beginning, we use very generic terms that cast a very wide net and then we narrow our search queries as we go. Somebody said to me, “Well, if we become better searchers, does that decrease the need for personalization?”  Do we lose some control in that?  Do we lose the ability to say “No, I want to see everything, and I will decide how I narrow or filter that query.  I don’t want Google filtering that query on the front end”?

Marissa: I think it really depends on how forcefully we’re putting forth personalization.  And right now we might be very forceful in getting people to sign up to it, or at least more forceful than we were. The actual implementation of personalized search is that as many as two pages of content, that are personalized to you, could be lifted onto the first page and I believe they never displace the first result, in our current substantiation, because that’s a level of relevance that we feel comfortable with.  So right now, at least eight of the results on your first page will be generic, vanilla Google results for that query and only up to two of them will be results from the personalized algorithm.  We’re introducing it in a fairly limited form for exactly the reason that you point out.  And I think if we tend to veer towards a model where there are more results that are personalized, we would have ways of making it clearer: “Do you want to explore this topic as a novice or with the personalization in place?” So the user will be able to toggle in a different filter form.  I think the other thing to remember is, even when personalization happens and lifts those two results onto the page, for most users it happens one out of every five times.  When you think about it, 20% of the queries are much better by doing that, but for 80% of the queries, people are, in fact, exploring topics that are unknown to them and we can tell from their search history that they haven’t searched for anything in this sphere before. There’s no other search like it. They’ve never clicked on any results that are related to this topic, and, as a result, we actually don’t change their query set at all because we know that they need the basic Google results.  The search history is valuable not only because it can help personalize the results but they’re also valuable because we can tell when not to.

Gord: There’s two parts to that: one is the intelligence of the algorithm to know when to push personalization and when not to push personalization, and two, as you said, right now this is only impacting one out of five searches where you may have a couple of new results being introduced into the top 10 as a result of personalization.  But that’s got to be a moving target.  As you become more confident in the technology and that it’s adding to the user experience, personalization will creep higher and higher up the fold and increasingly take over more of the search results page, right?

Marissa: Possibly.  I think that’s one of many things that could possibly happen, and I think that’s a pretty aggressive stance.  I look at our evolution and our foray into personalization, where we’re sitting here three or four years in, with some base technology that several years old already and it still has been very slight in a way that we have it interact with the user experience.  Mostly because we think that base Google is pretty good.  As it becomes more aggressive, certainly I would be pushing for an understanding of the ability of the user to know that these results are, in fact, coming from my personalization and not background and if I want to filter them out and get back to basics, that that would be possible.  One thing that we’ve struggled with is if we should actually mark the results are entering the page as a result of personalization but because team is currently and frequently doing experiments, we didn’t want to settle on a particular model or marker at this exact moment.

Gord: The challenge there is as you roll more personal results into the results page and get feedback from some users that they would want more control over what on the page is personalized and the degree of personalization and introduce more filters or more sophisticated toggles, it complicates the user experience. And as we know, that user experience needs to be very simple. Is it a delicate balance of how much control you give the user versus how much do you impact the 95% of the searches that are just a few seconds in duration and have to be really simple to do?

Marissa: There are two thoughts there.  One, even if we introduce them to filtering on the results page, it wouldn’t be any more complicated than what you had two weeks ago, so we already have that filter.  Two, we put the user first, and people have varying opinions about whether their search results page is too complicated, but the same people who designed that user experience will be the people who will be tackling this for Google, so I think you can expect results of a similar style and direction.

Gord: In the last few weeks, Google has introduced some new functionality, related searches and refine search suggestions, that are appearing at the bottom of the page for a number of searches.  To me that would seem to be a prime area that could be impacted by personalization opportunities that are coming.  As you make suggestions about other queries that you could be using, using that personalization data to refine those. Is that something you’re considering? And how long before personalization starts impacting the ads that are being presented to you on a search results page?

Marissa: Refinement is an interesting but a neophyte technology from our perspective.  We are finally now just beginning to develop some refining technologies that we believe in enough to use on the search results page.  A lot of people have been doing it for a lot longer. When you look at the overall utility, probably 1 to 5% of people will click those query refinements on any given search, where most users, probably more than two thirds of users, end up using one of our results. So in terms of utility and value that is delivered to the end user, the search results themselves and personalizing those are an order of magnitude more impactful then personalizing a query refinement.  So part of it is a question of, it’s such a new technology that we really haven’t looked at how we can make personalization make it work more effectively.  But the other thing is on a “bang for the buck” basis, personalizing those search results get us a lot more.

And as to ads, I think there are some easy ways to personalize ads that we’ve known for some time, but we’ve chosen at this point to focus on personalizing the search results because we wanted to make sure to delivered the end-user value on that, because that’s our focus, before we look at personalizing ads

Gord: So, no immediate plans for the personalization of ads?

Marissa: That’s right

Gord: Thank you so much for your time Marissa.

Webpronews Interview Now Posted

Had a chance to chat with Mike Macdonald from Webpronews at the recent Search Engine Strategies conference in Chicago. The video has just been posted on the site.

We chatted about SEMPO Institute and our latest eye tracking study.

Interview with Shuman Ghosemajumder about Click Fraud

Had a chance to chat with Shuman Ghosemajumder regarding click fraud. Shuman is Google’s point person on the click fraud issue. This follows up on the post Andy Beal made on MarketingPilgrim earlier this week. Most of what we chatted about was in my Search Insider column this week. However, not all of it made it into the column, as there is a cut off which I routinely ignore (thanks to MediaPost editor Phyllis Fine for keeping me in line).

Here’s some tidbits that didn’t make it into the column:

First of all, I wanted to take the media to task for crying the sky is falling around this issue. I know that’s what journalists do, but the portrayal of the click fraud issue has been very one sided to this point. That’s why I wrote the column. I think it’s important we get balancing viewpoints. In the absence of numbers universally regarded as accurate, one has to poll the extremes and guess that the true answer lies somewhere in the middle. Up to this point, all we’ve heard are the negative estimates, and these are based on some studies with methodolgy that’s questionable at best (i.e. the Outsell study)

Secondly, I believe it’s unfair that everyone seems to be taking aim at Google, and to a lesser extent, Yahoo on this issue. I know they’re easy targets, because the targets are so damned big, but when the real numbers finally do come out, I’d bet my 89 Mazda 626 (the car that just won’t die!) that it’s the 2nd and 3rd tier networks that are the hotbeds of click fraud.

I dealt with it briefly in the column, but one of the main sources of misrepresentation seems to be this question of what click fraud is. For me, the definition is pretty simple, fraudulent clicks that leave the advertiser financially impacted. But when it comes to most of the media portrayals, there are a number of clicks that get lumped together under the label “click fraud”, the majority of which don’t meet this definition. And Google’s point of contention with reports of click fraud that come from the media and various 3rd party fraud detection tools comes from this aggregation of questionable numbers. There’s no distinction made between actual fraud, the clicks that cost the advertiser, and attempted fraud, the ones that got caught. And often more benign clicks, i.e. multiple legitimate clicks coming from the same IP address, get mistakenly labelled as click fraud.

Another positive move by Google was the inclusion of invalid clicks in the advertiser’s reporting dashboard. Every move that Google makes towards greater transparency is a very positive one. And the best know Google evangelist for communication, Matt Cutts, indicated so on a blog post. By the way, Shuman also has a blog, where he goes into greater depth on this issue.

I can only imagine how frustrating this must be for Shuman and the Google Click Fraud team. They sit and listen to numbers be bandied about in the 15% plus range, knowing from first hand experience that the real number is likely much much lower (in the column, using assumptions that are probably on the high side, the actual amount of click fraud that an advertiser would have to challenge Google on is less than 0.18%). Yet, their tongues are tied, both by Google’s legal and corporate communications department.

Why is the media targeting click fraud and trying to scare the hell out of advertisers? In no other industry I can think of are reporters more prone to mix and match numbers without regard for accuracy. They do it, and get away with it, because there are no independent and reliable numbers to look at.

There are a number of reasons. Google is in the vanguard of disruptive change agents that are shaking the very ground of marketing. It’s somewhat defensive to look for an Achilles heel, and right now, click fraud seems to fit the bill. Google in particular is boldly stating they want to change everything. That scares people.

Part of it is that there is still a lot of people that would love to see Google be knocked down a few pegs. Much as we rever success, wildly successful companies or individuals generate jealousy and suspicion. Our society gets a nasty little thrill when the mighty fall.

But perhaps the biggest reason is the very strength of search and online marketing: it’s accountability. Nothing else is as measurable. So when something appears to be eating away at the cost effectiveness, we tend to go all forensic on it and analyze the hell out of it. Could you imagine the mainsteam press making a big deal out of a .18% hole in the accountability in television advertising, or radio, or print? Even a 10 to 15% hole? Of course not, because much bigger holes than that are accepted every day as being inherent in the channel. But search and online ad networks are apparently fair game.

Is click fraud happening? Absolutely. And if you switch the lens a bit, there are some sophisticated click fraud operations that are making a killing. In a response to my column, Chris Nielsen had this excellent observation:

The problem is not overt clicking on ads, competitors clicking on ads, or double-clicking on ads. The problem is with large-scale concerted efforts that are massive enough to to have enough variety of IP address, user agents, etc. and pose as “valid” user click activity.

Of course this activity varies some with the bid price of the clicks, but it’s really the old idea of stealing a penny from a million people. If anyone notices, who’s really going to care? The problem is that in some areas, there are hundreds or thousands of people stealing pennys, and it is noticible and it is a problem. The only real indication is the lack of bona fide conversions, and that’s hard to say for sure if it’s fraud or real factors with the marketing or web site.

But it comes down to which lens you look through. Do you look at those looking to profit from click fraud, some of them doing it very well? Or do you look at the scope of the problem over the big picture? The problem I have with the BusinessWeek report is that the reporting is trying to do both at the same time, and you can’t get a clear picture by doing so.

I just wanted to wrap up this post by mentioning some other initiatives on this front that Google is pursuing which didn’t make it into the original column. Obviously they’re working on proprietary techniques to filter out click fraud, but they’re also trying to attack the problem on an industry wide basis as well. They’re working with the IAB Click Measurement working group, in which SEMPO is also involved. And they’re calling for stringent and scientific independent auditing standards, so when we throw around terms like click fraud, we’re all dealing with a common reference framework. By the way, I also asked Shuman about impression fraud. We didn’t go into a lot of depth on the issue, but they feel they’re equally on top of that as well.

A Conversation with Ask’s CEO, Jim Lanzone

First published May 18, 2006 in Mediapost’s Search Insider

Last week, I had the chance to spend some time talking to Ask’s new CEO, Jim Lanzone. The first thing that become very clear is that Lanzone is tired of his company’s being compared to Google, MSN and Yahoo. I immediately slipped into the trap of asking how Ask intends to fight the big G and the two other contenders. It was obviously a question that he has heard all too often in the past. “Let’s begin by resetting the framework for the question,” Lanzone replied. “We don’t want to climb Everest right now. We’re not planning on knocking out Google. Our goal is to take our 20 million users, who are currently using us twice a month, and bump that up to four times a month. That doubles our market share,” he said.

“Search is not a zero sum gain,” Lanzone continued. “Americans use about 3.2 engines a month. We want to get on the list. We want to offer an alternative to Google.” As he pointed out, it’s much easier to post impressive market share growth percentages when you’re starting with a relatively small slice of the pie. “For Google to move the needle even a few points, they have to attract huge numbers of new searches,” he said. “We can achieve huge growth just by getting people to use us a few more times each month.”

Focusing on the User

Ask has come out of the gate strongly since the rebrand and the removal of Jeeves. The focus has been squarely on improving the search experience for its existing user. “We want to put the right tool in the right place at the right time,” said Lanzone. “We want to be waiting for the user when they need us.”

Distancing Ask from the helpful butler has proven itself to be difficult. “People are much more aware of us as Ask Jeeves than Ask. It will take time,” he said. For right or wrong, the efforts of Ask Jeeves to brand themselves as a natural query engine have stuck. A long-time member of Ask’s usability team, Michael Ferguson, pointed out the challenges of trying to change how people search. “For years, we’ve been telling people to ask us a question. We’ve perhaps been a little too successful in encouraging them. I once asked a lady what was the last thing she searched for on (then) Ask Jeeves. She was disappointed in the results she got when she typed in ‘Midnight basketball programs are more successful in LA than in St. Louis. Why is that?'”

I had written some time ago that I believe we’ve become used to truncating our search intent into a few words. Apparently, Ask now agrees. Rather than trying to accommodate natural language, Ask now takes a more standard approach to query construction.

Tools When You Need Them, Where You Need Them

Even the physical layout of Ask’s new home page tried to ease the transition from the once ubiquitous butler. Ferguson said, “We put the new tool palette where Jeeves used to be. It’s about the same height and size, so visually it has the same balance.”

The palette is a big part of the new usability focus of Ask. Lanzone explained, “We have some great tools, and we wanted to move them more upfront for the user. We didn’t want to hide them with tabs, which no one clicks on. With the palette, it’s right there, waiting for them.”

Based on initial numbers, people are using the tools more than ever before. Usage on most of them has doubled, with some, like image and map search, posting far higher gains.

Another change was a rethinking of how to use the real estate of the search results page. “The ads from the right rail are gone,” said Lanzone. “We’ve replaced that with something searchers can use, the ‘Narrow’ and ‘Broaden’ your search suggestions. Rather than 1 or 2 percent click-throughs on ads, we’re getting 30 percent click throughs on those suggestions. We know that search is an iterative process, so why not help make it easier by helping the user get to the right search faster?”

Another cleaned up area of the SERP is the top-sponsored ads. It used to be that organic results were pushed right off the page by far too many sponsored listings. Top-sponsored listings have since been restricted to three, in line with other major engines, with the rest shown at the bottom of the page. This used to be one of my pet peeves with Jeeves, and apparently I had plenty of sympathizers in the Ask usability team. “We were a public company, responding to demand for profits,” Lanzone explains. “A lot of us never agreed with that.” Kudos to parent company IAC for eventually listening to the champions of the user experience.

Not Bolder, Just Better

I wrapped up by telling Lanzone that I believed Ask was in a good position to become the bold innovator in search. Unlike Google and Yahoo, it isn’t solely dependent on a huge revenue stream from search, so it can afford to take some risks in testing new interfaces and developments. He replied, “You know, we’ve never considered what we’ve done, or plan to do, as being bold. It just has to be right for the user. All the changes we’ve made were done because in our testing, it felt right. We’re not chasing technology for its own sake. We have a laser focus on the user experience. We just want to do search right.”

I’m a huge believer in focused strategy and feet on the ground, practical user-centricity. They’re two commodities that are in drastically short supply in the current heated search space. If Ask sticks to its guns, it just may just get a shot at the big dogs in search.