I had the chance to interview Matt Cutts this week about personalization and it’s impact on the SEO industry. Excerpts from the interview and some additional commentary are in my Just Behave column on Search Engine Land today. As promised, here is the full transcript of the interview:

Gord: We’ve been awhile setting this up, and actually, this came from a discussion we had some time ago about geo-targeting of results in Canada, and we’re going to get to that a bit later. With this recent move by Google to move towards more personalization of the search results page, there’s some negative feedback and, to me, it seems to be coming from the SEO community. What’s your take on that?

Matt: I think that it’s natural that some people would be worried about change, but some of the best SEO’s are the SEO’s that are able to adapt, that are able to look down the road 4 or 5 years and say, “What are the big trends going to be?” and adjust for those trends in advance, so that when a search engine does make a change which you think is inevitable or will eventually happen, they’ll be in a good position. Personalization is one of those things where if you look down the road a few years, having a search engine that is willing to give you better results because it can know a little bit more about what your interests are, that’s a clear win for users, and so it’s something that SEO’s can probably predict that they’ll need to prepare for. At the same time, any time there’s a change, I understand that people need some time to adjust to that and need some time to think, “How is this going to affect me? How is this going to affect the industry? And what can I do to benefit from it?”

Gord: It seems to me, having a background in SEO, that the single biggest thing with personalization is the lack of a “test bed”, the lack of something to refer to when you’re doing your reverse engineering. You can’t look at a page of search results any more and say “that’s going to be the same page of test results that everyone’s seeing“. Given that, , more and more, we’re going to be seeing less of universal search results, is this the nail in the coffin for shady SEO tactics?

Matt: I wouldn’t say that it’s necessarily the nail in the coffin, but it’s clearly a call to action, where there’s a fork in the road and people can think hard about whether they’re optimizing for users or whether they’re optimizing primarily for search engines. And the sort of people who have been doing “new” SEO, or whatever you want to call it, that’s social media optimization, link bait, things that are interesting to people and attract word of mouth and buzz, those sorts of sites naturally attract visitors, attract repeat visitors, attract back links, attract lots of discussion, those sorts of sites are going to benefit as the world goes forward. At the same time, if you do choose to go to the other fork, towards the black hat side of things, you know you’re going to be working harder and the return is going to be a little less. And so over time, I think, the balance of what to work on does shift toward working for the user, taking these white hat techniques and looking for the sites and changes you can implement that will be to the most benefit to your user.

Gord: It would seem to be that there’s one sector of the industry that’s going to be hit harder by this, and I think it was Greg Boser or Todd Friesen who said, “You don’t take a knife to a gun fight.” So when you’re looking at the competitive categories, like the affiliates, where you don’t have that same site equity, you don’t have that same presence on the web to work with, that’s where it’s going to get hit, right?

Matt: I think one area that will change a lot, for example, is local stuff. Already, you don’t do a search for football and get the same results in the U.K. as you do in the U.S. So there are already a lot of things that return different search results based on country, and expect that trend to continue. It is, however, also the case that in highly commercial or highly spammed areas, if you are able to return more relevant, more personalized results, it gets a little harder to optimize, because the obstacles are such that you’re trying to show up on a lot of different searches rather than just one set of search engine result pages, so it does tilt the balance a little bit, yes.

Gord: I had a question about localization of search results, and I think being from Canada we’re perhaps a little bit more aware of it. How aware are American SEO’s that this is the case, that if  they’re targeting markets outside the U.S., they may not be seeing the same results that you’re seeing in the U.S.

Matt: I think that many SEO’s are relatively aware, but I’ve certainly talked to a few people who didn’t realize that if you do a search from the U.K., or from Canada, or from India, or from almost any country, you can get different results, instead of just the standard American results. And it’s definitely something that’s a huge benefit. If you’re in the United Kingdom and you type the query newspapers, you don’t want to get, necessarily, the L.A. Times or a local paper in Seattle, the Post-Intelligencer. Something like that. So I think it’s definitely started down that trend, and, over time, personalization will help a lot of people realize that it’s not just a generic set of results, or a vanilla set of results. You have to be thinking about how you’re going to show up in all of these markets, and personalization and localization complement each other in that regard.

Gord: Now one difference between localization and personalization is that personalization has the option of a toggle, you can toggle it on and off. Localization doesn’t have that same toggle, so as a Canadian, sometimes I may not want my results localized. Where does that put the user?

Matt: It’s interesting, because you have to gauge…and you talked to Marissa a couple times already, and from that you probably got a feel for the difficulty in making those decisions about just how much functionality to expose, in terms of toggles and advanced user preferences and stuff like that. So what we try to do is tackle the most common case and make that very simple. And a lot of the times, the functionality is such that you don’t even necessarily want someone that’s coming in from the U.K. to be able to search as if they’re coming in from Africa because it just makes things a lot more complicated. So, over time, I’d say we’re probably open to lots of different ways of allowing people to search. For example, you can select different countries for the advertisements. There’s a GL parameter I believe, where you can actually say, “now, show the ads as if I were searching from Canada. Okay, now I’m going to switch to Mexico”. Stuff like that. And that’s been very helpful, because if you giving Google money to buy ads, you want to be able to check and see what those ads would look like, in different regions. For search we haven’t historically made that as easy. It’s something that we’d probably be open to, but again, it’s one of those things where probably SEO’s are a lot more interested,you’re your regular user isn’t quite as interested.

Gord: And that gets to the ongoing problem. SEO’s have one perspective, users have another and arguably, yes, localization is good for the user, but for an SEO that deals with a lot of Canadian companies where the U.S. is their primary market. They’re looking at hitting that U.S. market. I guess this restricts them to making it look like their sites actually reside in the U.S. to get around it. So again, we’re trying to poke holes in the functionality, rather than live with it.

Matt: Well, one thing that should be possible is to indicate some sort of preference, or some sort of origin of location where you can indicate where you are. Historically Google has been ahead of the other search engines at the time by not just using the top level domain, so .ca, but also the I.P. address. So you can have .com hosted in Canada and that’s worked very well for many, many years. But we do continue to get feedback that people would like more flexibility, more options, so it’s a matter of deciding how many people that would help and just allocating resources on those types of things.

Gord: So we talked about personalization, we talked about localization. Are there other factors that are coloring the search results we should be aware of as we’re trying to consider all these aspects?

Matt: Once you’ve sort of “broken the mould” with different results for different countries, after that it’s good for people to move beyond the idea of a monolithic set of search results. If we had the ability to say someone is searching for Palo Alto or someone is searching for Kirkland or Redmond and give them local newspapers, truly local newspapers, that would be a good win for users as well.  So over time, I would expect search results to serve a broader and broader array of services.  The idea of a monolithic set of search results for a generic term will probably start to fade away, and you already see people expect that if I do a search and somebody else does the search, they can get slightly different answers. I expect that over time people will expect that more and more, and they’ll have that in the back of their heads.

Gord: Let’s take that “over time” and drill down a little more.  One of the things it was interesting for me when I was talking to Marissa with the fact that the Kaltix acquisition was made four years ago and it’s really taken four years for that technology to really show up in the search results.  Obviously a cautious approach to it.  And even with that we’re talking a couple of results being lifted into the top 10 and we’re talking one in five searches.  Also Marissa wasn’t exactly sure about this so I’ll clarify this with you.  She believed that it would never replace the number one organic result.

Matt: I believe that’s correct.  I’d have to double check to make sure.

Gord: So that’s a fairly tentative step in the direction of personalization, and you said over time we can expect this to continue to ship to be more of an individual experience.  Are we talking months, are we talking years, are we talking tomorrow?

Matt (chuckling): It’s usually not our policy to comment on exactly when stuff might roll out in the future, but personalization is an important trend and the ability to make search results better through personalization is really exciting to us here at Google.  I think if you look backwards over time, a lot of the reason why we might not have been able to personalize before was because Google was very much a “you come to the front page, you do a search, you get the results and you’re gone” type of model.  And there really weren’t that many opportunities to have a single sign on or some sort of Google account, where we could actually learn or know a little bit more about you to make your results more relevant.  So I think part of it involves getting all of the different ways of having an account together, so you can have personalized news, which rolled out a while ago, you could have a personalized homepage and those things give people a reason to sign in to Google.  Once you’re signed in to Google that helps us a lot more, by having your search history and the ability to offer personalization.  So at least looking backwards, I think some of the amount of time was just getting people ready to have a Google account and not just show up in Google, do a search and leave.

Gord: So part of it is that transition from a tool you use to more of a community you are engaged in.

Matt: Yes

Gord: That’s moving closer to your competition, notably Yahoo and Microsoft.  Google’s done very well as a tool.  Is this just the inevitable progression?

Matt: I think one nice thing is that Google adapts very well to what users want, and also the industry marketplace.  And so when our primary competition was a pure search engine, whether it be AltaVista or AlltheWeb or HotBot or Inktomi, then pure search mattered very much.  Search is still a part of everything we do.  It’s at the core of all the information that we organize and yet competing against sites like Yahoo and Microsoft involves a different set of strategies than competing against just a search engine for example.  So I think competition is very good for users, because it makes all of us work hard and it keeps us on our toes.  The one strength that Google has is that we do adapt and we look at the marketplace and we say, “What do we need to deliver next for our users to help them out and to encourage them to be more loyal to Google?”

Gord: So for your job, where you’re looking at the quality of the index and policing it, how does personalization change your job?

Matt: To some degree, it makes it easier, because it’s not one monolithic set of search results anymore.  But let me flip that around and say how we can make it easier for SEO’s as well.  I’m a computer graphics person, so if you go back to a concept called digital half toning, it’s this process where you have nothing but black and white, yet you are able to approximate different shades of gray. And if you look at the existing set of search results, a lot of people before had a very black or white mentality.  I’m ranking, or maybe I’m ranking number one or are not in the search results at all.  And that’s a very harsh step function, in terms of you not ranking where you think you should be, and maybe you’re not getting very much traffic at all.  If you are ranking number one, or very highly, you’re a very happy person.  And yet that monolithic set of search results may not serve users the best.  So now as we see that spread and soften, more people can show up at number one but for a smaller volume of queries.  And so individual users are happier because they’re getting more relevant search results and yet it’s not a winner take all mentality for SEOs anymore.  You can be the number one ranking set of results for your niche, whether it be a certain demographic or a certain locality, or something like that.  And I think that’s healthier overall, rather than having just a few people that are doing very well, you end up with a lot more SEO, and a lot more users who are happy and that’s softens the effect quite a bit.

Gord: What you’re talking about is a pretty fundamental shift in thinking on the part of a lot of SEOs…

Matt: yes

Gord: … a lot of SEOs are almost more engineers right now, where they’re looking at the algorithm and trying to figure out how to best it.  You’re asking them to become a lot of things, more marketing, PR, content developers, and know more about the user, more about user behavior online.  These are very different skill sets and often don’t reside in the same body.  What is this going to do to the SEO industry?

Matt: I think the SEO’s that adapt well to change an optimized for users are going to be in relatively good shape, because they’re trying to produce sites that are really pleasing and helpful to users.  It’s definitely the case that if all you care about is an algorithm than the situation grows more complicated for you with personalization.  But it’s also an opportunity for people to take a fresh look at how they do SEO.  So give you a quick example: we always say, don’t just chase after a trophy phrase.  There are so many people who think if I ranked number one for my trophy phase I win and my life will be good.  When, in fact, numerous people demonstrated that if you chase after the long tail and make a good site that can match many many different user’s queries you might end up with more traffic than if you had that trophy phrase.  So already the smart SEO, looking down the road, knows that it’s not just the head of the tail, it’s the long part of the tail and with personalization and the changes in how SEO will work, it will just push people further along the spectrum, towards looking at “it’s not just looking at a number one result for one query, how do we make it across a lot of queries.  What value do I deliver?  Am I looking at my server logs to find queries that I should be targeting?  And not just search engines, how do I target different parts of the search engine?  Like the local part of Google, the maps part of Google.  How do I target Google notebook and the other properties and how do I show up well across the entire portfolio of search properties?”  And that’s a healthy transition period that will push people towards delivering better value for their users and that’s better for everybody.

Gord: I get that and I’m an SEO.  My challenge comes in getting my clients, who in a lot of cases did their own SEO or worked with another SEO firm before they came to us and are used to that trophy phrase ranking.  How do we get them to get?  Because I see that being a challenge with a lot of SEOs. They will understand that, but getting the client to understand it could be a different matter

Matt: Sometimes I think you might have to do a demonstration like sign them into personalized search, do a query, sign them out, do query and show them, these are very different sets of results.  And sometimes the demonstration can be very visceral, you know, it can drive home the point that it’s not just going to be this one trophy phrase. People are going to have to think and look at the entire horizon of the space.

Gord: In Google there’s a very definite church versus state divide and traditionally the relationship with the advertiser was almost exclusively on one side of that divide.  But this could mark a fairly fundamental shift, and it will impact your advertisers, so as part of that community, will Google be doing anything to help those advertisers understand the organic part of their visibility on Google?  Will you be doing the same demonstration you just telling us we should be doing?

Matt: I think Google is always trying to communicate with the outside community, both with webmasters and advertisers.  So it’s really exciting to see some of the different techniques that we’ve used, everything from webinars to training materials to making videos available.  I would definitely say that every part of Google is going to keep their eyes open on how to best communicate how to stay on top of changes, because nobody wants anybody outside of Google to be unprepared for personalization or improvements in any of our technologies.

Katie (Katie Watson, Google PR representative who was sitting in on the interview) Something to actually cite there is that I know we recently just opened up our webmaster blog to outside comments, so that’s a good example of gradually moving forward to communicate even better.

Matt: You were couching the question in terms of advertisers, but if you look at the general story of webmaster communication and assume that that’s the leading edge, it’s pretty safe to assume that those smart ideas are percolating throughout the company and we’re trying to figure out all the different ways to communicate more.

Gord: So that’s the canary in the coal mine. Whatever’s happening in the webmaster community will act as a testbed for communication?

Matt: Exactly.

Gord: There is a debate raging right now about “is SEO rocket science”?  (Matt begins laughing) So what does personalization means for that debate?  Does it become more complicated?  You said it becomes easier in some ways and I countered that by saying that may be, but is also spreading out in a lot of different directions. Is there still a place for the pure SEO consultant out there?

Matt: I think there still is a place for you for a pure SEO consultant but it’s also true that over time those consultants have to keep adding to their skill set.  A few years ago no one would have even thought about the word Ajax and now people have to think about Ajax or Flash and how do I handle some of these new interfaces to still make sites crawlable?  So I definitely think there will still be places for consulting and improving crawlability of sites and advice on keywords and personalization will add some wrinkles to that, but I have faith that, over time we’ll see the benefit to users and if you make good site for your users, you will naturally benefit as a result.  Some people spend a lot of time looking at data centers and data center IP addresses and if people want to have that as a hobby they’re welcome to it but a lot of people don’t do that anymore and they’re just worried about making good results and yet, everything still comes out pretty well for them.

Gord: Some time ago I wrote a column along that line and said that, in many ways, the white hat SEO has helped clean up the Black hat side of the street because they enabled those good site to be crawled, to show up in the index and to assume their rightful place in the results.  It would seem to mean that personalization is just going to drive that process faster.

Matt: I think it will.  It’s making Black Hat tougher to do.  I think it’s interesting, it was designed primarily to improve the relevance for users but as a side effect, it definitely changes the game a lot more if you’re on the Black hat side of things then if you’re on the white hat side of things

Gord: Matt, I think that wraps things up for me..

Matt: Thanks, that was fun.