Search Engine Results: 2010 – Marissa Mayer Interview

marissa-mayer-7882_cnet100_620x433Just getting back in the groove after SES San Jose. You may have caught some of my sessions or heard we have released a white paper looking at the future of search and with some eye tracking on personalized and universal search results. We don’t have the final version up yet, but it should be available later this week. The sneak preview got rave reviews in SJ.

Anyway, I interviewed a number of influencers in the space, and I’ll be posting the full transcripts here on my blog over the next week. I already posted Jakob Nielsen’s interview. Today I’ll be posting Marissa Mayer’s, who did a keynote at SES SJ. It makes for interesting reading. Also, I’ll be running excerpts and additional commentary on Just Behave on Search Engine Land. The first half ran a couple weeks ago. Look for more (and a more regular blog schedule) coming out over the next few weeks. Summer’s over and it’s back to work.

Here’s my chat with Marissa:

Gord: I guess I have one big question that will probably break out into a few smaller questions.  What I wanted to do for Search Engine Land is speculate on what the search engine results page might look like to the user in three years time.  With some of the emerging things like personalization and universal search results and some the things that are happening with the other engines: Ask with their 3D Search, which is their flavor of Universal, it seems to me that we might be at a point for the first time in a long time the results that we’re seeing may have a significant amount of flux over the next 3 years.  I wanted to talk to a few people in the industry about their thoughts of what we might be seeing 3 years down the road.  So that’s the big over-arching question I’m posing.

Marissa: Sure, Minority Report on search result pages…Well, I’d like to say it’s going to be like that but I think that’s a little further out.  There are some really fascinating technologies that I don’t know if you’ve seen..some work being done by a guy named Jeff Han?

Gord: No.

Marissa: So I ran into Jeff Han both of the past years at TED. Basically he was doing multi-touch before they did it on the iPhone on a giant wall sized screen, so it actually does look a lot like Minority Report. It was this big space where you could interact, you could annotate, you could do all those things.  But let me talk first about what I see happening as some trends that are going to drive change.

One is that we are seeing more and more broadband usage and I think in three years everyone will be on very fast connections, so a lot more to choose from and  a lot more data without taking a large latency hit.  The other thing we’re seeing is different mediums, audio, video.  They used to not work.  If you remember getting back a year ago, everytime you clicked on an audio file or a movie file, it would be, like, ‘thunk’?  It needs a plug in, or “thunk”, it doesn’t work.  Now we’re coming into some standardized formats and players that are either browser or technology independent enough, or are integrated enough that they are actually going to work.  And also we’re seeing users having more and more storage on their end.  And those are the sort of 3 computer science trends that are things that are going to change things.  I also think that people are becoming more and more inclined to annotate and interact with the web. It started with bloggers, and then it moved to mash ups, and now people are really starting to take a lot more ownership over their participation on the web and they want to annotate things, they want to mark it up.

So I think when you add these things together it means there’s a couple of things.  One, we will be able to have much more rich interaction with the search results pages. There might be layers of search results pages: take my results and show them on a map, take my results and show them to me on a timeline.  It’s basically the ability to interact in a really fast way, and take the results you have and see them in a new light.  So I think that that kind of interaction will be possible pretty easily and pretty likely.  I think it will be, hopefully, a layout that’s a little bit less linear and text based, even than our search results today and ultimately uses what I call the ‘sea of whiteness’ more in the middle of the page, and lays out in a more information dense way all the information from videos to audio reels to text, and so on and so forth.  So if you imagine the results page, instead of being long and linear, and having ten results on the page that you can scroll through to having ten very heterogeneous results, where we show each of those results in a form that really suits their medium, and in a more condensed format.  A couple of years ago we did a very interesting experiment here on the UI team where we took three or 4 different designs where the problem was artificially constrained.  It was above the fold Google.  If you needed to say everything that Google needed to say above the fold, how would you lay it out?  And some came in with two columns, but I think two columns is really hard when it was linear and text based.  When you started seeing some diagrams, some video, some news, some charts, you might actually have a page that looks and feels more like an interactive encyclopedia.

Gord: So, we’re almost going from a more linear presentation of results, very text based, to almost more of a portal presentation, but a personalized portal presentation.

Marissa: Right and I think as people, one, are getting more bandwidth and two, as they’re more savvy with how they look at more information, think of it this way, as more of serial access versus random access.  One of my pet peeves is broadcast news, where I really don’t like televised news anymore.  I like newspapers, and I like reading online because when I’m online or with newspapers, I have random access.  I can jump to whatever I’m most interested in.  And when you’re sitting there watching broadcast news you have to take it in the order, at the pace and at the speed that they are feeding it to you.  And yes, they try to make it better by having the little tickers at the bottom, but you can’t just jump in to what you’re interested in.  You can only read one piece of text at a time, and it’s hard to survey and scan and hone in on one type of medium or another when it’s all one medium.  So certainly there is some random access happening with the search results today.  I think as the results formats becomes much more heterogeneous, we’re going to have a more condensed presentation that allows for better random access.  Above the fold being really full of content, some text, some audio, some video, maybe even playing in place, and you see what grabs your attention, and pulls you in.   But it’s almost like random access on the front page of the New York Times, where am I more drawn to the picture, or the chart, or this piece of content down here?  What am I drawn to?

Gord: Right.  If you’re looking at different types of stimuli across the page, I guess what you’re saying is, as long as all that content is relevant to the query you can scan it more efficiently than you could with the standardized text based scanning, linear scanning, that we’re seeing now

Marissa: That’s right.

Gord: Ok.

Marissa: So the eyes follow and they just read and scan in a linear order, where when you start interweaving charts and pictures and text, people’s eyes can jump around more, and they can gravitate towards the medium that they understand best.

Gord: So, this is where Ask is going right now with their 3D search, where it’s broken it into 3 columns and they’re mixing images and text and different things.  So I guess what we’re looking at is taking it to the next extreme, making it a richer, more interactive experience, right?

Marissa: Rather than having three rote columns, it would actually be more organic.

Gord: So more dynamic.  And it mixes and matches the format based on the types of material it’s bringing back.

Marissa: Well, to keep hounding on the analogy of the front page of the New York Times.  It’s not like the New York Times…I mean they have basically the same layout each time, but it’s not like they have a column that only has this kind of content, and if it doesn’t fill the column, too bad.  They have a basic format that they change as it suits the information.

Gord: So in that kind of format, how much control does the user have? How much functionality do you put in the hands of the user?

Marissa: I think that, back to my third point, I think that people will be annotating search results pages and web pages a lot.  They’re going to be rating them, they’re going to be reviewing them.  They’re going to be marking them up, saying  “I want to come back to this one later”.  So we have some remedial forms of this in terms of Notebook now, but I imagine that we’re going to make notes right on the pages later.  People are going to be able to say I want to add a note here; I want to scribble something there, and you’ll be able to do that.  So I think the presentation is going to be largely based on our perceived notion of relevance, which of course leverages the user, in the ways they interact with the page, and look at what they do and that helps inform us as to what we should do.  So there is some UI user interaction, but the majority of user interaction will be on keeping that information and making it consumable in the best possible way.

Gord: Ok, and then if, like you said, if you go one step further, and provide multiple layers, so you could say, ok, plot my search results, if it’s a local search, plot my search results on a map.  There’s different ways to, at the user’s request, present that information, and they can have different layers that they can superimpose them on.

Marissa: So what I’m sort of imagining is that in the first basic search, you’re presented with a really rich general overview page, that interweaves all these different mediums, and on that page you have a few basic controls, so you could say, look, what really matters to me is the time dimension, or what really matters to me is the location dimension.  So do you want to see it on a timeline, do you want to see it on a map?

Gord: Ok, so taking a step further than what you do with your news results, or your blog search results, so you can sort them a couple of different ways, but then taking that and increasing the functionality so it’s a richer experience.

Marissa: It’s a richer experience. What’s nice about timeline and date as we’re currently experimenting with them on Google Experimental is not only do they allow you to sort differently, they allow you to visualize your results differently.  So if you see your results on a map, you can see the loci, so you can see this location is important to this query, and this location is really important to that query.  And when you look at it in time line you can see, “wow, this is a really hot topic for that decade”.  They just help you visualize the nut of information across all the results in these fundamentally different ways that ‘sorts’ kind of get at. But it’s really allowing that richer presentation and that overview of results on the meta level that helps you see it.

Gord: Ok.  I had a chance to talk to Jakob Nielsen about this on Friday, and he doesn’t believe that we’re going to be able to see much of a difference in the search results in 3 years.  He just doesn’t think that that can be accomplished in that time period.  What you’re talking about is a pretty drastic change from what we’re seeing today, and the search results that we’re seeing today haven’t changed that much in the last 10 years, as far as what the user is seeing.  You’re really feeling that this is possible?

Marissa: It’s interesting, you know, I pay am lot of attention to how the results look.  And I do think that change happens slowly over time and that there are little spurts of acceleration.  We at Google certainly saw a little accelerated push during May when we launched Universal Search.  I’m of the view that maybe its 3 years out, maybe it’s 5 years out, maybe it’s 10 years out.  I’m a big subscriber to the slogan that people tend to overestimate the short term and underestimate the long term.  My analogy to this is that when I was 5, I remember watching the Jetson’s and being, like, this rocks!  When I’m thirty there are flying cars!  Right?  And here I am, I’m 32 and we don’t even have a good flying car prototype, and yet the world has totally changed in ways that nobody expected because of the internet and computing.  In ways that in the 1980s no one even saw it coming.  Because personal computers were barely out, let alone the internet.  It’s interesting.  We do our off site in August. I do an offsite with my team where we do Google two years out. There it’s really interesting to see how people think about it.  I take all the prime members on my team, so they’re the senior engineers, and everybody has homework.  They have to do a homepage and a results page of Google, and this year it’ll be Google 2009.

Gord: Oh Cool!

Marissa: Six months out, it’s really easy because if we’re working on it, because if it’s going to launch in 6 months and it’s big enough that you would notice, we’re working on it right now and we know it’s coming.  And five years or ten years out we start getting into the bigger picture things like what I’m talking to you about.  When the little precursors that get us ready for those advances happen between now and then that’s what’s shifting.   So I’m giving you the big picture so you can start understanding what some of the mini steps that might happen in the next 3 years, to get us ready for that, would be.  The two to three year timeframe is painful. Everybody at my offsite said, “this timeframe sucks!” So it’s just far enough out that we don’t have great visibility, will mobile devices be something that’s a really big new factor in three years?  Maybe, maybe not.  Some of the things are making fast progress now may even take a big leap, right, like it was from 1994 to 97 on the internet.  Or if you think about G-mail and Maps, like AJAX applications..you wouldn’t have foreseen those in 2002 or 2003.  So, two or three years is a really painful time frame because some things are radically different, but probably in different ways than you would expect.  You have very low visibility in our industry to that time frame.  So I actually find it easier to talk about the six month timeframe, or the ten year timeframe.  So I’m giving you the ten year picture knowing that it’s not like the unveiling of a statue, where you can just take the sheet, snatch it off and go, “Voila there it is”.  If you look at the changes we’ve made over time at Google search they’ve always been “getting this ready, getting this ready”.  So the changes are very slow and feel like they’re very incremental.  But then you look at them in summation over 18 months or two years, you’re like, “you know, nothing felt really big along the way, but they are fundamentally different today”.

Gord: One last question.  So we’re looking at this much richer search experience where it’s more dynamic and fluid and there are different types of content being presented on the page.  Does advertising or the marketing message get mixed into that overall bucket, and does this open the door to significantly different types of presentation of the advertising message on the search results page?

Marissa: I think that there will be different types of advertising on the search results page.  As you know, my theory is always that the ads should match the search results.  So if you have text results, you have text ads, and if you have image results, you have image ads.  So as the page becomes richer, the ads also need to become richer, just so that they look alive and match the page.  That said, trust is a fundamental premise of search.  Search is a learning activity.  You think of Google and Ask and these other search engines as teachers.  As an end user the only reason learning and teaching works, the only way it works, is when you trust your teacher.  You know you’re getting the best information because it’s the best information, not because they have an agenda to mislead you or to make more money or to push you somewhere because of their own agenda.  So while I do think the ads will look different, they will look different in format, or they may look different in placement, I think our commitment to calling out very strongly where we have a monetary incentive and we may be biased will remain.  Our one promise on our search results page, and I think that will stand, is that we clearly mark the ads.  It’s very important to us that the users know what the ads are because it’s the disclosure of that bias, that ultimately builds the trust which is paramount to search

Gord: Ok.  Great to see you’re a keynote at San Jose in August.

Marissa: Should be fun.  This whole topic has me kind of jazzed up so maybe I’ll talk about that.

Search Engine Results: 2010 – Interview with Danny Sullivan

Danny-SullivanHere’s another in the series of the Search:2010 transcripts, this one of my chat with Search Engine Land Editor Danny Sullivan:

Gord: The big question that I’m asking is how much change are we going to see on the search engine results page over the next three years.  What impact are things like universal search and personalization and some of the other things we’re seeing come out, how much of that is going to impact the actual interface the user is going to see.  Maybe let’s just start there.

Danny: I love the whole series to begin with because then I thought, Gosh, I never really sat down and tried to plot out how I would do it, and I wish I had had the time to do that before we talked (laughs).  But it would be nice to have a contest or something for the people who are in the space to say I think this is the way we should do it or where it should go.
But the thing at the top of my head that I expect or I assume that we’re going to get is… I think they’re going to get a lot more intelligent at giving you more from a particular database when they know you’re doing a specific a kind of search.  It’s not necessarily an interface change, but then again it is.  This is the thing I talked about when I was saying about when the London Car Bombing attempts happened, and I’m searching for “London Bombings”.  When you see a spike in certain words you ought to know that there’s a reason behind that spike.  It’s going to be news driven probably, so why are you giving me 10 search results? Why don’t you give me 10 news results?  And saying I’ve also got stuff from across the web, or I’ve got other things that are showing up in that regard.  And that hasn’t changed.  I‘d like to see them get that.   I’d like to see them figure out some intelligent manner to maybe get to that point.  Part of what could come along with that too is that as we start displaying more vertical results the search interface itself could change.  So I think the most dramatic change in how we present search results, really, has come off of local.  And people go “wow, these maps are really cool!” Well of course they’re really cool, they’re presenting information on a map which makes sense when we’re talking about local information.  You want things displayed in that kind of manner.  It doesn’t make sense to take all web search results and put them on a map. You could do it, but it doesn’t communicate additional information for you that’s probably irrelevant and that needs to be presented in a visual manner.  If you think about the other kinds of search that you tend to do, Blog search for instance, it may be that there’s going to be a more chronological display. We saw them do with news archive where they would do a search and they would tell you this happened within these years at this time.  Right now when I do a Google blog search, by default it shows me ‘most relevant’.  But sometimes I want to know what the most recent thing is, and what’s the most recent thing that’s also the most relevant thing right? So perhaps when I do a Search, a Google blog search, I can see something running down the left hand side that says “last hour” and within the last hour you show me the most relevant things in the last hour, the last 4 hours, and then the last day.  And you could present it that way, almost sort of a timeline metaphor. I’m sure there are probably things you could do with shading and other stuff to go along with that.  Image search…Live has done some interesting things now where they’ve made it much less textual, and much more stuff that you’re hovering over, that you can interact with it in that regard.  An I don’t know, it might be that with book search and those other kinds of things that there’ll be other kinds of metaphors that come into place that you can do when you know you are going to present most of the information just from those sorts of resources.  With Video search… I think we’ve already seen a lot of the thing with video search is just giving you the display and being able to play the videos directly.  Rather than having to leave the site because it just doesn’t make sense to have to leave the site in that regard.

Gord: When I was talking to Marissa, she saw a lot more mash ups with search functionality, and you talked about having maps and that with local search making sense, but its almost like you take the search functionality and you layer that over different types of interfaces that make sense, given the type of information your interacting with.

Danny: Right.

Gord: One thing I talked about with a few different people is ‘how much functionality do you put in the hands of the user?’ how much needs to be transparent? How hard are we willing to work with a page of search results?

Danny: By default, not a lot, you know if you’re just doing a general search, I don’t think that putting a whole lot of functionality is going to help you. You could put a lot of options there but historically we haven’t seen people use those things, and I think that’s because they just want to do their searches. They want you to just naturally get the right kind of information that’s there and a lot of the time if they give you that direct answer you don’t need to do a lot of manipulation.  It’s a different thing I think when you get into some very vertical, very task orientated kinds of searches, where you’re saying, ‘I don’t just need the quick answer, I don’t just need to browse and see all the things that are out there, but actually, I’m trying to drill down on this subject in a particular way’.  And local tends to be a great example. ‘Now you’ve given me all the results that match the zip code, but really I would like to narrow it down into a neighborhood, so how can I do that?’  Or a shopping search.  ‘I have a lot of results but now I want to buy something, so now I need to know who has it in inventory? Now I really need to know who has it cheapest? And I need to know who’s the most trusted merchant?’ Then I think the searcher is going to be willing to do more work on the search and make use of more of the options that you give to them.

Gord: Like you say, if you’re putting users directly into an experience where they are closer to the information that they were looking for, there’s probably a greater likelihood that they’re willing to meet you half way, by doing a little extra work to refine that if you give them tools that are appropriate to the types of results they are seeing.  So if it’s shopping search, filtering that by price, or by brand.  That’s common functionality with a shopping search engine and maybe we’ll see that get in to some of the other verticals. But I guess the big question is, in the next three years are the major engines going to gain enough confidence that they’ll be providing a deeper vertical experience as the default, rather than as an invisible tab or a visible tab.

Danny: I still tend to think that the way that they are going to give a deeper vertical experience is the visible tab idea, which is you know, that you are not going to be overtly asked to do it, it is just going to do it for you, and then give you options to get out of it, if it was the wrong choice. So, both Ask, and Google, which are getting all the attention right now, for universal search, you know, blended search if you wanna find a generic term for it that, doesn’t favor one service over the other.  The other term is federated search and I’ve always hated that because it always felt like something from that, you know, came out of the Star Trek Enterprise (laugh). No, I want Klingon search! (laugh) I think that in both of those cases you do the search and the default still is web.  And Ask will say, over here on the side we have some other results. Yes, universal search is inserting an item here or an item there but in most of the cases it still looks like web search, right? They still, really feel like OneBoxes. I haven’t had a universal search happen to me yet that I’ve come along and I’ve thought ‘that really was something I couldn’t have got just from searching the web’ except when I’ve gotten a map.  That’s come in when they’ve shown the map, and that is that kind of dramatic change, and I think at some point they will get to that point, that kind of dramatic change where you just search for “plumbers” and a zip code.  I’m so confident of it I’m just going to give you Google local. I’m not just going to insert a map and give you 7 more web listings that are down there. I’m going to give you a whole bunch of listings and I’m going to change the whole interface on you and if you’re going ‘well, this isn’t what I want’, then I’m going to be able to give you some options if you want to escape out of it.  I like what Ask does, in the sense that it’s easy to escape out of that thing because you just look off to the side and there’s web search over here, there’s other stuff over there.  I think it’s harder for Google to do that when they try to blend it all together. The difficulty remains as to whether people will actually notice that stuff off to the side, and make use of it.

Gord: That was actually something that Jacob Nielsen brought up. He said the whole paradigm of the linear scan down the page is such a dominant user behavior, that we’ve got so used to, you know engines like Ask can experiment with a different layout where they’re going two dimensional, but will the users be able to scan that efficiently?

Danny: I’ve been using this Boeing versus Airbus analogy when I’m trying to explain to people the differences between what Google is doing and what Ask is doing.  Boeing is going, ‘Well, we’ll build small fast energy-efficient jets’ and Airbus is saying ‘We’ll build big huge jets, and we’ll move more people so you’ll be able to do less flights’.  And when I look at the blended search, Google’s approach is, well, we’ve got to stay linear, we’ve got to keep it all in there. That’s where people are expecting the stuff and so we’re going to go that way.  Ask’s approach is we’re going to be putting it all over the place on the page and we’ve got this split, really nice interface.  And I agree with them. And of course Walt Mossberg wrote that review where he said ‘oh they’re so much nicer, they look so much cleaner’, and that’s great, except that he’s a sophisticated person, I’m a sophisticated person, you’re a sophisticated person, we search all the time.  We look at that sort of stuff. A typical person might just ignore it; it might continue to be eye candy that they don’t even notice. And that is the big huge gamble that is going on between these two sorts of players and then, yet again, it might not be a gamble because when you talk to Jim Lanzone, he says ‘My testing tells me this is what our people do’. Well, his people might be different from the Google people. Google has got a lot more new people that come over there that are like, ‘I just want to do a search, show me some things, where’s the text links? I’m done’. So I tend to look perhaps more kindly on what Google is doing, than some people who try to measure them up against Ask because I understand that they deal with a lot more people than Ask, and they have to be much more conservative than what Ask is doing.  And I think that what’s going to happen is those two are going to approach closer together.  The advantage, of course, Jim has over at Ask, is that he doesn’t have to put ads in that column so he’s got a whole column he can make use of, and it is useful, and it is a nice sort of place to tuck it in there. If you really want to talk about search interfaces, what will be really fun to envision is what happens when Ajax starts coming along and doing other things. Can I start putting the sponsored search results where they are hovering above other results? Is there other issues that come with that?  There may be some confusion as to why I’m getting this and why I’m getting that. Can I pop up a map as I hover over a result? I could deliver you a standard set of search results and I can also deliver you local results on top of a particular type of picture.  If I move my mouse along it, I could show you a preview of what you get in local and you might go “Oh wow, there’s a whole map there”. I want to jump off in that direction.  That would be really fun to see that type of stuff come along there, but I’m just not seeing anything come out of it.  What we typically have had when people have played with the interface is, these really WYSIWYG things like, ‘well we’ll fly you though the results, or we’ll group them’.  None of which is really something that you’d need, that added to the choices, “do I want to go vertical, do I not want to go vertical?”

Gord: When we start talking about the fact that the search results page could be a lot more dynamic and interactive, of course the big question is what does that do for monetization of the page?  One of the things that Jakob (Nielsen) talked about was banner blindness.  Do people start cutting out sections of the page?  We talked a little about that.  How do you make sure that the advertising doesn’t get lost on the page when there’s just a lot more visual information in there to assimilate?

Danny: Well I think a variety of things that are going to start happening there.  For example, Google doesn’t do paid inclusion, right, but Google has partnerships with YouTube and they have these channels, and they’re going to be sharing revenue from these channels with other people. So when they start including that stuff up, perhaps they are getting paid off of that.  They didn’t pay to put it in the index but, because they are better able to promote their video channels, more people are going over there, and they’re making money off of that as a destination.  So in some ways, they can afford to have their video results start becoming more relevant because they don’t have to worry about if you didn’t click on the ad from the initial search result, they sort of lost you.  In terms of how the other ads might go, I guess the concern might be if the natural results are getting better and better why would anyone click on the ads anyway?  Maybe people will reassess the paid results and some people will come through and say that paid search results are a form of search data base as well.  So we’re going to call them classifieds or we’re going to call them ads, we’re going to move them right into the linear display.  You know there’ll be issues, because at least in the US, you have the FCC guidelines that say that you should really keep them segregated.  So if you don’t highlight them or blend them in some way, you might run into some regulatory problems.  But then again, maybe those rules might start to change as the search innovation starts to change, and go with it from there.  I don’t know, the search engines might come up with other things.  You know we’re getting toolbars that are appearing more on all of our things. Google might start thinking, ‘Well, let’s put ads back onto that toolbar’.  We used to have those sorts of things, and everyone seems to catch on, but they might come back, and that might be another way that some of the players, especially somebody like Google, might make money beyond just putting the ad on the search result page.

Gord: In the next three years, are we going to get to the point where search starts to become less of a destination activity like the way it is now, and the functionality  sits underneath more of Web 2.0 or semantic web or whatever you want to call it.  It almost becomes a mash up of functionality that underlies other types of sites. Are we going to stop going to a Google or a Yahoo as much to launch a distinct search as we do now?

Danny: You know people have been saying that for at least 3 or 4 years now, especially with Microsoft. ‘Oh you’re not even going to go there, you’re going to do it from your desktop.’  Vista, which I have yet to actually use.  I’ve got the laptop and I’m about to start playing with it! Apparently, it’s supposed to be even more integrated than it was with XP.  But I still tend to think, you know what? We do stuff in our browsers.  I know widgets are growing and I know there’s more stuff that’s just drawing stuff into your computer as well, but we still tend to do stuff in our browser.  I still see search as something where I’m going to go to a search engine and do the search.  With the exception of toolbars. I think we’re going to do a lot more searching through toolbars.  Tool bars are everywhere; it’s really rare for me to start a search where I’m actually not doing it from the toolbar.  I just have a toolbar that sits up there, and I don’t need to be at the search engine itself.  But I still want the results displayed in my browser.  Because I think most of the stuff I’m going to have to deal with is going to be in my browser as well.  So it doesn’t really help to be able to search from Microsoft Word, right?  Because I don’t want all these sites in a little window within Word. I’m probably going to have to read what they say, so I’m probably going to have to go there.  I think that will change though if I have a media player, then I think it makes much more sense for me, and you can already do this with some media players, where you can do searches, and have the results flow back in.  iTunes is a classic example. iTunes is basically a music search engine.  Sure, it’s limited to the music and the podcasts that are within iTunes, but it doesn’t really make any sense for me to go to the Apple website. Although, interestingly, here’s an example where Apple is just a terrible failure.  They’ve got all this stuff out there, they’ve got stuff that perhaps you might be interested in even if you don’t use their software and there’s just no way to get to it on the web.  The last time I looked you really had to do the searches in iTunes.  So they’re missing out on being a destination for those people who say ‘I’m not going to use iTunes’  or ‘I don’t have iTunes’ or ‘I’m on a different version.’ I don’t know if you’ve downloaded it recently but it takes forever and it’s just a pain.

Gord: I think that covers off the main questions I wanted to cover off in this.  Is there anything else as far as search in the next three years that you wanted to comment on?

Danny: You know, it’s hard because if you’d asked me that three years ago, would I have told you, ‘watch for the growth of verticals and watch for the growth of blended search’, (laughs) right?  I’ve been thinking really hard because, I’m like, ‘Gosh, now what am I going to talk about because they’re doing both of those things’. I think personalized search is going to continue to get strong.  I do think that Google is onto something with their personalized search results.  I don’t think that they’re going to cause you to be in an Amazon situation where you’re continuing to be recommended stuff you’re no longer interested in.  I think that people are misunderstanding how sophisticated it can be.  I think that the next big trend is that, ironically from what I just said to you, search is going to start jumping into devices.  And everything is going to have a search box.  But it will be appropriate.  My iPod itself will have a search capability within it.  And the iPhone, to some degree, maybe is going to be that look at how it’s happening already. But I’ll be able to search, access, and get information appropriate to that device within it.  Windows Media Center, when I first got that in 2005, I said, this is amazing, because it’s basically got TV search built into it.  I do the search and then of course, it allows me to subscribe to the program, and records the program, and knows when the next ones are coming up.  And it makes so much more sense for that search to be in that device than it did for me to have it elsewhere.  I use it all the time, when I want to know when a programs on, I don’t have to find where the TV listings are on the web, I just walk over to my computer and do a search from within the Media Center player.  So I think we’re going to have many more devices that are internet enabled, and there’s going to be reasons why you want to do searches with them, to find stuff for them in particular.  That’s going to be the new future of search and search growth will come into it.  And in terms of what that means to the search marketer, I think it’s going to be crucial to understand that these are going to be new growth areas, because those searches when they start are going to be fairly rudimentary. It’s going to be back in the days of, OK, they’re probably going to be driven off of meta data, so you got to make sure you have your title, and your description and making sure the item that your searching for is relevant.

Gord: So obviously all that leads itself to the question of mobile search, and will mobile search be more useful by 2010?

Danny: Sure, but it’s going to be more useful because it’s not going to be mobile search.  It’s just the device is going to catch up and be more desktop-like.  I have a Windows mobile phone at the moment, and I have downloaded some of the applets like Live Search and Google Maps, and those can be handy for me to use, but for the most part, if I want to do a search, I fire up the web browser, I look for what I’m looking for, the screen is fairly large, and I can see what I wanted to find.  And I think that you’re going to find that the devices are going to continue to be small and yet gain larger screens, and have the ability for you to better do direct input. So if you want to do search, you can do a search. It’s not like you’re going to need to have to have something that’s designed for the mobile device that only shows mobile pages.  I think that’s going to change.  You’re going to have some mobile devices that are specifically not going to be able to do that and those people in the end are going to find that no one is going to be trying to support you.

Gord: Thanks Danny.

Breaking “Auction Order” Explained

One of the things that raised eyebrows in my interview with Diane Tang and Nick Fox was the following section regarding how Google determines which ads rank first and climb into the all important top sponsored locations:

Nick: Yes, it’s based on two things.  One is the primary element is the quality of the ad. The highest quality ads get shown on the top. The lower quality ads get shown on the right hand side. We block off the top ads from the top of the auction, if you really believe those are truly excellent ads…

Diane: It’s worth pointing out that we never break auction order…

Nick: One of the things that’s sacred here is making sure that the advertiser’s have the incentive. In an auction, you want to make sure that the folks who win the auction are the ones who actually did win the auction. You can’t give the prize away to the person who didn’t win the auction. The primary element in that function is the quality of the ad. Another element of function is what the advertiser’s going to pay for that ad. Which, in some ways, is also a measure of quality. We’ve seen that in most cases, where the advertiser’s willing to pay more, it’s more of a commercial topic. The query itself is more commercial, therefore users are more likely to be interested in ads. So we typically see that queries that have high revenue ads, ads that are likely to generate a lot of revenue for Google are also the queries where the ads are also most relevant to the user, so the user is more likely to be happy as well. So it’s those two factors that go into it. But it is a very high threshold. I don’t’ want to get into specific numbers, but the fraction of queries that actually show these promoted ads is very small.

This seemed a little odd to me in the interview and I made a note to ask further about that, but what can I say, I forgot and went on to other things. But when the article got posted on Searchengineland, Danny jumped on it at Sphinn

“Seriously? I mean, it’s not an auction. If it were an auction, highest amount would win. They break it all the time by factoring in clickrate, quality score, etc. Not saying that’s bad, but it’s not an auction.”

This reminded me to follow up with Nick and Diane. Diana Adair, on the Google PR team, responded with this clarification:

We wanted to follow up with you regarding your question below.  We wanted to clarify that we rank ads based on both quality score and by bid.  Auction order, therefore, is based on the combination of both of those factors.  So that means that it’s entirely possible that an ad with a lower bid could rank higher than an ad with a higher bid if the quality score for the less expensive ad is high enough.

So, it seems it’s the use of the word “auction” that’s throwing everyone off here. Google’s use of the term includes ad quality. The rest of the world thinks of an auction as somewhere where the highest bid (exclusively) determines the winner. Otherwise, like Danny said, “it’s not an auction”. So, with that interpretation, I then assume that Nick and Diane’s (which sounds vaguely like a title of a John Mellencamp song) comment means that Google won’t arbitrarily hijack these positions for other types of packages which may include presence on the SERP, as in the current Bourne Ultimatum promotion.

Interview with Google’s Nick Fox and Diane Tang on Ad Quality Scoring

I had the chance to talk to Nick Fox and Diane Tang from Google’s Ad Quality team about quality scoring and how it impacts the user experience. Excerpts from the article along with additional commentary will be in Friday’s Just Behave column, but here is the full transcript.

Gord: What I wanted to talk about a little bit was just how the  quality, particularly in the top sponsored ads, impacts user experience and talk a little about relevancy. Just to set the stage, one of the things I talked about at SES in Toronto was just the fact that as far as a Canadian user goes, because the Canadian Ad market isn’t as mature as the American one, we’re not seeing the same acceptance of those sponsored ads at the top.  Just because you’re not seeing the brands that you would expect to see for a lot of the queries. You’re not seeing a lot of trusted vendors in that space. They just have not adopted search the same way they have in the States.  What we’ve seen in some of our user studies is a greater tendency to avoid that real estate … or at least to quickly scan it and then move down.  So, that’s the angle I really want to take here.  Just how important it is ad quality and ad relevance to impacting that user experience and then also talking about one thing I’ve always noticed in the number of our user studies. Of all the engines, Google seems to be the most stringent on what it takes to be a qualified ad. To get promoted from the right rails to the top sponsored ads. So that sets a broad framework of what I wanted to talk about today.

Nick: Let me give you a quick overview of who I am and who Diane is and what we work on and then we’ll jump into the topics that you’ve raised.  So what Diane and I work on is called Ad Quality and it is essentially everything about how we decide which ads to show on Google and our partners and what they should look like, how much we charge for them and all those types of things. How the auction works…everything from soup to nuts.  If you ask us what our goal is…our goals is to make sure our users love our ads. If you ask Larry Page what our goal is…it’s to make our ads as good as our search results. So it’s a heavy focus on making sure that our users are happy and that our users are getting back what they want out of our ads.  We sort of think of ourselves as among the first that work on the average product for Google. We represent the user, to make sure the user is getting what they really need.  We’re very similar to what we do on the search quality side, making sure that search results are very good.

I think a lot of the things you’ve picked up on are very accurate. In terms of the focus on top ad quality..in general, the focus on quality..I think what you picked up on in your various  reports as well as the study in Canada are pretty accurate and pretty much what drives what we are working on here.  The big concern that I would have, the main motivation for why I think ad quality is important is as a company we need to make sure users continue to trust our ads.  If users don’t trust our ads, they will stop looking at the ads, and once they stop looking at the ads they’ll stop clicking on the ads and all is lost. So what we need to make sure we are doing in long run  is that the users believe that the ads will provide them what they are looking for and they will continue looking at the ads as valuable real estate and to continue to trust that.
So that is what we are going for. I think as we look at the competitors landscape as well, we see a lot of what you see. We certainly have historically, and continue to do so, have much more of a focus on the quality of the ads. Making sure we’re not doing things where we trade off the user experience against revenue. We all have the ability to show more ads or worse ads, but we take a very stringent approach, as you’ve noticed, to making sure we only show the best ads that we believe the user will actually get something out of. If the user’s not going to get something out of the ad, we don’t show the ad. Otherwise the user is going to be less likely to consider ads in the future.

Diane: It’s worth pointing out that basically what we’re saying is that we are taking a very long term view towards making sure our users are happy with our ads and it’s really about making them trust what we give them.

Gord: One thing I’ve noticed in all my conversations whether they’re with Marissa or Matt or you, the first thing that everyone always says at Google is the focus around the user experience. The fact that the user needs to walk away satisfied with their experience. When we’re talking about the search results page, that focuses very specifically on what we’ve called in our reports the “area of greatest promise”. That upper left orientation on the search results page and making sure that whatever is appearing in that area had better be the most relevant result possible for the user.  In conversations with other engines I hear things like balanced ecosystems and communities that include both users and advertisers. I’ve always been struck by the focus at Google and I’ve always been a strong believer that corporations need sacred cows, these untouchable driving principles that everyone can rally around.  Is that what we’re talking about here with Google?

Nick: I think it is.  I think it comes from the top and it comes from the roots. If we were doing a proposal to Larry and Sergey and Eric where we’re saying, “Hey, let’s show a bunch of low quality ads”  the first question they’re going to ask is “Is this the right thing for the user?”  And if the answer is no, we get kicked out of the room and that’s the end of conversation. So you get that from the top and it permeates all the way through. You hear it when you speak to Marissa and Matt and us. It permeates the conversations we have here as well.  It’s not just external when we talk about the user; it’s what the conversation is internally as well. It just exudes through the company because it’s just part of what we think. I wouldn’t say that there isn’t a focus on the advertiser too, it’s just that our belief is that the way you get that balance is by focusing on the user, and as long as the user’s happy, the user’s clicking on the ad, and as long as the user’s clicking on the ad, the advertiser’s getting leads and everything works. If you focus on the advertiser’s in the short term, maybe the advertisers will be happy in the short term, but in the long term that doesn’t work. That used to be a hard message to get across. It used to be the case that advertiser’s didn’t really get that. And one of the most rewarding things for me is that the advertisers see that, they get that. Some of the stuff we do in the world of ad quality is frustrating to advertisers because in some cases we’re preventing their ads from running in cases where they’d like it to run. We’ve seen that the advertiser community is actually more receptive to that recently because they understand why we’re doing it and they understand that in the long term, they’re benefiting from it as well. I think that you are seeing that there is a difference in approach between us and our competitors. That we believe the ecosystem thrives if you focus on the users first.

Gord: I’d like to focus on what, to me, what’s a pretty significant performance delta between right rail and top sponsored. We’ve seen the scan patterns put top sponsored directly in the primary scanning path of users where right rail is more of a side bar that may be considered after the primary results are scanned. With whatever you can share, can you tell me a little about what’s behind that promotion from right rail to top sponsored?

Nick: Yes, it’s based on two things.  One is the primary element is the quality of the ad. The highest quality ads get shown on the top. The lower quality ads get shown on the right hand side. We block off the top ads from the top of the auction, if you really believe those are truly excellent ads…

Diane: It’s worth pointing out that we never break auction order…

Nick: One of the things that’s sacred here is making sure that the advertiser’s have the incentive. In an auction, you want to make sure that the folks who win the auction are the ones who actually did win the auction. You can’t give the prize away to the person who didn’t win the auction. The primary element in that function is the quality of the ad. Another element of function is what the advertiser’s going to pay for that ad. Which, in some ways, is also a measure of quality. We’ve seen that in most cases, where the advertiser’s willing to pay more, it’s more of a commercial topic. The query itself is more commercial, therefore users are more likely to be interested in ads. So we typically see that queries that have high revenue ads, ads that are likely to generate a lot of revenue for Google are also the queries where the ads are also most relevant to the user, so the user is more likely to be happy as well. So it’s those two factors that go into it. But it is a very high threshold. I don’t’ want to get into specific numbers, but the fraction of queries that actually show these promoted ads is very small.

Gord: One thing we’ve noticed is, actually in an eye tracking study we did on Google China, there where the search market is far less mature, you very, very seldom see those ads being promoted to top sponsored. So I would imagine that that’s got to be a factor. Is the same threshold applied across all the markets or does it vary, does the quality threshold vary from market to market?

Nick:  I don’t want to get too much into the specifics of that kind of detail. We do certainly take an approach in market that we believe is most effective for that market. Handling everything at a global level doesn’t really make a lot of sense because in some cases you have micro markets that, or, in the case of China, a large market, where it makes sense to tailor our approach to what makes sense for that market…what users from that market are looking for, what the maturity of that market is. A market that has a different level of search quality, for example, it might make sense to take a different approach in how we think about ads as well. So that’s what I want to say there. But you’re right, in a market like China that’s less mature and at the early stage of it’s development, you do see fewer ads at the top of the page, there are just fewer ads there that we believe are good enough to show at the top of the page. Contrast that with a country like the U.S. or the U.K., where these markets are very mature and have the high quality ads we feel comfortable showing at the top, we show top ads.

Diane: But market maturity is just one area we look at. There’s also user sophistication with the internet and other key factors. We have to take all this into account to really decide what the approach is on a market basis.

Gord: One of the questions that always comes up every time I sit on a panel that has anything to do with quality scoring is what’s in an ad that might generate a click through is not necessarily what will generate a quality visitor when you carry it forward into conversion. For instance you can entice someone to click through but they may not convert and, of course, if you’re enticing them to click through you’re going to benefit from the quality scoring algorithm.  How do we correct that in the future?

Nick:  I think there are two things. One is, in general, an ad’s that’s being honest, and gets a high click rate from being honest,  is essentially a very relevant ad and therefore gets a high click through rate. We’ll typically see that that ad also has a high conversion rate. In cases where the advertiser’s not being dishonest, the high click through rate is generally correlated with a high conversion rate. And it’s simply because that ad is more relevant, it’s more relevant in terms of getting the user to click on that ad in the first place, it’s also more relevant in delivering what that user is looking for once they actually got to the landing page. So you see a good correlation there.

There are cases where advertisers can do things where they’re misleading in their ad text and create an incentive for a user to click on their ad and then not be able to deliver, so the advertiser could say “great deals on iPods” and then they sell iPod cases or something. In that case, the high click through rate is unlikely to be correlated with a high conversion rate because the users are going to be disappointed when they actually end up on the page. The good thing for us is that the conversion rate typically gets reflected in the amount that the advertiser’s actually willing to pay, so that’s one of the reasons why the advertiser’s bid is a relatively decent metric of the quality, for example in this ipod cases case, because that conversion rates likely to be low, the advertiser’s not likely to bid as much for that. The click just isn’t worth as much to them, therefore they’ll bid less and end up getting a lower rank as a result of that. So, in many cases, this doesn’t end up being a problem because that just sort of falls out of the ranking formula. It’s a little bit convoluted.

Gord: Just to restate it to make sure I’ve got it here. You’re saying that if somebody is being dishonest, ultimately the return they’re getting on that will dictate that they have to drop their bid amount, so it will correct itself. If they’re not getting the returns on the back end, they’re not going to pay the same on the front end and ultimately it will just find it will just find it’s proper place.

Nick: What an advertiser should probably be thinking most about is mostly ROI per click…it’s actually ROI per impression. From the ad that’s likely to generate the most value for the user, and therefore the most value to Google as well as the most value to the advertiser, all aligned in a very nice way, is the ad that’s the most likely to generate the most ROI per impression. And because of our ranking formula, those are the ads that are most likely to show up at the top of the auction. And the ones that aren’t fall out. So the advertiser should care click through rate, but they shouldn’t care about click through rate exclusively to the extent that that results in a low conversion rate and a low ROI per click for them.

Gord: We talked a little bit about ads being promoted to the top sponsored and over the past three or four years, you have experimented a little bit with the number of ads that you show up there. When we did our first eye tracking study, usually we didn’t see any more than two ads, and that increased to three shortly after. Have you found the right balance with what appears above organic results as far as sponsored results?

Diane: I would say that it’s one of those things where the user base is
constantly shifting, the market is constantly shifting. It’s something that we definitely reevaluate frequently. It was definitely a very thought through decision to move to three, and we show three actually very rarely. We seriously consider that when we show three, is it in the best interest for the user? There’s a lot of evaluation of the entire page at that point and not even just the ads, whether or not it was the right thing. We’re very careful to make sure that we’re constantly at the right balance. It’s definitely something that we look at.

Gord: One of the things we’ve noticed in our eye tracking studies is that there’s a tendency on the part of users to “break off” results in consideration sets and the magic number seems to be around four, so what we’ve seen is even if they’re open to looking at sponsored ads, they want to include at least the number one organic result as well, as kind of a baseline for reference. They want to be able to flip back and forth and say, “Okay, that’s the organic result, that how relevant I feel that is. If one of the sponsored ads is more relevant, than fine, I’ll click on it.” It seems like that’s a good number for the user to be able to concentrate on at one time, quickly and then make their decision based on that consideration set that would usually include one or two sponsored ads and at least one organic listing, and where the highest relevancy is. Does that match what you guys have found as well?

Nick: I don’t think we’ve looked at it in the way of consideration sets, along those lines. I think that’s consistent with the outcomes that we’ve had and maybe some of the thought process that lead us to our outcome. The net effect is the same outcome. One of the things that we are careful about is trying to make sure that you don’t want to create an experience where you show no organic results on the page, you know, or at least above the fold on the page. You want to make sure that the user is going to be able to make that decision, regarding what they want to click on and if you just serve the user with one type of result you’re not really helping the user make that type of decision. What we care more about is what the user sees in the above the fold real estate, not quite so much the full result. And probably relatively consistent on certain sets of screen resolutions.

Gord: One of the things that Marissa said when I talked to her a few days ago was that as Google moves into Universal Search results and we’re starting to see different types of results appear on the page, including in some cases images or videos, that opens the door to potentially looking at different presentations of advertising content as well. How does that impact your quality scoring and ultimately how does that impact the user?

Nick: We need to see. I don’t think we know yet. Ultimately it would be our team deciding whether to do that or not, so fortunately we don’t have to worry too much about hooking up the quality score because we would design a quality score that would make sense for it. The team that focuses on what we call Ad UI, that’s the team that’s looking at …it’s sub group within that, that’s the team that essentially thinks about what should the ads actually look like?

Diane: And what information can we present that’s most useful to the user?

Nick: So in some cases, that information may be an image, in some cases that information may be a video. We need to make sure in doing this that we’re not just showing video ads, because video happens to be catchy. We want to make sure that we’re showing video ads because the video is what actually contains the content that’s actually useful for the user. With Universal Search we found that video search results, for example, can contain that information, so it’s likely that their paid results set could be the same as well. Again, just as in text ads, we’d need to make sure that whatever we do there is user driven rather than anything else and that the users are actually happy with it. There would be a lot of user experimentation that would happen before anything was launched along those lines.

Diane: You can track our blogs as well. All of our experiments show up at some point there.

Gord: Right. Talking a little bit about personalization, you started off by saying that Larry and Sergey have dictated that the ads should be more relevant than the organic results in an ideal situation and just as a point of interest, in our second eye tracking study, when we looked at the success rate of click throughs as far as people actually clicking through to a site that appeared to deliver what they were looking for, for commercial tasks, it was in fact the top sponsored ads that had the highest average success rate of all the links on the page. When we’re looking at Personalization, one of the things that, again, Marissa said is we don’t want our organic results and our sponsored results to be too far out of sync. Although personalization is rolling out on the organic side right now, it would make sense, if that can significantly improve the relevancy to the user, for that to eventually fold into the sponsored results as well. So again, that might be something that would potentially impact quality scoring in the future, right?

Nick: Yes. So we have been looking at some.. I’m not sure if the right word is personalization or some sort of user based or task based…what the right word is..changes to how we think about ads. We have made changes to try to get a sense of what the user’s trying to do right now. Whether they’re, for example, in a commercial mind set and alter how we do ads somewhat based on that type of an understanding of the user’s current task. We’ve done much less with trying to..we’ve done nothing really…with trying to build profiles of the user and trying to understand who the user is and whether the user is a man or woman or a 45 year old or a 25 year old. We haven’t seen that that’s particularly useful for us. You don’t want to personalize users into a corner, you don’t want to create a profile of them that’s not actually reflective of whom they are. We don’t want to freak the user out. If you have a qualified user you could risk alienating that user. So we’ve been very hesitant to move in that direction and in general, we think that there’s a lot more we can that doesn’t require profiles down that path.

Diane: You can think of personalization in a couple of different ways, right? It can manifest itself in regards to the results you actually show. It can also be more about how many ads or even the presentation of those ads with regards to actual information. Those sorts of things. There are many possible directions that can be more fruitful than, like Nick points out, profiling.

Gord: Right, right.

Nick: For example, one of the things that you could theoretically do is, as you know, we changed the background color of our top ads from blue to yellow, because we found that yellow works better in general. You might find that for certain users, green is better, you might find that for certain users, blue is actually better. Those types of things, where you’re able to change things based on what users are responding to, is more appealing to us than these broad user classification types of things. It seems somewhat sketchy.

Gord: It was funny. Just before those interview, I was actually talking to Michael Ferguson at Ask.com and one of the things he mentioned that I thought was quite interesting was a different take on personalization. It may get to the point where it’s not just using personalization for the sake of disambiguating intent and improving relevancy, it might actually be using personalization to present results or advertising messages in the form that’s most preferred by the user. So some may prefer video ads. Some may prefer text ads and they may prefer shorter text ads or longer text ads. And I just thought that that was really interesting. Looking at personalization to actually customize how the results are being presented to you. In what format.

Nick: Yes.

Gord. One last question. You’ve talked before about quality scoring and how it impacts two different things. Whether it’s the minimum bid price or whether it’s actually position on the page. And the fact that there’s more factors, generally, in the “softer” or “fuzzier” minimum bid algorithm than there is in the “hard” algorithm that determines position on the page. And ideally you would like to see more factors included in all of it. Where is Google at on that line right now?

Nick: There are probably two things. One is that when setting the minimum bid, we have much less information available to us. We don’t know what the specific query is that the user issued. We don’t know what time of day it is. We know very little about the context of what the user is actually trying to do. We don’t know what property that user’s on. There’s a whole lot that we don’t know. What we need to do when we set a minimum bid is much coarser. We just need to be able to say, what do we think this keyword is, what do we think the quality of the ad is, does the keyword meet the objective of the landing page and make a judgment based on that. But we don’t have the ability to be more nuanced in terms of actually taking into account the context of how the ad is likely to actually show up. There’s always going to be a difference in terms of what we can actually use when we set the minimum bid versus what we use at auction time to set the position. The other piece of it though is there are certain pieces that only affect the minimum bid. Let me give you an example. Landing page quality normally impacts the minimum bid but it doesn’t impact your ranking. The reason for that is mostly from the standpoint of our decision to launch the product and what we thought was the most expedient way to improve the landing page quality of our ads rather than what we think will be the long term design of the system. So I’d expect things like that, where signals like landing page quality should impact not only the minimum CPC but also rank which ads show at the top of the page and things like that as well. That’s where you’ll see more convergence. But there’s always going to be context that we can get at query time to use for the auction than we can for minimum CPC.

Ask: The Reasoning Behind Ask 3D

Last week in my interview with Jakob Nielsen, he called Ask’s 3D label “stupid”. Just to refresh your memory, here’s how the exchange went:

Gord: Like Ask is experimenting with right now with their 3D search. They’re actually breaking it up into 3 columns, and using the right rail and the left rail to show non-web based results.

Jakob: Exactly, except I really want to say that it’s 2 dimensional, it’s not 3 dimensional.

Gord: But that’s what they’re calling it.

Jakob: Yes I know, but that’s a stupid word. I don’t want to give them any credit for that. It’s 2 dimensional. It’s evolutionary in the sense that search results have been 1 dimensional, which is linear, just scroll down the page, and so potentially 2 dimensional (they can call it three but it is two) that is the big step, doing something differently and that may take off and more search engines may do that if it turns out to work well.

My friend Michael Ferguson at Ask (who has his own interview coming up soon) sent me a quick email with the reasoning behind the label:

The 3D label came from the 3 dimensions of search we folded onto one page: query expression in the left rail, results in the center, and content on the right (vs. the one dimension of returning solely results).

Interview with Jakob Nielsen on the Future of the SERP (and other stuff)

jakob-nielsen_cropped.jpg.400x400_q95_crop_upscaleI recently had the opportunity to talk to Jakob Nielsen for a series I’m doing for Search Engine Land about what the search results page will look like in 2010.  Jakob is called a “controversial guru of Web design” in Wikipedia (Jakob gets his own shots in at Wikipedia in this interview) because of his strongly held views on the use of graphics and flash in web design. I have a tremendous amount of respect for Jakob, even though we don’t agree on everything, because of his no frills, common sense approach to the user experience. And so I thought it was quite appropriate I sound him out on his feelings about the evolution of the search interface, now that with Universal search and Ask’s 3D Search we seem to be seeing more innovation in this area in the last 6 months than we’ve seen for the last 10 years. Jakob is not as optimistic about the pace of change as I am, but the conversation was fascinating. We touched on Universal Search, personalization, banner blindness on the SERP and scanning of the web in China, amongst other things. Usability geeks..enjoy!

Gord: For today I only really have one question, although I’m sure there be lots of branch offs from it. It revolves around what the search engine results page may look like in 2010.  I thought you would be a great person to lend your insight on that.

Jakob: Ok, sure.

Gord: So why don’t we just start? Obviously there are some things that are happening now with personalization and universal search results. Let’s just open this up. What do you think we’ll be seeing on a search results page in 3 years?

Jakob: I don’t think there will be that big a change because 3 years is not that long a time. I think if you look back three years at 2004, there was not really that much difference from what there is today.  I think if you look back ten years there still isn’t that much difference.  I actually just took a look at some old screen shots in preparation before this call at some various search engines like Infoseek and Excite and those guys that were around at that time, and Google’s Beta release, and the truth is that they were pretty similar to what we have today as well.  The main difference, the main innovation seems to have been to abandon banner ads, which we all know now really do not work, and replace them with the text ads, and of course that affected the appearance of the page.  And of course now the text ads are driven by the key words, but in terms of the appearance of the page, they have been very static, very similar for 10 years.  I think that’s quite likely to continue. You could speculate the possible changes. Then I think there are three different big things that could happen.

One of them that will not make any difference to the appearance and that is a different prioritization scheme. Of course, the big thing that has happened in the last 10 years was a change from an information retrieval oriented relevance ranking to being more of a popularity relevance ranking. And I think we can see a change maybe being a more of a usefulness relevance ranking. I think there is a tendency now for a lot of not very useful results to be dredged up that happen to be very popular, like Wikipedia and various blogs. They’re not going to be very useful or substantial to people who are trying to solve problems. So I think that with counting links and all of that, there may be a change and we may go into a more behavioral judgment as to which sites actually solve people’s problems, and they will tend to be more highly ranked.

But of course from the user perspective, that’s not going to look any different. It’s just going to be that the top one is going to be the one that the various search engines, by what ever means they think of, will judge to be the best and that’s what people will tend to click first, and then the second one and so on. That behavior will stay the same, and the appearance will be the same, but the sorting might be different. That I think is actually very likely to happen

Gord: So, as you say, those will be the relevancy changes at the back end. You’re not seeing the paradigm of the primarily text based interface with 10 organic results and  8-9 sponsored results where they are, you don’t see that changing much in the next 3 years?

Jakob: No.  I think you can speculate on possible changes to this as well. There could be small changes, there could be big changes.  I don’t think big changes. The small changes are, potentially, a change from the one dimensional linear layout to more of a two dimensional layout with different types of information, presented in different parts of the page so you could have more of a newspaper metaphor in terms of the layout. I’m not sure if that’s going to happen.  It’s a huge dominant user behavior to scan a linear list and so this attempt to put other things on the side, to tamper with the true layout, the true design of the page, to move from it being just a list, it’s going to be difficult, but I think it’s a possibility.  There’s a lot of things, types of information that the search engines are crunching on, and one approach is to unify them all into one list based on it’s best guess as to relevance or importance or whatever, and that is what I think is most likely to happen.  But it could also be that they decide to split it up, and say, well, out here to the right we’ll put shopping results, and out here to the left we’ll put news results, and down here at the bottom we’ll put pictures, and so forth, and I think that’s a possibility.

Gord: Like Ask is experimenting with right now with their 3D search. They’re actually breaking it up into 3 columns, and using the right rail and the left rail to show non-web based results.

Jakob: Exactly, except I really want to say that it’s 2 dimensional, it’s not 3 dimensional.

Gord: But that’s what they’re calling it.

Jakob: Yes I know, but that’s a stupid word. I don’t want to give them any credit for that. It’s 2 dimensional. It’s evolutionary in the sense that search results have been 1 dimensional, which is linear, just scroll down the page, and so potentially 2 dimensional (they can call it three but it is two) that is the big step, doing something differently and that may take off and more search engines may do that if it turns out to work well.  But I think it’s more likely that they will work on ways on integrating all these different sources into a linear list. But those are two alternative possibilities, and it depends on how well they are able to produce a single sorted list of all these different data sources.  Can they really guess people’s intent that well?

All this stuff..all this talk about personalization, that is incredibly hard to do. Partly because it’s not just personalization, based on a user model, which is hard enough already. You have to guess that this person prefers this style of content and so on.  But furthermore, you have to guess as to what this person’s “in this minute” interest is and that is almost impossible to do. I’m not too optimistic on the ability to do that.  In many ways I think the web provides self personalization, you know, self service personalization. I show you my navigational scheme of things you can do on my site and you pick the one you want today, and the job of the web designer is to, first of all, design choices that adequately meet common user needs, and secondly, simply explain these choices so people can make the right ones for them.  And that’s what most sites do very poorly. Both of those two steps are done very poorly on most corporate websites. But when it’s done well, that leads to people being able to click – click and they have what they want, because they know what they want, and its very difficult for the computer to guess what they want in this minute.

Gord:  When we bring it back to the search paradigm, giving people that kind of control to be able to determine the type of content that’s most relevant to them requires them interacting with the page in some way.

Jakob: Yes, exactly, and that’s actually my third possible change. My first one was changing to the ranking scheme; the second one was the potentially changing to two dimensional layouts. The third one is to add more tools to the search interface to provide query reformulation and query refinement options. I’m also very skeptical about this, because this has been tried a lot of times and it has always failed.  If you go back and look at old screen shots (you probably have more than I have) of all of the different search engines that have been out there over the last 15 years or so, there have been a lot of attempts to do things like this. I think Microsoft had one where you could prioritize one thing more, prioritize another thing more. There was another slider paradigm. I know that Infoseek, many, many years ago, had alternative query terms you could do just one click and you could search on them, which was very simple. Yet most people didn’t even do that.

People are basically lazy, and this makes sense.  The basic information foraging theory, which is, I think, the one theory that basically explains why the web is the way it is, says that people want to expend minimal effort to gain their benefits.  And this is an evolutionary point that has come about because the people, or the creatures, who don’t exert themselves, are the ones most likely to survive when there are bad times or a crisis of some kind. So people are inherently lazy and don’t want to exert themselves. Picking from a set of choices is one of the least effortful interaction styles which is why this point and click interaction in general seems to work very well. Where as tweaking sliders, operating pull down menus and all that stuff, that is just more work.

Gord: Right.

Jakob: But of course, this depends on whether we can make these tools useful enough, because it’s not that people will never exert themselves.  People do, after all, still get out of bed in the morning, so people will do something if the effort is deemed worthwhile.  But it just has to be the case that if you tweak the slider you get remarkably better results for your current needs.  And it has to be really easy to understand. I think this has been a problem for many of these ideas. They made sense to the search engine experts, but for the average user they had no idea about what would happen if they tweaked these various search settings and so people tended to not do them.

Gord: Right. When you look at where Google appears to be going, it seems like they’ve made the decision, “we’ll keep the functionality transparent in the background, we’ll use our algorithms and our science to try to improve the relevancy”, where as someone like Ask might be more likely to offer more functionality and more controls on the page. So if Google is going the other way, they seem to be saying that personalization is what they’re betting on to make that search experience better.  You’re not too optimistic that that will happen without some sort of interaction on the part of the user?

Jakob: Not, at least, in a small number of years. I think if you look very far ahead, you know 10, 20, 30 years or whatever, then I think there can be a lot of things happening in terms of natural language understanding and making the computer more clever than it is now. If we get to that level then it may be possible to have the computer better guess at what each person needs without the person having to say anything, but I think right now, it is very difficult.  The main attempt at personalization so far on the web is Amazon.com. They know so much about the user because they know what you’ve bought which is a stronger signal of interest than if you had just searched for something.  You search for a lot of things that you may never actually want, but actually paying money; that’s a very, very strong signal of interest.  Take myself, for example. I’m a very loyal shopper of Amazon. I’ve bought several hundred things from them and despite that they rarely recommend (successfully)…sometimes they actually recommend things I like but things I already have. I just didn’t buy it from them so they don’t know I have it. But it’s very, very rare that they recommend something where I say, “Oh yes, I really want that”. So I actually buy it from them.  And that’s despite the (fact that the) economic incentive is extreme, recommending things that people will buy. And they know what people have bought. Despite that and despite their work on this now for already 10 years (it’s always been one of their main dreams is to personalize shopping) they still don’t have it very well done. What they have done very well is this “just in time” relevance or “cross sell” as it’s normally called. So when you are on one book on one page, or one product in general, they will say, here are 5 other ones that are very similar to the one you’re looking at now. But that’s not saying, in general, I’m predicting that these 5 books will be of interest to you. They’re saying, “Given that you’re looking at this book, here are 5 other books that are similar, and therefore, the lead that you’re interested in these 5 books comes from your looking at that first book, not from them predicting or having a more elaborate theory about what I like.

Gord: Right.

Jakob: What “I like” tends not to be very useful.

Gord: Interesting. Jakob, I want to be considerate of your time but I do have one more question I’d love to run by you.  As the search results move towards more types of images, we’re already seeing more images showing up on the actual search results page for a lot of searches. Soon we could be seeing video and different types of information presented on the page. First of all, how will that impact our scanning patterns?  We’ve both done eye scanning research on search engine results, so we know there is very distinct patterns that we see.  Second of all, Marissa Mayer in a statement not that long ago seemed to backpedal a bit about the fact that Google would never put display ads back on a search results page, seeming to open a door for non text ads.  Would you mind commenting on those two things?

Jakob: Well they’re actually quite related.  If they put up display ads, then they will start training people to exhibit more banner blindness, which will also cause them to not look at other types of multimedia on the page. So as long as the page is very clean and the only ads are the text ads that are keyword driven, then I think that putting pictures and probably even videos on there actually work well.  The problem of course is they are inherently a more two dimensional media form, and video is 3 dimensional, because it’s two dimensional – graphic, and the third dimension is time, so they become more difficult to process in this linear type of scanned document “down the page” type of pattern.  But on the other hand people can process images faster, with just one fixation and you can “grok” a lot of what’s in an image, so I think that if they can keep the pages clean, then it will be incorporated in peoples scanning pattern a little bit more. “Oh this can give me a quick idea of what this is all about and what type of information I can expect”.  This of course assumes as well one more thing which is that they can actually select good pictures.

Gord: Right.

Jakob: I would be kind of conservative until higher tweaking with these algorithms, you know, what threshold should you cross before you put an image up.  I would really say tweak it such so that you only put it up when you’re really sure that it’s a highly relevant good image.  If there starts becoming that there are too many images, then we start seeing the obstacle course behavior. People scan around the images, as they do on a lot of corporate websites, where the images tend to be stock photos of glamour models that are irrelevant to what the user’s there for.  And then people involve behavior where they look around the images which is very contrary to first principals of perceptual psychology type of predicting which would be that the images would be attractive. Images turn out to be repelling if people start feeling like they are irrelevant. It’s a similar effect to banner blindness. If there’s any type of design element that people start perceiving as being irrelevant to their needs, then they will start to avoid that design element.

Gord: So, they could be running the risk of banner blindness, by incorporating those images if they’re not absolutely relevant…

Jakob: Exactly.

Gord: …to the query. Ok thank you so much.  Just out of interest have you done a lot of usability work with Chinese?

Jakob: Some. I actually read the article you had on your site. We haven’t done eye tracking studies, but we did some studies when we were in Hong Kong recently, and to that level the findings were very much the same. In terms of pdf was bad and how people go though shopping carts. So a lot of the transactional behavior, the interaction behavior, is very, very similar.

Gord: It was interesting to see how they were interacting with the search results page.  We’re still trying to figure out what some of those interactions meant

Jakob: I think it’s interesting. It can possibly be that the alphabet or character set is less scannable, but it is very hard to say because when you’re a foreigner, these characters look very blocky, and it looks very much like a lot of very similar scribbles.  But on the other hand, it could very well be the same, that people who don’t speak English would view a set of English words like a lot of little speck marks on the page, and yet words in English or in European languages are highly scannable because they have these shapes.

Gord: Right.

Jakob: So I think this is where more research is really called for to really find out.  But I think it’s possible, you know the hypothesis is that it’s just less scannable because the actual graphical or visual appearance of the words just don’t make the words pop as much.

Gord: There seems to be some conditioning effects as well and intent plays a huge part.  There’s a lot of moving pieces with that and we’re just trying to sort out. The relevancy of the results is a huge issue because the relevancy in China is really not that good so…

Jakob: It seems like it would have a lot to do with experience and amount of information.  If you compare back with uses of search in the 80’s, for example, before the web started, that was also a much more thorough reading of search results because people didn’t do search very well. Most people never did it actually, and when you did do it you would search through a very small set of information, and you had to carefully consider each probability. Then, as WebCrawler and Excite and AltaVista and people started, users got more used to scanning, they got more used to filtering out lots of junk. So the paradigm has completely changed from “find everything about my question” to “protect myself against overload of information”.  That paradigm shift requires you to have lived in a lot of information for awhile.

Gord: I was actually talking to the Chinese engineering team down at Yahoo! and that’s one thing I said. If you look at how the Chinese are using the internet, it’s very similar to North America in 99 or 2000. There’s a lot of searching for entertainment files and MP3s. They’re not using it for business and completing tasks nearly as much. It’s an entertainment medium for them, and that will impact how their browsing things like search results. It’ll be interesting to watch as that market matures and as users get more experienced, if that scanning pattern condenses and tightens up a lot

Jakob: Exactly. And I would certainly predict it would. There could be a language difference, basically a character set as we just discussed, but I think the basic information foraging theory is still a universal truth. People have to protect themselves against information overload, if you have information overload. As long as you’re not accustomed to that scenario, then you don’t evolve those behaviors. But once you get it… I think a lot of those people have lived in an environment where there’s not a lot of information.  Only one state television channel and so forth and gradually they’re getting satellite television and they’re getting millions of websites. But gradually they are getting many places where they can shop for given things, but that’s going to be an evolution.

Gord: The other thing we saw was that there was a really quick scan right to the bottom of the page, within 5 seconds, just to determine how relevant these results were, were these legitimate results? And then there was a secondary pass though where they went back to the top and then started going through. So they’re very wary of what’s presented on the page, and I think part of it is lack of trust in the information source and part of it is the amount of spam on the results page.

Jakob: Oh, yes, yes.

Gord: Great thanks very much for your time Jakob.

Jakob: Oh and thank you!

Interview with Ask’s Michael Ferguson

I recently had the opportunity to chat with one of my favorite usability people, Michael Ferguson at Ask.com. You can find excerpts of the interview, along with commentary, on Search Engine Land in this week’s Just Behave column. Some of Michael’s comments are particularly timely now, given Google’s announcement of Universal search.

Gord: How does Ask.com approach the search user experience and in big terms, what is your general philosophy?

Michael: A lot of what we do is, to some extent, informed by core search needs but also by our relevant market share, understanding that people have often experienced other engines before they come to us, not necessarily in that session but generally on the web. People have at least done a few searches on Google and Yahoo, so they have some context coming from those search experiences. So often, we’re taking what we’ve learned from best practices from competitors and others and then, on top of that, trying to add a lot of product experience and relevance experiences that are differentiated. Of course, we’re coming from this longer history of the company where we’ve had various user experiences over the time that we’ve been around. We’ve marketed around natural language, in the late 90’s and answered people’s questions at the top of the page, but in the last year and a half or so, we’ve rebranded and really focused on getting the word out to the end users that we are a keyword search engine, an everyday search engine.

A lot of the things that we’ve done with users have been to try to, implicitly, if not explicitly, inform users that are coming to the site you can use it very much like you can use any other kind of search engine you’ve been on before. Or, if they’re current users and people are coming back to the site, to let them know that the range of experiences and the type of information we bring back to them has greatly expanded. So that’s pretty much it. It’s informed by the context of not just a sense of pure search and information retrieval and all the research that’s gone on that in the last 35 or 40 years but also the dynamics of the experiences that we’ve had before and people’s previous experiences with Ask. Then, an acknowledgement that they’ve often searched on other sites and looked for information.

Gord: You brought up a number of topics that I’d like to touch on, each in sequence. You mentioned that in a lot of cases, they’re coming to Ask and they’ve used Google or Yahoo or they’ve used another engine as one of their primary search tools. Does Ask’s role as a supplemental engine or an alternative engine give you a little more latitude? You can add things from a functionality point of view to really differentiate yourselves. I actually just did a search and see that you, at least on my computer here, have made the move to incorporate some of the things that you were testing on AskX into the main site. Maybe we’ll start there. Is that an ongoing test? Am I just part of a beta test on that or this rollover complete now?

Michael: We’re still in testing with that and it will roll out. We have decided because of a lot of the user experience metrics that we’re getting from the beta test that we’re going to go for it. We have decided to move the full experience over to the AskX experience. Of course, there are variants to that, but the basic theme of, in a smart way, bringing together results from different search verticals and wrapping those around the core organic results (as well as) a sponsored experience. So that will happen sometime this year. We don’t know exactly when, but just a couple of days ago, we really decided we’ve seen enough and we’re pretty excited about that.

Google has a really great user experience going, and Yahoo does too, but they have so many different levers that move so much revenue and traffic and experience metrics that I think it’s harder for them to take chances and to move things around and get buy-offs at a bureaucratic level. To some extent, we see ourselves as having permission and a responsibility to really innovate on the user experience. It’s definitely a good time for us because we have such great support from IAC and they’re very much invested in us improving the user experience and getting more traffic and getting frequency and taking market share and they’re ready to very much invest in that. So we don’t need to cram the page with sponsored links and things like that. It’s mostly a transitional time when we’re getting people to reconsider the brand and the search engine as a full keyword based, everyday search engine that has lots to offer. I’m talking to people all the time about Ask and there’s definitely still people that say, “Hey, last night, it came up with my buddies at the bar, this trivia question about the Los Angeles Lakers, 1966 to 1972 (and I went to Ask and asked a question)”. Then there are other people that see us as evolving beyond that but still really surprised that we haven’t had image search.  Now with AskX we’ll have preview search and there’s lots of other stuff coming along now. So yes, it’s a great place to be. I love working with it. There are so many things that, in an informed way, we can take chances on, relative to our competitors.

Gord: So does this mean that the main site becomes more of an active site? Are you being more upfront with the testing on Ask.com rather than on AskX.com?

Michael: Well, I think the general sense of what we’re going to do is that, at some point this year, the AskX experience will, at least at a wireframe level, become the default experience and, of course, we have a lot of next generation “after that” stuff queued up that we’re thinking about and we’re actively testing right now but not in any live sense.  So potentially, things will slide in behind the move of the full interface going out and then AskX will remain a sandbox for another instance of, hopefully, new and really useful and differentiated search experience coming after that. A general thing that we’re going to try to do, instead of having 15 or 18 different product managers and engineering teams working on all these different facets of information retrieval and services, we’re going to stay search focused and just have one sandbox area where people go in and see multiple facets of what we’re thinking about.

Gord: Let’s talk about the sponsored ads for a bit. I notice that for a couple of searches that I’ve done while we’ve been talking that they’ve definitely been dialed down as far as the presence of sponsored on the page. I’m only seeing top sponsored appear, so you’re using the right rail to add additional search value or information value, whether it be suggested searches or on a local search, where it brought me back the current weather and time. So what’s the current strategy on Ask as far as presentation of sponsored results and the amount of real estate devoted to them?

Michael: Just to fit along with the logic of Eye Tracking II (Enquiro’s second eye tracking study), those ads are not a delineated part of the user experience for the end user and they’re relevance and their frequency can color the perception of the rest of the page and especially the organic listings below them. Right now, as I said, we’re very much focusing on improved user experience and building frequency and retention of customers, which all the companies are, I’m sure. But we’re really being, basically, cautious with the ads and getting them there when they’re appropriate and, as best we can, adjust them over time, so that when they’re there, they’re going to valuable for the user and for the vendor.

Gord: That’s a fairly significant evolution in thinking about what the results page looks like from say, two years ago, with Ask. Is that purely a function of IAC knowing that this is a long term game and it begins with market share and after that comes the monetization opportunities?

Michael: Actually, I think way before we got acquired by IAC we knew that. We test like other engines would. We test lots of different ad configurations and presentations and things like that but definitely you want to balance that. Way before we got acquired we realized that there’s one thing that’s kind of fun about making the quarter and blowing through it a little bit and then there’s another thing about eroding customers. And definitely there’s a lifetime value that can be gained by giving people what you know is a better user experience over time, so once we did become part of the IAC family, we brought them up to speed with the results that we were finding that were pointing to taking that road and they’ve very much been in support of it. And, of course, their revenue is spread amongst a lot of different pieces of online and offline business so their ability to absorb it is probably more flexible than ours was as a stand alone company.

Gord: That brings me to my next question, which is, with all the different properties that IAC has and their deep penetration into some of the vertical areas, you had talked about the opportunity to bring some of that value to the search results page. What are we looking at as far as that goes? Are we going to see more and more information pulled from other IAC into the main AskX interface?

Michael: Maybe the most powerful thing about the internet is that you as an individual now have a very empowered position relative to other producers of information, other businesses where you can consume a bunch of different points of view. You have a bunch of different opportunities to do business and get the lowest price and read reviews that the company itself hasn’t sanctioned, or anything like that.  You have access to your peer network and to your social networks. Search, like the internet, becomes, and it necessarily needs to be, a proxy for that neutral, unbiased view of all the information that’s available. This probably gets a little bit into what may or not may work with something like Google’s search history. Users over time have said again and again, “Don’t hide anything from me or don’t over think what you may think I might want. Give me all of the best stuff, use your algorithms to rank all that, but if I get the sense that anything’s biased or people are paying for this, then I’m not going to trust you and I’m going to go somewhere else where I can get that sense of empowerment again.”

As I’ve sat in user experience research over time, I’ve seen people..and I know this isn’t true of Google and I know it isn’t true of Ask right now with the  retraction from paid inclusion…but you ask users why they think this came up first on Google, maybe with a navigational query like Honda or Honda Civic and Honda comes up first. They’ll say, “Oh, Honda paid for that.” So even with the engines that aren’t doing paid inclusion, there’s still this kind of wariness that consumers have of just generally somebody on the internet, somewhere, behind the curtains, trying to take advantage of them or steer them in some way. So as soon as we got acquired by IAC, we have made it very much part of their perception of this and their culture. Their product management point of view is that you can’t sacrifice that neutrality. You can’t load a bunch of IAC stuff all over the place. The relationship with IAC does give us access to proprietary databases that we can do lots of deep dives in and get lots of rich information out  that can help the user in their instance of their search needs that other companies wouldn’t be able to get access to, while maintaining access to everything else.

The way we approached AskCity was a great example of this. We had leveraged a lot of CitySearch data but at the same time, we know that when people go out and want to see reviews, they want to see reviews from AOL Neighborhoods, they want to see reviews from Yelp they want to see reviews from all these other points of view too. So we go and scrape all those and fold them into the CitySearch stuff. We give access to all those results that come up on AskCity. If they’re, for instance, at a restaurant, you can get Open Table reviews and you can get movie reservations through Fandango and other stuff like that. Those companies have nothing to do with IAC. Those decisions were borne from user needs and from us looking as individuals in particular urban areas, and saying “Hey, what would I want to come up?” We know from previous experience from AOL that the walled garden thing doesn’t work. It’s just not what people expect from search and not what they expect from the internet, so that lesson’s been learned. I don’t know how much it would be different if we had some dominant market share over search, but that’s even more reason for us to be appealing to as wide a population as possible. That’s my philosophy right now.

Gord: I guess the other thing that every major engine is struggling with right now is in this quest to disambiguate intent, where is the trade-off with user control? Like you said, just show me a lot of the best stuff and I’ll decide where I want to drill down and I’ll change the query based on what I’m seeing to filter down to what I want. In talking to Marissa at Google and their moves towards personalization and introducing web history, I  think for anyone who understands how search engines work, it’s not that hard to see the benefits of personalization but from a user perspective there does seem to be some significant push back against that. Some users are saying, “I don’t want a lot of things happening in the background that are not transparent to me. I want to stay in control.” How is Ask approaching that?

Michael: The other major thing that’s going on right now is that we have fully revamped how we’re taking this. We developed the Direct Hit late 90’s technology. And then the Teoma technology we acquired. And really, it’s not that we’re taking those to the next level, we got all of that stuff together and over the past three years, we’ve been saying, “Okay, what do we have and what’s unique and differentiated?” There’s a lot of great user behavior data that Direct Hit understands.  We have a whole variety of things there and that’s unlocked, that’s across all the people coming in and out over time but not any personally identifiable type of stuff. And then there’s Teoma, which is good at seeing communities on the web, expertise within the communities and how communities relate. So right now, even though we have personalization stuff and My Stuff and other things that are coming up, we’re investing a lot more in the next version of the algorithm and the infrastructure for us to grow called Edison. And we started talking about that a week ago since A.G. (Apostolos Gerasoulis) mentioned it. Across a lot of user data it understands a lot about the context from the user intention side and because we’re constantly capturing the topology of the web and it’s communities and how they’re related, we then match the intention and the map of the web as it stands and the  blogosphere as it stands and other domains as they stand. Our Zoom product, which is now on the left under the search box in the AskX experience and it’s on the right on the live site, is the big area that we’re going to more passively offer people different paths.

For example, just like with AskX, you search for U2, it’s going to bring up news, and product results, and video results and images, and a Smart Answer at the top of the page. It’s also going to know that there’s U2 as the entity, the music band and therefore search the blogosphere but just search within music blogs. So what it’s doing, over time, is trying to give a personalized experience that’s informed by lots of behavior and trying to capture the structure of the web, basically. So that’s where we are there.

There’s a book that came out in early 1999 called Net Worth, which you might want to read. I almost want to revisit it myself now. It’s a Harvard Business School book that Marc Singer and John Hagel came out with. It talked about infomediaries and it imagined this future where there’d be these trusted brands and companies. They were thinking along the lines of American Express or some other concurrent banking entity at the time, but these infomediaries would have outside vendors come to them and they would entrust all their information, as much as they wanted to, they could control that, both online and offline.  You were talking in your latest blog post about understanding in the consideration phase where somebody is and presenting, potentially, websites that they hadn’t seen yet or ones that they might like at that point in the car purchase behavior. But the way that they were imagining it was that there would be a credit card that might show that someone had been taking trips from the San Francisco Bay area to the Tahoe region at a certain time of year and had maybe met with real estate agents up there and things like that. But these infomediaries, on top of not just web history but even offline stuff, would be a broker for all that information and there would be this nice marketplace where someone could come and say, “I want to pay $250 to talk to this person right now with this specific message”. So it seems that Google is doing a lot of that, especially with the DoubleClick acquisition. But I’m just wondering about the other side of it, keeping the end user aware of and empowered over that information and where it’s at. So Net Worth is a neat book to check out because the way they were describing it, the end user, even to the broker, would seep out exactly what they wanted to seep out at any given time. It wouldn’t be this passive recording device thing that’s silently taping. My experience so far of using the Google Toolbar that’s allowing the collection of history, is that it’s ambiguous to me about how much of my behavior is getting taken up by that system and used. We’ll see where it goes but right now we don’t have strong plans to do anything with that for search.

Gord: It’s going to be really interesting because, up to now, the tool bar was collecting data but there was no transparency into what it was collecting, and now that they’ve done that, we’ll see what the user response is to that. Now that they can go into their web history and have that initial shock of realizing how much Google actually does know about them.

One other question, and this is kind of a sidelight, but it’s always something that I’ve been interested in. Now that you have the search box along the left side there and it gives search suggestions as you’re typing, have you done any tracking to see how that’s altered your query logs? Have you noticed any trends in people searching differently now that you’re suggesting possible searches to them as they’re typing?

Michael: There are two broad things that are encouraging to us. One is that over time, the natural language queries are down tremendously. Our queries, because we promoted in the late nineties this “ask a question” thing, tended to be longer and more phrase based, more natural language based.  That’s really gone down and is approaching what we would consider normal for an every day search engine profile as far as the queries. And we really think that this zooming stuff has really helped that because it’s often keyword based. You will sometimes see some natural language stuff in there. There are communities on the web that are informing us that there’s an interest in this topic that’s related to the basic topic so it is helping change the user behavior on Ask.

And the other result of that is as people use it more for everyday keyword based search engine, the topics or the different categories of queries that people see are normalizing out too. Less and less they’re reference type stuff and more and more they’re transactional type queries, so that’s a good thing. And that’s just been happening as we rebranded and we presented Zoom.

And then with the AskX experience, we are definitely seeing that even more because of the fact that they’re just in proximity to the search box. We always knew that these suggestions should ideally be close to the search box so that people understand fully what we’re trying to offer them. For instance, on the current site, we do see users that will sometimes type a query in the search box on top and because they’re used to seeing ads on the right rail on so many other sites and because they don’t necessarily know what narrow and expand your search is they think those are just titles to other results or websites. It’s a relatively small portion. Most people get what it is, but there was that liability there. Now in the AskX experience, it’s close and visually grouped with the search box. It’s definitely getting used more and guiding queries and people seem even more comfortable putting general terms in. We’ve made it that you can just arrow down to the one and hit return. It’s definitely driving the queries differently.

Gord: I’ve always liked what you guys have done on the search page. I think it’s some of the most innovative stuff with a major search property that I see out there and I think that there’s definitely a good place for that kind of initiative. So let me wrap up by asking, if you had your way, in two years, what part would Ask be playing in the total search landscape?

Michael: We’d definitely have significantly more than 10% market share. My point of view, from dealing with the user experience, is that I’ve been proud of the work that we’ve done and I really think that we’ve been very focused and innovative with a very talented team here and we’re really hoping that as we look at the rest of the year and we put out Edison and the AskX experience, that we become recognized for taking chances and presenting the user experience in a differentiated way that people have to respond to us in the market and start adopting some of the things that we’re doing. Because of the amount of revenue that Microsoft, Yahoo and Google are dealing with on the search side, they often get a lot of press but our hope is really to take share and to hopefully have a user experience that inform and improve the user experience of our competitors.

Gord: Thank you for your time Michael.