Author Archives: ghotchkiss

Universal Search and Other Surprises from Google’s Searchology

When Google yesterday invited a number of reporters to come down to Mountain View for an event they called Searchology, I figured they had something in the works. I had to turn down the invitation because of other commitments, but we sent Enquiro’s Director of Technology and analytics blogger, Manoj Jasra down in my stead. Sure enough, just after noon yesterday, I received a press release announcing the introduction of universal search. I haven’t had a chance to talk to Manoj about what else Google may have unveiled in Mountain View yesterday, but even just working my way through the official release from Google gave me plenty of food for thought. For the extensive list of the announcements and some running commentary, check out Danny’s post on Searchengineland.

To me, the one thing that jumps out in this is the announcement of Universal Search. Basically, Universal Search is the breaking down of the information silos that currently exist on Google and blending them into a single set of results. The changes right now are very subtle. Web results still dominate the typical results page and the primary thing that would be noticeable by the user are additional dynamically generated navigation links that sit just about the results.

universalsearch

The key to universal search results is an on-the-fly algorithm that looks across all of Google’s information sources and prioritizes and ranks all the items coming from these disparate sources based on the user intent. Now, it’s in those last five words, “based on the user intent” that the really important piece of this comes out. Just a few weeks ago, I interviewed Marissa Mayer about the inclusion of Web history in the dataset to calculate personalized search results. This just gives Sep Kamvar and his personalization algorithm a lot more to chew on as they determine user intent. During the interview, I asked Marissa Mayer if personalization allows Google to be more confident in delivering vertical results. Marissa indicated that this was not an area they were currently looking at.

There are a lot of different things that we could do with this data. I’ll be totally honest. Verticals isn’t something that has been first and foremost in our minds so I don’t really think there’s a strong vertical angle here at the moment.

To me it just didn’t make sense. Couple that with yesterday’s announcement of Universal search results and I’ve got to conclude that Marissa was throwing up a smokescreen.

Personalized search is the engine is going to drive universal search. The two are inextricably linked. When you look at the wording the Google throws around about the on-the-fly ranking of content from all the sources for Universal Search, that’s exactly the same the wording they use for the personalization algorithm. It operates on-the-fly, looks at the content in the Google index and re-ranks it according to be perceived intent of the user, based on search history, Web history and other signals. It’s not a huge stretch to extend that same real-time categorization of content across all of Google’s information silos. That is, in fact, what Google’s announcement yesterday said. Call it a silo, call it a vertical, the end result is the same. As Google gains more confidence in disambiguating user intent, more specific types of search results, extending beyond Web results, will get included on the results page and presented to the user.

This introduces something else that opens up some interesting implications for Google. And again, if they choose to go down this path, it flies in the face of something that Marissa Mayer has previously stated. On the search results page as we know it, display or other types of advertising just don’t work that well. The search results pages is heavily text-based. We look for text, we respond to text, we click on text. Anything that’s not text acts as an interruption and distraction. There’s no place on this page for display or rich media advertising.

But if you mix up the search results page and start including things like images, video clips, maps, icons for audio files, you move away from the common paradigm of the text based search results page. The Google page becomes much more like a personalized, on-the-fly portal based around the intent of our query. As such, it includes stimuli from a lot of different sources, presented in a lot of different ways. There will be many things fighting for your attention. And in this paradigm, perhaps display and rich media advertising works better. In another announcement from Google, Marissa Mayer appears to have backtracked and open the door for this.

Yesterday, Marissa responded to a question about possible inclusion of non text-based ads in this way:

Well we don’t have anything to announce on that today. I do think this opens the door for the introduction of richer media into the search results page. We are now going to understand how users interact with that. And as Alan always likes to say search is about finding the best answer, not just the best URL or the best textual snippet.  

For us ads are answers as well. Searching ads is just as hard as searching the Web, as searching images. And so I was hoping that we could bring some of these same advances in terms of the richness of media to ads.

Greg Sterling, in his post on Search Engine Land, calls it something of a bombshell (Greg, I’m now regreting that I didn’t attend, as I would have loved to chat to you about this) and I agree. This is a significant retraction of Google’s long running stand on keeping display ads off the SERP:

There will be no banner ads on the Google homepage or web search results pages. There will not be crazy, flashy, graphical doodads flying and popping up all over the Google site. Ever.

Google said in their announcements that the changes for the user will be subtle at first. In fact, the position of the dynamically generated navigation links that appear about the search results will largely be ignored by most users. They won’t even know they exist. But in typical Google fashion, this tentative presentation of new functionality will be an incremental one. The typical path that Google takes when introducing new functionality is

  • subtly introduce new navigation options in the way of links that tend to be out of the primary scan path
  • make it an opt in experience for the user
  • gradually roll this functionality into a default opt in
  • eventually integrate more fully into the standard presentation of results
  • move to full integration and remove the ability for the user to opt out

if Google goes down this path with both universal and personal search, you can expect to see a substantially different look for search results in the near future. And as with most things we’ve talked about that Google is looking at introducing, there will be a trade-off between overall functionality for most users and a relinquishing of control for a small number of users.

My final point for this post is the speed of which Google is introducing new search innovations. A few weeks ago I posted that Google may be treating search as the forgotten child, devoting more attention to the sexier new channels they were acquiring, including pretty much everything under the sun. Matt Cutts was quick to post a comment saying that Google was still very much involved with search and that there would be a number of new things rolling out in the near future. It appears that I didn’t know what the hell I was talking about and now have to eat my words, as the announcements over the last few weeks have indicated that Google is still very much in the search game and is moving forward at, what for them, is breakneck pace.

I’ve often stated before the Google was the victim of their own success. Because they have such a large slice search market, any changes to the actual presentation of the search pages came with a lot of risk. It’s a major monetization channel for them, their biggest one by far, and any changes in user experience through the introduction of new functionality comes with the potential of dramatically reducing click through on sponsored ads. I predicted that this would make it tough for Google to really innovate with search and we would probably be looking to the smaller players to aggressively pursue innovation. Interestingly, much of my recent conversation with Ask’s usability team lead, Michael Ferguson, revolved around this point. That interview will be running tomorrow on Search Engine Land, with full transcript posted to this blog. If you look at what Ask is been doing with AskX:

AskX

 It’s very similar to what Google says they will be doing with universal search results. It’s taking content from a number of different sources and rolling it into one combined search results page. It came as a complete surprise to me when I read the release indicating that Google is moving aggressively down the same path. Google will not be taking the path that Ask is, by aggressively presenting new functionality on their main site, Google will introduce it incrementally, bit by bit. But expect the evolution of the search experience on Google to move fairly quickly.

All of Google’s announcements in the last few months point in the same direction. They all point to a highly personalized, highly relevant portal to all of Google’s information. Here’s my other prediction. While Marissa was very careful in past interviews to state that personalization is currently impacting only the organic search results, with no work being done on the personalized presentation of sponsored content, I smell another smokescreen. Personalized presentation of advertising content is just too huge a revenue opportunity for Google and we’ll be seeing it in the very near future.

Interview with Ask’s Michael Ferguson

I recently had the opportunity to chat with one of my favorite usability people, Michael Ferguson at Ask.com. You can find excerpts of the interview, along with commentary, on Search Engine Land in this week’s Just Behave column. Some of Michael’s comments are particularly timely now, given Google’s announcement of Universal search.

Gord: How does Ask.com approach the search user experience and in big terms, what is your general philosophy?

Michael: A lot of what we do is, to some extent, informed by core search needs but also by our relevant market share, understanding that people have often experienced other engines before they come to us, not necessarily in that session but generally on the web. People have at least done a few searches on Google and Yahoo, so they have some context coming from those search experiences. So often, we’re taking what we’ve learned from best practices from competitors and others and then, on top of that, trying to add a lot of product experience and relevance experiences that are differentiated. Of course, we’re coming from this longer history of the company where we’ve had various user experiences over the time that we’ve been around. We’ve marketed around natural language, in the late 90’s and answered people’s questions at the top of the page, but in the last year and a half or so, we’ve rebranded and really focused on getting the word out to the end users that we are a keyword search engine, an everyday search engine.

A lot of the things that we’ve done with users have been to try to, implicitly, if not explicitly, inform users that are coming to the site you can use it very much like you can use any other kind of search engine you’ve been on before. Or, if they’re current users and people are coming back to the site, to let them know that the range of experiences and the type of information we bring back to them has greatly expanded. So that’s pretty much it. It’s informed by the context of not just a sense of pure search and information retrieval and all the research that’s gone on that in the last 35 or 40 years but also the dynamics of the experiences that we’ve had before and people’s previous experiences with Ask. Then, an acknowledgement that they’ve often searched on other sites and looked for information.

Gord: You brought up a number of topics that I’d like to touch on, each in sequence. You mentioned that in a lot of cases, they’re coming to Ask and they’ve used Google or Yahoo or they’ve used another engine as one of their primary search tools. Does Ask’s role as a supplemental engine or an alternative engine give you a little more latitude? You can add things from a functionality point of view to really differentiate yourselves. I actually just did a search and see that you, at least on my computer here, have made the move to incorporate some of the things that you were testing on AskX into the main site. Maybe we’ll start there. Is that an ongoing test? Am I just part of a beta test on that or this rollover complete now?

Michael: We’re still in testing with that and it will roll out. We have decided because of a lot of the user experience metrics that we’re getting from the beta test that we’re going to go for it. We have decided to move the full experience over to the AskX experience. Of course, there are variants to that, but the basic theme of, in a smart way, bringing together results from different search verticals and wrapping those around the core organic results (as well as) a sponsored experience. So that will happen sometime this year. We don’t know exactly when, but just a couple of days ago, we really decided we’ve seen enough and we’re pretty excited about that.

Google has a really great user experience going, and Yahoo does too, but they have so many different levers that move so much revenue and traffic and experience metrics that I think it’s harder for them to take chances and to move things around and get buy-offs at a bureaucratic level. To some extent, we see ourselves as having permission and a responsibility to really innovate on the user experience. It’s definitely a good time for us because we have such great support from IAC and they’re very much invested in us improving the user experience and getting more traffic and getting frequency and taking market share and they’re ready to very much invest in that. So we don’t need to cram the page with sponsored links and things like that. It’s mostly a transitional time when we’re getting people to reconsider the brand and the search engine as a full keyword based, everyday search engine that has lots to offer. I’m talking to people all the time about Ask and there’s definitely still people that say, “Hey, last night, it came up with my buddies at the bar, this trivia question about the Los Angeles Lakers, 1966 to 1972 (and I went to Ask and asked a question)”. Then there are other people that see us as evolving beyond that but still really surprised that we haven’t had image search.  Now with AskX we’ll have preview search and there’s lots of other stuff coming along now. So yes, it’s a great place to be. I love working with it. There are so many things that, in an informed way, we can take chances on, relative to our competitors.

Gord: So does this mean that the main site becomes more of an active site? Are you being more upfront with the testing on Ask.com rather than on AskX.com?

Michael: Well, I think the general sense of what we’re going to do is that, at some point this year, the AskX experience will, at least at a wireframe level, become the default experience and, of course, we have a lot of next generation “after that” stuff queued up that we’re thinking about and we’re actively testing right now but not in any live sense.  So potentially, things will slide in behind the move of the full interface going out and then AskX will remain a sandbox for another instance of, hopefully, new and really useful and differentiated search experience coming after that. A general thing that we’re going to try to do, instead of having 15 or 18 different product managers and engineering teams working on all these different facets of information retrieval and services, we’re going to stay search focused and just have one sandbox area where people go in and see multiple facets of what we’re thinking about.

Gord: Let’s talk about the sponsored ads for a bit. I notice that for a couple of searches that I’ve done while we’ve been talking that they’ve definitely been dialed down as far as the presence of sponsored on the page. I’m only seeing top sponsored appear, so you’re using the right rail to add additional search value or information value, whether it be suggested searches or on a local search, where it brought me back the current weather and time. So what’s the current strategy on Ask as far as presentation of sponsored results and the amount of real estate devoted to them?

Michael: Just to fit along with the logic of Eye Tracking II (Enquiro’s second eye tracking study), those ads are not a delineated part of the user experience for the end user and they’re relevance and their frequency can color the perception of the rest of the page and especially the organic listings below them. Right now, as I said, we’re very much focusing on improved user experience and building frequency and retention of customers, which all the companies are, I’m sure. But we’re really being, basically, cautious with the ads and getting them there when they’re appropriate and, as best we can, adjust them over time, so that when they’re there, they’re going to valuable for the user and for the vendor.

Gord: That’s a fairly significant evolution in thinking about what the results page looks like from say, two years ago, with Ask. Is that purely a function of IAC knowing that this is a long term game and it begins with market share and after that comes the monetization opportunities?

Michael: Actually, I think way before we got acquired by IAC we knew that. We test like other engines would. We test lots of different ad configurations and presentations and things like that but definitely you want to balance that. Way before we got acquired we realized that there’s one thing that’s kind of fun about making the quarter and blowing through it a little bit and then there’s another thing about eroding customers. And definitely there’s a lifetime value that can be gained by giving people what you know is a better user experience over time, so once we did become part of the IAC family, we brought them up to speed with the results that we were finding that were pointing to taking that road and they’ve very much been in support of it. And, of course, their revenue is spread amongst a lot of different pieces of online and offline business so their ability to absorb it is probably more flexible than ours was as a stand alone company.

Gord: That brings me to my next question, which is, with all the different properties that IAC has and their deep penetration into some of the vertical areas, you had talked about the opportunity to bring some of that value to the search results page. What are we looking at as far as that goes? Are we going to see more and more information pulled from other IAC into the main AskX interface?

Michael: Maybe the most powerful thing about the internet is that you as an individual now have a very empowered position relative to other producers of information, other businesses where you can consume a bunch of different points of view. You have a bunch of different opportunities to do business and get the lowest price and read reviews that the company itself hasn’t sanctioned, or anything like that.  You have access to your peer network and to your social networks. Search, like the internet, becomes, and it necessarily needs to be, a proxy for that neutral, unbiased view of all the information that’s available. This probably gets a little bit into what may or not may work with something like Google’s search history. Users over time have said again and again, “Don’t hide anything from me or don’t over think what you may think I might want. Give me all of the best stuff, use your algorithms to rank all that, but if I get the sense that anything’s biased or people are paying for this, then I’m not going to trust you and I’m going to go somewhere else where I can get that sense of empowerment again.”

As I’ve sat in user experience research over time, I’ve seen people..and I know this isn’t true of Google and I know it isn’t true of Ask right now with the  retraction from paid inclusion…but you ask users why they think this came up first on Google, maybe with a navigational query like Honda or Honda Civic and Honda comes up first. They’ll say, “Oh, Honda paid for that.” So even with the engines that aren’t doing paid inclusion, there’s still this kind of wariness that consumers have of just generally somebody on the internet, somewhere, behind the curtains, trying to take advantage of them or steer them in some way. So as soon as we got acquired by IAC, we have made it very much part of their perception of this and their culture. Their product management point of view is that you can’t sacrifice that neutrality. You can’t load a bunch of IAC stuff all over the place. The relationship with IAC does give us access to proprietary databases that we can do lots of deep dives in and get lots of rich information out  that can help the user in their instance of their search needs that other companies wouldn’t be able to get access to, while maintaining access to everything else.

The way we approached AskCity was a great example of this. We had leveraged a lot of CitySearch data but at the same time, we know that when people go out and want to see reviews, they want to see reviews from AOL Neighborhoods, they want to see reviews from Yelp they want to see reviews from all these other points of view too. So we go and scrape all those and fold them into the CitySearch stuff. We give access to all those results that come up on AskCity. If they’re, for instance, at a restaurant, you can get Open Table reviews and you can get movie reservations through Fandango and other stuff like that. Those companies have nothing to do with IAC. Those decisions were borne from user needs and from us looking as individuals in particular urban areas, and saying “Hey, what would I want to come up?” We know from previous experience from AOL that the walled garden thing doesn’t work. It’s just not what people expect from search and not what they expect from the internet, so that lesson’s been learned. I don’t know how much it would be different if we had some dominant market share over search, but that’s even more reason for us to be appealing to as wide a population as possible. That’s my philosophy right now.

Gord: I guess the other thing that every major engine is struggling with right now is in this quest to disambiguate intent, where is the trade-off with user control? Like you said, just show me a lot of the best stuff and I’ll decide where I want to drill down and I’ll change the query based on what I’m seeing to filter down to what I want. In talking to Marissa at Google and their moves towards personalization and introducing web history, I  think for anyone who understands how search engines work, it’s not that hard to see the benefits of personalization but from a user perspective there does seem to be some significant push back against that. Some users are saying, “I don’t want a lot of things happening in the background that are not transparent to me. I want to stay in control.” How is Ask approaching that?

Michael: The other major thing that’s going on right now is that we have fully revamped how we’re taking this. We developed the Direct Hit late 90’s technology. And then the Teoma technology we acquired. And really, it’s not that we’re taking those to the next level, we got all of that stuff together and over the past three years, we’ve been saying, “Okay, what do we have and what’s unique and differentiated?” There’s a lot of great user behavior data that Direct Hit understands.  We have a whole variety of things there and that’s unlocked, that’s across all the people coming in and out over time but not any personally identifiable type of stuff. And then there’s Teoma, which is good at seeing communities on the web, expertise within the communities and how communities relate. So right now, even though we have personalization stuff and My Stuff and other things that are coming up, we’re investing a lot more in the next version of the algorithm and the infrastructure for us to grow called Edison. And we started talking about that a week ago since A.G. (Apostolos Gerasoulis) mentioned it. Across a lot of user data it understands a lot about the context from the user intention side and because we’re constantly capturing the topology of the web and it’s communities and how they’re related, we then match the intention and the map of the web as it stands and the  blogosphere as it stands and other domains as they stand. Our Zoom product, which is now on the left under the search box in the AskX experience and it’s on the right on the live site, is the big area that we’re going to more passively offer people different paths.

For example, just like with AskX, you search for U2, it’s going to bring up news, and product results, and video results and images, and a Smart Answer at the top of the page. It’s also going to know that there’s U2 as the entity, the music band and therefore search the blogosphere but just search within music blogs. So what it’s doing, over time, is trying to give a personalized experience that’s informed by lots of behavior and trying to capture the structure of the web, basically. So that’s where we are there.

There’s a book that came out in early 1999 called Net Worth, which you might want to read. I almost want to revisit it myself now. It’s a Harvard Business School book that Marc Singer and John Hagel came out with. It talked about infomediaries and it imagined this future where there’d be these trusted brands and companies. They were thinking along the lines of American Express or some other concurrent banking entity at the time, but these infomediaries would have outside vendors come to them and they would entrust all their information, as much as they wanted to, they could control that, both online and offline.  You were talking in your latest blog post about understanding in the consideration phase where somebody is and presenting, potentially, websites that they hadn’t seen yet or ones that they might like at that point in the car purchase behavior. But the way that they were imagining it was that there would be a credit card that might show that someone had been taking trips from the San Francisco Bay area to the Tahoe region at a certain time of year and had maybe met with real estate agents up there and things like that. But these infomediaries, on top of not just web history but even offline stuff, would be a broker for all that information and there would be this nice marketplace where someone could come and say, “I want to pay $250 to talk to this person right now with this specific message”. So it seems that Google is doing a lot of that, especially with the DoubleClick acquisition. But I’m just wondering about the other side of it, keeping the end user aware of and empowered over that information and where it’s at. So Net Worth is a neat book to check out because the way they were describing it, the end user, even to the broker, would seep out exactly what they wanted to seep out at any given time. It wouldn’t be this passive recording device thing that’s silently taping. My experience so far of using the Google Toolbar that’s allowing the collection of history, is that it’s ambiguous to me about how much of my behavior is getting taken up by that system and used. We’ll see where it goes but right now we don’t have strong plans to do anything with that for search.

Gord: It’s going to be really interesting because, up to now, the tool bar was collecting data but there was no transparency into what it was collecting, and now that they’ve done that, we’ll see what the user response is to that. Now that they can go into their web history and have that initial shock of realizing how much Google actually does know about them.

One other question, and this is kind of a sidelight, but it’s always something that I’ve been interested in. Now that you have the search box along the left side there and it gives search suggestions as you’re typing, have you done any tracking to see how that’s altered your query logs? Have you noticed any trends in people searching differently now that you’re suggesting possible searches to them as they’re typing?

Michael: There are two broad things that are encouraging to us. One is that over time, the natural language queries are down tremendously. Our queries, because we promoted in the late nineties this “ask a question” thing, tended to be longer and more phrase based, more natural language based.  That’s really gone down and is approaching what we would consider normal for an every day search engine profile as far as the queries. And we really think that this zooming stuff has really helped that because it’s often keyword based. You will sometimes see some natural language stuff in there. There are communities on the web that are informing us that there’s an interest in this topic that’s related to the basic topic so it is helping change the user behavior on Ask.

And the other result of that is as people use it more for everyday keyword based search engine, the topics or the different categories of queries that people see are normalizing out too. Less and less they’re reference type stuff and more and more they’re transactional type queries, so that’s a good thing. And that’s just been happening as we rebranded and we presented Zoom.

And then with the AskX experience, we are definitely seeing that even more because of the fact that they’re just in proximity to the search box. We always knew that these suggestions should ideally be close to the search box so that people understand fully what we’re trying to offer them. For instance, on the current site, we do see users that will sometimes type a query in the search box on top and because they’re used to seeing ads on the right rail on so many other sites and because they don’t necessarily know what narrow and expand your search is they think those are just titles to other results or websites. It’s a relatively small portion. Most people get what it is, but there was that liability there. Now in the AskX experience, it’s close and visually grouped with the search box. It’s definitely getting used more and guiding queries and people seem even more comfortable putting general terms in. We’ve made it that you can just arrow down to the one and hit return. It’s definitely driving the queries differently.

Gord: I’ve always liked what you guys have done on the search page. I think it’s some of the most innovative stuff with a major search property that I see out there and I think that there’s definitely a good place for that kind of initiative. So let me wrap up by asking, if you had your way, in two years, what part would Ask be playing in the total search landscape?

Michael: We’d definitely have significantly more than 10% market share. My point of view, from dealing with the user experience, is that I’ve been proud of the work that we’ve done and I really think that we’ve been very focused and innovative with a very talented team here and we’re really hoping that as we look at the rest of the year and we put out Edison and the AskX experience, that we become recognized for taking chances and presenting the user experience in a differentiated way that people have to respond to us in the market and start adopting some of the things that we’re doing. Because of the amount of revenue that Microsoft, Yahoo and Google are dealing with on the search side, they often get a lot of press but our hope is really to take share and to hopefully have a user experience that inform and improve the user experience of our competitors.

Gord: Thank you for your time Michael.

Google’s Perfect Marketplace

In my recent conversation with Michael Ferguson, he brought up the book Net Worth and the concept of infomediaries. I hadn’t read the book (an oversight I’m correcting) but I did a little quick online research. First, here was Michael’s comments:

There’s a book that came out in early 1999 called Net Worth, which you might want to read. I almost want to revisit it myself now. It’s a Harvard Business School book that Marc Singer and John Hagel came out with. It talked about infomediaries and it imagined this future where there’d be these trusted brands and companies. They were thinking along the lines of American Express or some other concurrent banking entity at the time, but these infomediaries would have outside vendors come to them and they would entrust all their information, as much as they wanted to, they could control that, both online and offline.  You were talking in your latest blog post about understanding in the consideration phase where somebody is and presenting, potentially, websites that they hadn’t seen yet or ones that they might like at that point in the car purchase behavior. But the way that they were imagining it was that there would be a credit card that might show that someone had been taking trips from the San Francisco Bay area to the Tahoe region at a certain time of year and had maybe met with real estate agents up there and things like that. But these infomediaries, on top of not just web history but even offline stuff, would be a broker for all that information and there would be this nice marketplace where someone could come and say, “I want to pay $250 to talk to this person right now with this specific message”. So it seems that Google is doing a lot of that, especially with the DoubleClick acquisition. But I’m just wondering about the other side of it, keeping the end user aware of and empowered over that information and where it’s at. So Net Worth is a neat book to check out because the way they were describing it, the end user, even to the broker, would seep out exactly what they wanted to seep out at any given time. It wouldn’t be this passive recording device thing that’s silently taping. My experience so far of using the Google Toolbar that’s allowing the collection of history, is that it’s ambiguous to me about how much of my behavior is getting taken up by that system and used.

So, as Michael says, Google seems to be positioning themselves to be this infomediary. Think about the nexus that’s forming between personalization and Google’s acquisition of every available marketing channel. Google is creating the perfect customer acquisition marketplace. And what’s their typical pricing model? Yes, auction based pricing.

So let’s walk down this path a little. Let’s assume that Google is successful in pushing a high degree of personalization on a significant portion of the population. If you capture all the search history and web history, you have a great data set to predict, with a high degree of accuracy, a consumer’s needs at any given time. The math behind this is not that intimidating for the brain trust that Google has assembled.

Then, let’s factor in Semantic Web functionality. Now, through a series of useful apps, Google takes that personalization data and further adds user value by letting them interact with information. It’s Google’s recent announcement of Universal Search, taken to a new and much more functional level. They’ve already warned us that Universal Search is just the beginning. Google powers the web as our personal assistant, so that for any given life or consumer event, Google is determining our intent, either implicitly or explicitly, and providing us with commercial recommendations. In this case, it’s not really advertising, it’s a helpful recommendation.

Finally, through the Google web of properties, both online and offline, you have the opportunity to present these “commercial recommendations” through a number of reinforced touchpoints. The odds of connecting with an engagement consumer and eliciting the desired conversion are almost 100%.

It’s a perfect marketplace, the ideal match between a prospect and a solution.

So now you have the perfect marketplace, complete with a Google console that lets you target the consumer you want in the way you want. Let’s add one more piece of the puzzle, the pricing model. Auction based pricing has worked pretty well for Google in the past. Why should this be any different. There will of course be a quality scoring component to this. Google is way too obsessive about user experience to just open the bidding to anyone. But let’s say that the Google quality scoring mechanism goes deeper than it does right now, determining exactly the best vendor fits with the determined need and intent of the consumer. Let’s say that Google narrows the list down to the top 10, and then from their database of potential advertisers, who have all indicated what they’re willing to pay for an almost guaranteed customer with an already predetermined ROI (remember, we know with a high degree of accuracy what it is that the prospect is likely to buy), they present the advertiser (or perhaps a few options, as we all like to see options) with the combination of the highest bid price and the highest degree of consumer intent relevancy. Once the bid is accepted, a packaged and personalized message goes out to the prospect through the appropriate channels.

Think for a moment what this does to the entire world of advertising. Hmm…some pretty hefty food for thought.

Shedding Some Light on B2B Purchasing

First published May 17, 2007 in Mediapost’s Search Insider

This week, we released our latest B-to-B research study based on a survey of almost 1,100 respondents. Today, I wanted to share a few high-level findings with you.

The Mirrored Worlds of Online and Offline

One of the challenges in B-to-B marketing is that you’re not marketing to just one person; you’re marketing to an organization. So you’re marketing to different people within that organization at different times. This adds a significant amount of complexity to business-to-business marketing. We wanted to capture this aspect of the B-to-B buying process, so we grouped respondents into four different categories of buyers: user buyers, technical buyers, coach buyers and economic buyers, the one who actually sign the check.

Another thing we wanted to look at was the impact of both online and off-line influencers in the purchase decision. How important was visiting a Web site, compared to seeing a vendor at a trade show or an ad in a trade publication?

In the study, one thing became clear. Online influences have gained a tremendous amount of ground over traditional influences. In fact, they’ve even caught up with the traditional off-line winner, word-of-mouth. The vendor’s own Web site was listed as the most important influence, together with word-of-mouth from a colleague or peer. Close behind were search engines, distributor Web sites and word-of-mouth from a friend.

When B-to-B buyers enter the purchase cycle, online activity is a natural result of off-line brand awareness. As the buyer becomes aware of a potential product or solution, the first reaction is to turn online to find out more about it. Across all phases of the buying cycle, including awareness, research, negotiation and purchase, over 85% of respondents said they will go online to help them make the right purchasing decision. This online activity was highest during the awareness and research phase, with a full 92% of respondents indicating that they would turn to online resources then. The percentage was lowest during negotiation, but even so, two out of every three respondents indicated that they would go online during this phase.

The Search Intersection

Also, the vast majority of purchasers start their online journey at the search engine. Although this varies by phase of the buying process, over all phases one in two users turn to a search engine first to help them find the online resources they’re looking for. This is highest during the beginning of the purchase process, in the awareness and research phases. At the awareness phase, 65% of respondents indicated the first place they would go would be a search engine.

There’s also a distinct evolution in the use of search engines as buyers move through the purchase process. Near the beginning, the first places they turn are the major portals, and the overwhelming favorite is Google, the first choice of 77% of the respondents. By the way, in a simulated search we incorporated into the survey, 74.2% of the clicks happened on organic listings. This matches up quite well with the organic/sponsored breakdown we’ve seen in other studies.

But as buyers begin moving through the phases, the role of the vertical B-to-B search engine (such as Business.com or Knowledgestorm) becomes more important. Buyers use these engines to build their consideration set and dig deep for the information about the product or solution alternatives they’re considering. While only 7.3% of respondents indicated they would turn first to the B-to-B vertical engine in the awareness phase, 22.1% indicated this would be their first choice during the negotiation stage.

K.I.S.S. Works for B-to-B, Too

The biggest influence for the B-to-B buyer? The vendor’s own Web site. But when it comes to accessing information on that site, simpler is definitely better. Buyers said they were looking for clear, extensive product information provided in an easily transferable, text-based format. The No. 1 priority was clear pricing information. This was followed closely by extensive product information, comparisons with competitors and downloadable papers and product sheets. The least important factors to the buyer were things like podcasts, webinars and online chat functions. B-to-B buyers are very task-oriented; they want to get in, find the information they’re looking for and get out. They have little patience for linear multimedia presentations that force them to gather information on the vendor’s timeline, not their own.

B-to-B purchases are often complex, long-cycle affairs that generate a tremendous amount of online activity. The wonderful thing for the marketer is that much of that activity funnels through a search engine at some point. This gives the marketer that understands this process a tremendous advantage, because it’s easier to determine the most traveled intersections online. But that understanding is the key. I hope research like this adds to our rather limited body of data on B-to-B purchases.

Personalization: Google’s Defensible Trump Card?

A thought that came up in a conversation with Michael Ferguson, Ask’s usability guy (which is probably why I like talking to him. He always greases the mental machinery) was Google’s defensible position that personalization offers.

Google is betting the farm on personalization. And really, they’re possibly the only search engine that can make this work. Here are the required components:

  • A high enough degree of additional user value to convince people to opt in to personalization. As I’ve talked about before, that’s why it’s being rolled out in organic search first. Expect a slew of other value adds in the near future, all powered by personalization and all aimed and getting you to hit the opt in box.
  • An extensive network so you can maintain multiple touch points for the delivery of targeted advertising. Nobody has a bigger network that Google’s AdSense network
  • Critical mass amongst users. With Google’s almost 65% market share and the highest penetration of installed tool bars (42% plus in a recent B to B study we did), Google also has the required components to tap into a significant slice of the available market. And future Gadgets and tools will likely either require personalization to be turned on, or will provide an enhanced level of functionality when they are. Expect Google to get aggressive with forcing adoption in the next year or so.

It came to light when I was talking to Michael about Ask’s algo and whether personalization will play a part (by the way, this is part of an interview that will be on Search Engine Land next week). After the interview, I realized it’s not an option for Ask, at least not at the level that Google’s contemplating. Even if they did move to personalization, they just don’t own enough of the total online user experience to push them to opt into personalization. They’d never gain the critical mass needed to make it work.

Microsoft has an outside chance through Messenger, but it would be a long shot. Yahoo also has a long shot at it (although better than Microsoft’s) but they’d have to start gaining market share, and there are a number of huge obstacles in their way. Google is by far the best bet to force personalization on the market and have it be adopted at significant rates.

So what are the options for the other engines? Well, again, there’s an interesting twist there as well. One thing that’s touted heavily by the contenders is social search. I have severe doubts about the scalability of anything that requires a human element, and I’ve written about this in the past. But then I realized that personalization gives Google social search in a way that others just can’t touch.

If Google is collecting both web and search history, they’re collecting implicit votes for the quality of every property on the web. They create their own community, and with every click, that community votes for the quality and relevance of every site they visit. It’s social search in a very powerful and completely transparent form. In this form, social search requires no additional action on the part of the user (one of the critical risk areas of social search) and is completely scalable, because there’s no human bottleneck (the other critical risk area).

The more I think about personalization, the more I think that Google has just trumped the entire search space…again.

The Three Cs of Search

First published May 10, 2007 in Mediapost’s Search Insider

Since most of the Search Insiders are in Bonita Springs this week, chances are that you’ll be hearing a lot of what’s happening down here in the Florida Everglades (other than the brush fires which appear to have us surrounded). Aaron Goldman shared his Buzz-o-meter with us on Tuesday, where he measures the words that seem to be dropped with the greatest frequency. It appears that my opening remarks set a tone that has been picked up in a number of sessions, and two words breaking into the top 10 are “connection” and “community.” Aaron added a third “c”: “content.”

To me, these words sum up a transition that’s happening in search. Expect the activity of searching on a search engine to gradually disappear, to be replaced with the functionality of search as an underpinning to the workings of many things on the Web. Search will become the engine that drives the semantic web, which Esther Dyson talked about in her keynote. She’s looking for search to move beyond “search and fetch” to her ideal, “deliver, act and transact.”

Search will be the connector between what we want and what best matches our want out there on the Web. And rather than a singular task (i.e. go look for this query) it will become a self-guided series of tasks, with intelligent agents in between to set search on its new direction. An entire trip, include flight reservations, hotel bookings, ground transportation, notifications of friends in the area and restaurant reservations, could be booked by intelligent Web agents, powered by search. And as came up in a panel discussion with the Search Insiders, when the presentation of commercial messaging appears in this context, it’s not advertising, it’s a helpful recommendation.

The piece that drives this is personalization, and that’s why Google’s moves are potentially so important. They take us much closer to the semantic web that Dyson envisions. This is the first “c”: connections.

Redefining Community

The second “c” speaks to the very transformation of our society: community. The way we relate to each other is being totally rewired by the Internet. By sheer physical necessity, communities have previously been defined by geography. We shared a common space, which enable communication, which created community. But today, the Internet has made physical distance irrelevant. Our communities are now defined by commonly held ideas or interests. Communities form around ideas, and search connects us to those communities. Every time we do online research for a product or service, we step into a community. In the course of a day, we can belong to several different communities. They are constantly shifting, as people move in and out of them, depending on the longevity of the engagement with the idea that forms the community.

Content Trails

And a third “c,” content, is the trail that the other members of that community leave behind through their conversations. These are the telltale signs that someone has already gone this way, and left a permanent record of his or her engagement with the community. Every Wikipedia entry is part of a community, as are many MySpace pages, blog posts and other virtual outposts. Search is the thread that loops them together at the user’s initiative. In fact, the algorithm of the engine is the de facto definer of community with each given search. The engine goes out, defines the landscape of community, and connects you with the citizens of that community and the content trails they leave behind.

It’s a fascinating world, which is being born as we speak. It’s a sociological experiment of vast magnitude in the making, and I don’t think we know what the repercussions will be. Whatever they are, it’s too late to turn back now. Technology moves fast, but people move slowly, and not in one mass. Small degrees of technological change can create seismic shifts in the sociological landscape. And we’re subjecting ourselves to a degree of technological change unparalleled in history. Who knows what we’ve unleashed?

 

Blog Overload

Hello Gord..where are you?

Well, to be honest, I don’t even know. I was in NY last week for SEMPO’s planning retreat (great board this year, by the way..look for good things) and right now I’m in Florida (which is literally burning up around me) for the Search Insider Summit. So, my blogging has been sporadic on this site, but I have been doing some posts on the Search Insider Summit Blog, along with David Berkowitz, Aaron Goldman and Lee Odden. Try to catch it…

I’ve got a back log (or would that be back-blog?) of stuff and I’m actually in the office for more than a few days (then it’s off to China and I promised a few people I’d try to do a bit of a travelogue, as I did when I visited Europe last summer) so hopefully I’ll be doing some catch up.

By the way, another reason for the sporadic blogging is that I’ve been up to my ears in new research. We’ve got the first release of new B to B data coming out very soon, we’ve done a Chinese Eye Tracking study (and that promises to have some very interesting data) that I’ll be unveiling in Xiamen, and we’ve got a few other things up our sleeves. So I apologize, but it’s not like I’ve been lounging around the pool or anything…honest!

Thoughts on Yahoo and Microsoft Merging

Note: This was actually written on Friday, but I haven’t had a chance to post it til now. I’ve been travelling and access has been an issue. But I just came back from the opening reception at the MediaPost Search Insider Summit and the latest seems to be that the hype of this deal is far ahead of any actual discussions. That said, I think my comments are still valid, because as we’ve learned, things can happen fast in this industry.

Friday, May 4

The big news this morning as I was burning off some calories on the stair climber was the possible acquisition of Yahoo by Microsoft. I was actually in New York when I heard the story break, and one of my meetings today was at the Microsoft New York office, so I thought it would be interesting to ask my contact there what she thought. She indicated that this story has been going on for years now, but apparently they’re going back to the table. As we were chatting in a conference room, someone walked by outside asking somebody else if they had bought Yahoo stock. The media speculation was good news for Yahoo stock, not so for Microsoft.

Obviously, there’s a lot to mull over here. Rumor has it that Steve Ballmer is not taking Google’s DoubleClick scoop lightly. In fact, he’s downright pissed. And he may be preparing to make Terry Semel an offer he can’t refuse. Semel’s played hard to get before, but this time the shotgun marriage just might take.

The obvious question is how the two search properties will combine. In this case, it might be a case of two wrongs not making a right. Yahoo has managed to keep their search share from eroding too badly with Google’s domination, but Microsoft has been sputtering out of the starting gate from day one. The problem is that Yahoo and Live search duplicate each other in many ways, rather than complement each other. The biggest problem with both engines is too much focus on revenue generation and not enough on user experience. They each have their different flavors, but the combined Microhoo (or YahSoft) is in no way a Google killer. In fact, with the turmoil of a merger and the inevitable awkwardness of combining search teams, I see the focus on the user suffering even more. Both engines desperately need a clearly focused user champion to revamp the search experience (ala Google’s power usability troika, Larry, Sergey and Marissa) and this deal just doesn’t produce that.

I think the rationale of the deal has much less to do with search and more to do with a rather petulant online land grab. Yahoo does bring some interesting assets into the Microsoft fold. Microsoft is definitely eyeing the Asian market, and Yahoo has dominates in most of these markets, with the exception of China, and that’s a whole other story. Yahoo also brings a lot of users and online real estate as well, with roughly double Microsoft’s user base. This move looks like a strategy to bolster the front line for a head to head confrontation with Google in the ad serving space. Of course, it could just be the Ballmer has a lot of cash burning a hole in his pocket and everytime he goes to spend it, Google snatches the acquisition away from him. Steve wants to buy a ball he can actually take home.

One really interesting aspect of this is what it will do in the search space. While I really don’t think Yahoo’s search assets are the impetus for the deal, the potential combining of Live Search and Yahoo cleans up the search landscape a bit, and my guess is there will be significant user fall out from this. This will not be good news for the users of these two engines in the short term. But it could be extremely good news for Ask.

I just did an interview with Michael Ferguson, Ask’s usability point person (coming in Search Engine Land next week) and the IAC team are doing some really smart and relatively innovative things with their engine. And they’re probably the least aggressive in jamming ads on the page right now. Diller has provided a big enough bankroll to allow Jim Lanzone and his team to take a long run at capturing marketshare and this just may be the break they need. Based on what I’ve seen, Ask is paying a lot of attention to the user experience, and they may well pick up some converts and some pretty significant marketshare lift because of that. Perhaps Microsoft employees should be eyeing IAC stock. Or perhaps Steve Ballmer is starting to jot them down on his shopping list. After all, Google will probably scoop Yahoo out from underneath him at the last minute anyway!

Connecting the Dots with a Global Marketplace

First published May 3, 2007 in Mediapost’s Search Insider

Late last week I got to spend a couple of very enjoyable days in the desert heat of Tucson together with the sales team of ThomasNet.com. I was the guest speaker at their national sales conference. This week, likely as you read this, I’ll be in New York for the SEMPO Planning Retreat, and in another day or so, I’ll be on a plane to Florida for the Search Insider Summit. I get back for one week, briefly acclimatize myself and then it’s off to China for Search Engine Strategies. The point of rattling off my travel itinerary, other than gloating about the frequent flier miles I’m racking up? All this hopscotching around the globe can be tied together with one common theme. It was topic of my talk in Tucson. While preparing for it, I found some interesting details that speak of a groundswell of change that will impact every industry.

What Web Site? I Don’t Need No Stinking Web Site!

One of the challenges faced by ThomasNet, or for that matter, any online property that is targeting industrial manufacturers, is in convincing some of the advertisers of the need for establishing a Web presence. These are traditional and, very often, conservative businesses that have been around for decades, and they cast a jaundiced eye at anything too new, too trendy or anything that even vaguely smacks of “geekiness.” In many cases, they’ve been turning out steel widgets and doodads that have a very specific niche market. They know their customers, and their customers know them. So why would they need a Web site? Why would they need to advertise on a search engine? And why do they have to worry about a global marketplace? All the reasons can be summed up in two words: things change.

Agents of Change

In 1990, the travel industry was a relatively stable place. Travelers went to the local travel agents and the travel agents acted as the channel for the information from various airlines, cruise lines, hotel chains and vacation companies to the consumer. They served a vital part of the value chain in the industry. And with something as highly personalized and variable as travel, it was hard to imagine how these travel experts could ever be disintermediated.

Even when the Internet started to gain traction and the first online agencies popped up in the mid-’90s, travel agent’ place seemed relatively secure, because of many of the same reasons we currently hear from manufacturers: They knew their customers, their customers knew them and the exchange of information back and forth between the two parties proved the value of this relationship.

In 1995, the number of single-office travel agencies peaked at almost 22,000, according to the Airlines Reporting Corporation. And then things changed. The online travel agents upped the ante. They demystified travel and opened up control of information to anyone who had Internet access. Airlines and hotels readjusted their booking channels to be able to go first to online agencies, and ultimately, direct to savvy travelers. Online communities formed that allowed travelers to connect with others who’d been there, seen it and done it, getting firsthand advice of where to stay and how to get there. And by 2004, the number of single-office travel agencies had been cut in half. Less than 10 years and an industry was decimated. Things change quickly!

Look East for the Future

In 1999 Intel Chairman Andy Grove said, “In five years, all companies will be Internet companies, or they won’t be companies at all.” Grove may have been a touch optimistic in his timing (imagine, someone over-hyping the Internet in 1999), but I don’t believe that takes away from the importance of his message. One of the mistakes that travel agents made, and the mistake that many small manufacturers are making again, is to assume that just because they’re not interested in a global market, all other competitors are likewise uninterested in their market.

The balance of power in the manufacturing world is dramatically swinging eastward. Another sobering fact that I came up with in the preparation for my presentation was the fact that in the U.S., there are currently about 14 million jobs in manufacturing. In all G-7 countries combined (U.S., Canada, the UK, France, Italy, Germany and Japan), there are about 53 million manufacturing jobs. In China alone, there are almost 110 million jobs in manufacturing. A manufacturing powerhouse the likes of which we’ve never seen before is gearing up in Asia. And those Asian companies are desperately eager to learn how to use the Internet to connect with new markets right here, in our backyard. To add to what Andy Grove said, not only will all companies be Internet companies, we’ll also have to become global companies. At the very least, we’ll have to be acutely aware of our global competition.

And that brings me to the other destinations on my travel agenda. One of the things the SEMPO board will be discussing this Thursday in New York will be the driving trends in search. Globalization will be near the top of the agenda. Then, a few days later in Florida, at the Search Insider Summit, we’ll be gathering together in the Everglades to talk about emerging issues. Search’s expansion beyond its early consumer-based, direct-response successes into areas like manufacturing and other business-to-business verticals is almost sure to be discussed. Finally, I have to see for myself the economic explosion that’s happening in China. I was a little shameless in wrangling myself an invite to speak at Search Engine Strategies. But it seems that no matter where you go, one thing remains true. All roads lead online, and they all intersect with search at some point.

 

What’s Hot at the Search Insider Summit? Two Words: Sep Kamvar

First published April 26, 2007 in Mediapost’s Search Insider

I was fortunate enough to be asked to MC the Search Insider Summit in Bonita Springs, Fla. in a little over a week from now.  As the MC, I get to open each day with a few pithy comments and hopefully insightful observations about the emerging trends and notable events in the search engine space.  Let me give you, as a faithful reader of this column, the inside track on at least one of the names I’ll drop regularly. In fact, take a moment to go find yourself a pen to jot this name down, because it will become vitally important to you in the next year or two… Sep Kamvar.

Who, you ask?  As I was writing this I took a quick scan of the regular search marketing columns, including this one, to see how much ink Sep has received in the past week.  It’s a great injustice that when Kevin Federline launches his own search engine we all rush (and I use first-person plural intentionally, I know I wrote about it too) to add our insightful commentary to the buzz surrounded this relative nonevent.  But when perhaps the most important announcement to be made in the search space in years occurred last Thursday, it passed with nary a whisper.  A quick search on Google News showed that the only blogging about this announcement, other than Google’s official post, was a couple of blogs I did on my own site that got picked up in a few other places.  Danny Sullivan also wrote a fairly lengthy post on the announcement. But other than that, not a ripple on the normally turbulent waters of the Internet.

But Sep Kamvar could become one of the most important people at Google very quickly.  In fact, his name could become as well known as Larry and Sergey.  Last Thursday, Google announced that they were adding Web History to their search personalization algorithm.  Sep is the guy behind the algorithm.  I’ve been blogging and writing about personalization for the last few months, telling everyone that they have to pay attention to this.  But other than a handful of people that I’ve spoken to recently, I don’t think that most search marketers or users get how important this potentially is, not just for search but for online marketing in general.The lack of pickup on Google’s announcement is evidence of this.

Three weeks ago I wrote a column called “Google’s Gargantuan Footprint.”  A key piece of that puzzle was Google’s ability to move towards behavioral targeting, and I speculated on how that might happen.  I mentioned the Google Toolbar and its PageRank feature as one of the key elements.  Less than two weeks later I got an e-mail from one of my favorite PR people at Google, Katie Watson, letting me know that Marissa Mayer wanted to chat with me about the company’s plans for personalization.  Sep Kamvar would be joining her on the call.  I juggled my schedule so I could make that call, because I knew it was going to be important.  I was not disappointed.

Google is now offering an opt-in choice for users to include Web History (all the sites you’ve visited) as a data set that will power their search personalization.  Thinking into the near future, you can see that the implications of this are vast on several different levels. Being able to roll Web History into Search History and monitoring a user’s click stream to help refine search results is a huge step toward disambiguation that will substantially alter our individual search experience.

The question for users is: are they willing to make the trade-off necessary by providing all this clickstream data to Google with their consent?  The fact is, if you have PageRank enabled on your toolbar, this information is being sent to Google anyway.  But Google’s recent move toward opting into Web History increases the level of transparency into what information the company is gathering — and how it will be using that information to refine your search experience.

But it’s not the personalization of search results that makes this a sea change.  It’s the ability for Google to close the loop around one individual based on his online behavior — and use that to offer multiple advertising opportunities across their network.  For the interactive marketer, this represents targeting nirvana.  And if one considers Google’s recent acquisition of DoubleClick combined with its contextual network and the ever-spreading Web of touch points that Google now controls, my speculation about the gargantuan footprint that Google is leaving on the online landscape moves several steps closer to reality.

I simply cannot speak enough about how important this is to every search user and every search marketer out there.  At the user level, there will probably be very little in the way of noticeable change for the immediate future.  Google’s move was simply to give Sep and his team a nice clean opt-in database that they can play with to improve the personalization algorithm.  But as Sep and his team begin to refine personalization, expect it to be aggressively rolled into multiple aspects of your Google experience.  It’s the engine that will power the future of Google for the foreseeable future.  It will eventually surpass the PageRank algorithm in importance, giving Google the ability to match content to very specific and unique user intent on the fly.

And for that reason, Sep Kamvar is a name to pay attention to.