Monthly Archives: September 2004

Do We Want a Smarter Search Engine?

First published in Mediapost’s Search Insider – September 28, 2004

Search engines are knocking on the door of the future. The 800-pound gorillas of search and brash new upstarts like Blinkx and Gurunet are working to find a way to make search more intuitive, ubiquitous, and intelligent.

Someday, search will be interwoven with everything we do and consequentially, we won’t lift a finger. Links to the sites we’re looking for will suddenly appear in a discrete search pane or a pop-up window. It’s inevitable.

Or is it? Is this what we as search users want? Do we want to hand control over to an omniscient, eerily cheerful search assistant that knows far too much about our user patterns and personal tastes?

Friendly Search Engine Seeking Single Middle Aged Woman

The personalization of search has been bandied about as the next big step forward in the industry. We need (so we are told) search engines that get to know us and what we like. Blinkx is already heading down this road, and Microsoft has a team of engineers busy in their Cambridge, England lab working on making their brand of search more attuned to the user and the job currently at hand. If you look at the type of people Google hires, you’ll find the following interest areas featured prominently: profiling, artificial intelligence, and machine learning.

But to make these search enhancements work to their full potential, we crawl onto the very slippery slope of user profiling. As the search engine observes us, it can create a profile of our interests, usage patterns, and other very personal information. It can then use this information to make its search results more relevant to the individual user.

Marketing Gold, Privacy Dynamite

The question is what happens to this profile? Obviously, each profile is worth its weight in gold to advertisers. With information, they can target their message to exactly the right audience. But even if search technologies offer this capability, the vast majority of providers have resisted the temptation.

The Google Toolbar has been around for four years, and if you install the advanced version, the End User License Agreement says that the toolbar will send information about every URL you visit back to Google.

While this is for the relatively benign purpose of retrieving that site’s page rank and other information, the fact is that Google has the ability to track each and every Web page you visit. At least Google is up front about letting you know. Other search tool bars and spyware apps aren’t quite so ethical.

Suranga Chandratillake, the CTO of Blinkx, indicated that they had the ability to incorporate a fairly basic level of artificial intelligence into the product to make the search suggestions more relevant to the individual user but had chosen not to. “Early users indicated they didn’t like this idea. Privacy is a big concern for them.”

It appears that search providers and contextual advertising providers are setting wheels in motion that will bring an inevitable head on collision between the wishes of advertisers hoping for better targeted vehicles and users concerned about their privacy.

At this point, the more ethical advertisers are standing on the right side of a gray and sketchy line that’s determining what should be done with the mounds of information collected from users.

Either contextual advertising is being served based solely on the nature of the job at hand with no reference to a stored profile, or the information is just sitting there, collecting dust in a data warehouse on some gargantuan hard drive. But no one’s saying they’re throwing that data away. I get the feeling they’re all eyeing each other nervously, waiting for somebody to step over the line.

Search has the ability to become smarter and more helpful. But as anyone old enough to remember “2001: A Space Odyssey” can tell you, an all-knowing, irritatingly polite computer isn’t always mankind’s best friend.

David vs. Goliath… vs. Goliath: Blinkx and the Future of Search

First published Sept 14, 2004 in the Search Insider

While the elephants dance, there’s sometimes an opportunity for a mouse to change the world. And that may be just what’s happening in the world of search. In the aftermath of Google’s IPO and the continuous stream of search speculation from Microsoft, a small technology start up named Blinkx may have just changed the way we search.

With over a million downloads in under two months, Blinkx has been a hot topic on the download forums and blogs. Ironically, the big draw for Blinkx has more to do with finding the right piece of information on your desktop than with making the Web easier to search. No matter, the same technology powers both types of searches and introduces a paradigm shift for the entire industry. Almost all of Blinkx’s functionality sounds very similar to projects currently underway in Microsoft’s research labs. The difference is, Blinkx is out there and working right now, not two years from now.

I had a chance to chat with Suranga Chandratillake, one of the co founders of Blinkx and the CTO. It was fascinating.

Concepts rather than Words Blinkx takes the idea of search a step farther than the keyword bound interface used by Google and every other engine. Rather than asking the user to interpret what they’re looking for into a string of keywords and then launch a query, Blinkx examines the context of their current activity and tries to distill the desired concept from it.

For example, if you’re on a Web site about collecting 1970 Volkswagen Beetles, Blinkx will use all the relevant content on that page to create the key concept of what you might be looking for. It will then suggest matching sites based on the entire concept, rather than just two or three words. Chandratillake points out that this gives a much more accurate match, because you’re using the entire text of a page. If you want to restrict the text used to provide the match, just highlight the desired content and Blinkx will refine the query.

Search Suggestions in any Application The content used to create the concept doesn’t have to be a webpage. It could be an e-mail, a text document or even a spreadsheet. Once Blinkx is installed, it will take a screen shot of whatever you’re working on and quietly suggest both local files and web sites that seem to match the key concept of your file. The Blinkx tool bar loads icons on the top right of the toolbar and when it finds a match, the appropriate tab changes color. Blinkx has tabs for 7 different channels, including News, Products (shopping), Web, E-mails, Video Clips, Blogs and Local Files. The suggested links appear in a window when you roll over the tab.

Now Serving Suggestions Daily One of the interesting results that Blinkx found with the original users is a dramatic increase in the number of links chosen in a day. Traditionally, most users may turn to a search engine 5 to 10 times a day. This means most users might see 50 to 100 links. But when Blinkx is always there, suggesting relevant links, the number of links seen by the user rises dramatically. The average Blinkx user generally sees 200 to 250 links a day. As Chandratillake is quick to point out, this opens up some real potential in the area of search marketing. “When search is ubiquitous, users will look at it more. They don’t have to stop what they’re doing, go to the Web and search. We’re always there, making suggestions.”

Do the Elephants Care? Chandratillake is the first to admit Blinkx isn’t perfect. “There are still a lot of bits to figure out. The biggest hurdle for us right now is the size of our index, but that’s improving every day.” Small interface and usability issues aside, the question is how will Google, Yahoo!, and Microsoft respond? This is potentially the biggest innovation in search in the past eight years. And the fact that it came from a small start-up rather than a major player indicates that the frontier of search is still alive and well. Copernic reportedly has a similar search appliance coming out soon.

Blinkx will have to find its place, and search users will have to understand this paradigm shift. But I’m betting those two things are inevitable. As Google, Yahoo!, and Microsoft continue their dance, they’ll have to pay attention to this mouse, because it’s changing the tune of the entire search industry