Is Search in a Rut?

At the fall Ad:Tech in New York, there was an abundance of search on the session list. There were search sessions on selecting an agency, selecting search terms, understanding the search consumer (which I participated in), paid search campaign optimization, organic search, local search and even some performance art lampooning search. After the nearly total absence of search from the spring Ad:Tech show, it was a great rebound.

Or was it?

The sessions were hardly standing room only. In many, there were more empty seats than full ones. A sadly defining moment was when Piper Jaffray analyst Safa Rashtchy raised his expected revenue for search in 2007 by almost 10 billion dollars. This should be huge news. Those of us in the industry did a double take. But the announcement echoed hollowly through an almost empty room. Apparently, the marketing industry didn’t really care.

In a quick conversation with organizers of the show, they expressed frustration at the lack of interest in search. As an industry we had lobbied hard to have a presence at the New York show, and Ad:Tech accommodated. They filled the roster with 9 different sessions that touched on search. But other topic areas drew much bigger crowds.

You Can Dress Search Up, But You Can’t Take Her Out.

I hate to point this out, but search just isn’t sexy. It’s the marketing you have to do, not the marketing you want to do. For those of us that are passionate about search, this is hard to understand. “What do you mean you don’t find my girlfriend attractive? ”

Perhaps the problem is that we tend to be a little myopic in our view. We’re immersed in the day-to-day detail. We can actually get excited about algorithmic changes and new file types being indexed. This is hardly the stuff of great cocktail conversations. There are not many 20-something-year-old marketers who would use their knowledge of dynamic URL rewriting as a pick up tactic at the post-show networking party.

An Hour to Learn, a Lifetime to Master

The other problem is that search is like golf. It’s not that hard to do it halfway right and get results. Grasping the basic concepts is pretty easy, at least on the sponsored side. Most marketers think they’ve got search nailed and there’s nothing more to learn.

But like golf, there’s a whole other dimension to search that almost no one has scratched, and it’s incredibly hard to master. Very few companies have an effective search strategy that spans both organic and sponsored channels. Even fewer have tapped into what I call the missing 40 percent of search strategies, truly understanding the mindset of the search consumer and effectively starting a relationship with them.

The other frustration in trying to paint a more enticing picture of search is that the truly intriguing developments are happening in the research labs of Google, Microsoft, and Yahoo! It would be fascinating to get these engineers to open up about where they want to take search, but no one is talking. They can’t, because the balance of power in a multi-billion dollar turf war is at stake.

Looking From the Outside In?

Perhaps the answer for Ad:Tech is to look at search from the outside in, rather than the inside out. More and more people from outside search are looking in and trying to understand how they can leverage this extremely effective marketing channel. “We know people use search, so let’s figure out how we can use that to our advantage.”

I know of brand strategists, academics, market research consultants, and consumer behavior experts that are starting to pay attention to the search phenomenon. They’re bringing fresh insights and approaching search from a much more strategic perspective. This outside interest is also indicative of the disappearing distinctions between search and other online ad delivery channels.

As you know, the lack of strategic thought in search is one of my greatest frustrations with our industry. Quit talking about A/B testing, bid gaps, and link building! Tell me about how, when, and why my target consumer will use search. Tell me how to effectively capture that consumer in a prospect pipeline that begins to build a rich relationship. Take a step back and show me the big picture, and how search integrates with it. And tease me with some really insightful discussion about how search will become the functional engine of advertising delivery online, delivering just the right message to the right person at the right time.

I’m not sure what the answer is, but I do know that the marketing industry is a long way from knowing everything there is to know about search. I would hate to see search be conspicuously absent from the spring Ad:Tech line up.

Google Does It Right, and Still Makes Money

We’ve done three studies now on how people react with search engines, and one finding that has been consistent across all three studies still amazes me.

All other things being equal, Google dominates as the search engine of choice. In the latest survey, Google was picked as the favorite engine by a walloping 82.9 percent of respondents. Nobody else even came close. Yahoo! limped into second place with a scarce 7.5 percent of the vote.

So, why does Google dominate to this extent? Is it the best search engine? Well, it’s good, but arguably there are others at least as good. The brand has certainly become a household word, but Yahoo! was there first, and look how their market share has dropped.

Google’s success with users comes primarily from an unwavering respect for those users. Google has always put the user experience front and center, and this commitment to has bought them fierce loyalty.

Sometimes the Bottom Line Isn’t Always About Money

Search engines make money when people click on the sponsored listings. Except for some paid inclusion revenue, minimal compared to the paid placement streams, search engines make no money from organic listings.

Following that line of reasoning, it seems to make sense to encourage as many people to click on sponsored listings as possible. Every click puts more money in the bank account.

This line of reasoning has been used at most of the other engines. MSN, Yahoo!, Ask Jeeves and others have all tried to maximize the amount of sponsored real estate on their results page. Ask Jeeves won’t usually show an organic listing without the user having to scroll.

The engines love to see more click-throughs on sponsored listings. The advertisers love it. So why not? Everybody wins, right?

Not the user. We go to search engines for organic results. Sure, we’ll click on a sponsored listing about 30 percent of the time if it appears relevant, but that’s not why we go to the search engine. We go for the unpaid listings. If we can’t find them on one engine, we’ll go to another where we can.

Tale of Two Engines

Google gets this. They have the lowest sponsored click-through rates of any of the major engines. While sponsored listings have been given prominent placement, the organic listings always have center stage. They’ve never tried to trick us with vague labeling of sponsored results or tried to push organic off the page by increasing the number of sponsored ads on top.

Short-term, that strategy would certainly boost Google’s revenues, but what would happen in the long term? Well, perhaps we have to look no further than Ask Jeeves. They’ve played games with their organic rankings, trying to boost their sponsored revenue. And if you look at click-through percentages as recorded in our surveys, it’s worked.

About half the click-throughs on Ask Jeeves occur on sponsored listings. Google only captures 23.3 percent click-through on sponsored ads. But 82.9 percent of our 1,500 respondents said they use Google and only o.6 percent said they use Ask Jeeves.

When you slice the pie that way, you’re going to end up with a lot more revenue, no matter what your click-through percentage is.

The User Giveth and the User Taketh Away

Online loyalty is a fickle thing. Google has never forgotten where it came from. It emerged from nowhere and gained leading market share because it focused on one thing: the user. Then, it found ways to create revenue from that model without jeopardizing the delicate balance between profit and the user experience.

Google has realized that user traffic is a privilege, and you can never lose sight of that fact. If you treat them right, you will be rewarded. Fail to do so, and you’ll end up with a very small slice of the pie.

Search Innovation: Looking for the Average Joe

First published Oct 27, 2004 in Mediapost’s Search Insider

Google’s beta release of their desktop search tool was their shot across the bow of the USS Microsoft Search. Following hard on the heels of promising technology releases from Blinkx and Copernic, Google is staking their claim to the desktop search space.

And Microsoft seems to have been caught flat-footed, as they continue to push back the deadline for the release of Longhorn, which will integrate desktop search with the operating system. Many seem to think a search related announcement out of Redmond, Wash. is imminent.

So, if one looks at what’s come out of the major search engine labs lately, you’ll see a rush of new technologies centered on the ideas of desktop search, local search, indexing of rich media, and personalized search. It seems that everything we’ve been talking about in the past three years is suddenly coming on the market in one fell swoop.

This has prompted a number of analysts to start asking where search is going. I’ve been firmly seated on that particular bandwagon, adding my own prognostications to the many that are out there. But sometime last week, I slapped my forehead and pronounced myself an idiot (beating several others to the punch).

It really isn’t the release of technology that will dictate where search is going. It’s the public’s acceptance of that technology. And I’m not speaking about a few techno geeks huddled in the cool blue glow of a LCD flat screen. I’m talking about the masses.

Innovation Only Makes a Difference When It’s Accepted

There is a dilemma that is inherent in technical innovation. As the innovators, we tend to get caught up in the possibilities of technology and base our business decisions on it. But the fact is that the success of technology only takes place with widespread adoption by the general public.

In the high tech business, we tend to be surrounded with others like ourselves. We are the classic early adopters, looking for the latest technological gizmo. We tweak our computers and other various electronic gadgets, spend hours tracking down problems with drivers, and happily put up with bug after bug to gain an edge over the less technically savvy.

In our biz, we all tend to be members of this relatively small segment of the real world. Unfortunately, we sometimes make decisions that seem valid because everyone we talk to agrees with us. It’s not until the public flatly rejects our innovations that we realize it was never capable of going beyond the research lab.

So, if you ask me, or any of the others who have been writing about this, which of the emerging search innovations will make the difference in the industry, you might be asking the wrong person. I’m a geek. Go ask your Aunt Mildred, but be prepared to spend some time explaining what you’re talking about.

Public Adoption Can’t be Rushed…

As early adopters, we work on a radically different time line than the general public. We tend to leap into new technology before it’s fully tested. We move in a matter of weeks or months to try the latest new thing. The rest of the world takes years. So as we try out local search, desktop search, and personalized search, remember that the latest developments will probably take a long time to trickle down to the average Joe… unless you leave no options!

This opens up a rather interesting advantage for Microsoft in the search game. Because of their domination of so many parts of our interaction with our computers, they can force adoption of a new technology to an extent no one else can. Every other player, including Google, has to convince us that their technological innovations are worth using. Microsoft can leave us with no choice. So Google will continue to roll things out of their search lab, and we will eagerly install the latest beta.

Someday (probably soon) a major development will come out of Redmond about Microsoft Search. We “in the know” will rush to pronounce it a failure, or success, but the judgment isn’t really ours to make. It’s the millions of people who have no idea that Google now has a desktop search tool, or that Microsoft is integrating search into their operating system, that will ultimately make the difference. And they will only bestow that success when they’re good and ready.

The winner of search will be the one who is the shrewdest about controlling that timeline.

Do We Want a Smarter Search Engine?

First published in Mediapost’s Search Insider – September 28, 2004

Search engines are knocking on the door of the future. The 800-pound gorillas of search and brash new upstarts like Blinkx and Gurunet are working to find a way to make search more intuitive, ubiquitous, and intelligent.

Someday, search will be interwoven with everything we do and consequentially, we won’t lift a finger. Links to the sites we’re looking for will suddenly appear in a discrete search pane or a pop-up window. It’s inevitable.

Or is it? Is this what we as search users want? Do we want to hand control over to an omniscient, eerily cheerful search assistant that knows far too much about our user patterns and personal tastes?

Friendly Search Engine Seeking Single Middle Aged Woman

The personalization of search has been bandied about as the next big step forward in the industry. We need (so we are told) search engines that get to know us and what we like. Blinkx is already heading down this road, and Microsoft has a team of engineers busy in their Cambridge, England lab working on making their brand of search more attuned to the user and the job currently at hand. If you look at the type of people Google hires, you’ll find the following interest areas featured prominently: profiling, artificial intelligence, and machine learning.

But to make these search enhancements work to their full potential, we crawl onto the very slippery slope of user profiling. As the search engine observes us, it can create a profile of our interests, usage patterns, and other very personal information. It can then use this information to make its search results more relevant to the individual user.

Marketing Gold, Privacy Dynamite

The question is what happens to this profile? Obviously, each profile is worth its weight in gold to advertisers. With information, they can target their message to exactly the right audience. But even if search technologies offer this capability, the vast majority of providers have resisted the temptation.

The Google Toolbar has been around for four years, and if you install the advanced version, the End User License Agreement says that the toolbar will send information about every URL you visit back to Google.

While this is for the relatively benign purpose of retrieving that site’s page rank and other information, the fact is that Google has the ability to track each and every Web page you visit. At least Google is up front about letting you know. Other search tool bars and spyware apps aren’t quite so ethical.

Suranga Chandratillake, the CTO of Blinkx, indicated that they had the ability to incorporate a fairly basic level of artificial intelligence into the product to make the search suggestions more relevant to the individual user but had chosen not to. “Early users indicated they didn’t like this idea. Privacy is a big concern for them.”

It appears that search providers and contextual advertising providers are setting wheels in motion that will bring an inevitable head on collision between the wishes of advertisers hoping for better targeted vehicles and users concerned about their privacy.

At this point, the more ethical advertisers are standing on the right side of a gray and sketchy line that’s determining what should be done with the mounds of information collected from users.

Either contextual advertising is being served based solely on the nature of the job at hand with no reference to a stored profile, or the information is just sitting there, collecting dust in a data warehouse on some gargantuan hard drive. But no one’s saying they’re throwing that data away. I get the feeling they’re all eyeing each other nervously, waiting for somebody to step over the line.

Search has the ability to become smarter and more helpful. But as anyone old enough to remember “2001: A Space Odyssey” can tell you, an all-knowing, irritatingly polite computer isn’t always mankind’s best friend.

David vs. Goliath… vs. Goliath: Blinkx and the Future of Search

First published Sept 14, 2004 in the Search Insider

While the elephants dance, there’s sometimes an opportunity for a mouse to change the world. And that may be just what’s happening in the world of search. In the aftermath of Google’s IPO and the continuous stream of search speculation from Microsoft, a small technology start up named Blinkx may have just changed the way we search.

With over a million downloads in under two months, Blinkx has been a hot topic on the download forums and blogs. Ironically, the big draw for Blinkx has more to do with finding the right piece of information on your desktop than with making the Web easier to search. No matter, the same technology powers both types of searches and introduces a paradigm shift for the entire industry. Almost all of Blinkx’s functionality sounds very similar to projects currently underway in Microsoft’s research labs. The difference is, Blinkx is out there and working right now, not two years from now.

I had a chance to chat with Suranga Chandratillake, one of the co founders of Blinkx and the CTO. It was fascinating.

Concepts rather than Words Blinkx takes the idea of search a step farther than the keyword bound interface used by Google and every other engine. Rather than asking the user to interpret what they’re looking for into a string of keywords and then launch a query, Blinkx examines the context of their current activity and tries to distill the desired concept from it.

For example, if you’re on a Web site about collecting 1970 Volkswagen Beetles, Blinkx will use all the relevant content on that page to create the key concept of what you might be looking for. It will then suggest matching sites based on the entire concept, rather than just two or three words. Chandratillake points out that this gives a much more accurate match, because you’re using the entire text of a page. If you want to restrict the text used to provide the match, just highlight the desired content and Blinkx will refine the query.

Search Suggestions in any Application The content used to create the concept doesn’t have to be a webpage. It could be an e-mail, a text document or even a spreadsheet. Once Blinkx is installed, it will take a screen shot of whatever you’re working on and quietly suggest both local files and web sites that seem to match the key concept of your file. The Blinkx tool bar loads icons on the top right of the toolbar and when it finds a match, the appropriate tab changes color. Blinkx has tabs for 7 different channels, including News, Products (shopping), Web, E-mails, Video Clips, Blogs and Local Files. The suggested links appear in a window when you roll over the tab.

Now Serving Suggestions Daily One of the interesting results that Blinkx found with the original users is a dramatic increase in the number of links chosen in a day. Traditionally, most users may turn to a search engine 5 to 10 times a day. This means most users might see 50 to 100 links. But when Blinkx is always there, suggesting relevant links, the number of links seen by the user rises dramatically. The average Blinkx user generally sees 200 to 250 links a day. As Chandratillake is quick to point out, this opens up some real potential in the area of search marketing. “When search is ubiquitous, users will look at it more. They don’t have to stop what they’re doing, go to the Web and search. We’re always there, making suggestions.”

Do the Elephants Care? Chandratillake is the first to admit Blinkx isn’t perfect. “There are still a lot of bits to figure out. The biggest hurdle for us right now is the size of our index, but that’s improving every day.” Small interface and usability issues aside, the question is how will Google, Yahoo!, and Microsoft respond? This is potentially the biggest innovation in search in the past eight years. And the fact that it came from a small start-up rather than a major player indicates that the frontier of search is still alive and well. Copernic reportedly has a similar search appliance coming out soon.

Blinkx will have to find its place, and search users will have to understand this paradigm shift. But I’m betting those two things are inevitable. As Google, Yahoo!, and Microsoft continue their dance, they’ll have to pay attention to this mouse, because it’s changing the tune of the entire search industry

The Growing Pains of Search

First published August 19, 2004 in the Search Insider

Ahh…our fledgling little industry is growing up.

The little search marketing companies are unfurling their new wings and peering over the edge of the nest. Suddenly, venture capital firms are looking at search marketing technology with stars in their eyes. It’s beginning to seem that a few people are going to make a lot of money in this industry. And we’ve even got internal politics. All signs of an industry that’s finally ready for a run at the main stage. All of the signs became more apparent at the recent Search Engine Strategies conference in San Jose this month. This event, for right or wrong, has become the regular group therapy session of search. It doesn’t seem to matter where it’s held, you’ll see all the familiar faces at Search Engine Strategies. “If it’s August, this must be San Jose!” We come, we bitch, we share…and then we go home, just to do it all again in a few months.

With each new show, the latest hot topic spreads from table to table at the networking lunches and cocktail parties. In Chicago (December) it was Google’s Florida Update. In New York (March) it was Yahoo’s unveiling of SiteMatch. And this time, for a change, it didn’t have anything to do with one of the big engines. This time, it was our own little industry and its new professional organization (SEMPO) that stirred up conversation along the hallways of the San Jose Conference Center.

It’s not easy to gain maturity. Lord knows it’s still a struggle for me. As search tries to claw its way onto Fortune 500 marketing budgets, the driving forces of the industry still have to figure out where it is they’re driving too. Is search still a cottage industry, a loose amalgam of hundreds of small shops, or is it starting to become big business?

Will search marketing be overseen by internal departments in the biggest companies and advertising agencies? Will the best of the little guys be gobbled up in the next few years, with the rest left to find a niche to survive in, or wither on the vine?

To make prognosticating even more difficult, we still have to wait for the technology to mature before we can see where the industry could go. Something like providing results targeted to geographic locations can have huge implications for us all.

The biggest thing I saw in San Jose was the beginning of a chasm developing in our industry. A handful of more sophisticated and forward thinking search marketers are starting to really explore what can be done in search. They’re thinking research and strategy, rather than linking tactics and meta tag optimization. They’ve refocused their vision to look at the large and emerging picture of search. In their wake, they’re leaving the more traditional firms behind. The firms are usually quite small and are still using tactics from 4 or 5 years ago.

One of the things that have frustrated many about our industry was the lack of differentiation between search marketing firms. Despite the huge spread in prices, we all pretty much said we offered the same things. The minute we started talking to a potential client, we started spouting works like organic optimization, link building, landing pages, bidding strategie, and ROI tracking. All of this is relatively unique to our industry and reflects an exclusively tactical approach.

In San Jose, I noticed a few search marketing companies starting to use a different vocabulary. Not new, different. It’s terminology that comes from marketing and is strategy based. We’re beginning to talk about customer profiling, identifying attitudes, the nature of the buying cycles, and the role of brand awareness. It’s a new way of speaking aimed at marketers, not Webmasters.

I believe years from now that the 2004 San Jose show will be a milestone in the industry. I think it will mark the beginning of a year that will dramatically alter the nature of the search marketing industry. We will grow up, and that will mean significant pain for many. Search will become much more sophisticated, and the price of entry to play the game may prove to be too expensive for many smaller providers. Alliances will form and total solutions will begin to emerge. Google and Yahoo! will have to address the huge amount of time and effort required to manage a large, sponsored search campaign. Real money will start to be invested and made.

And to think, one day I’ll be able to say I was there.