The Future of SEO in a Personalized Search Interface

This is a debate that seems to have legs. A few posts back, I came to the defense of SEO from the user’s perspective.

In catching up with a few articles and chatting with a few key people in the industry, I’ve got another perspective that I’d like to share.

First of all, Joe Laratro in last Friday’s SearchInsider debunked Three SEO Myths, one of which was “Natural Search is Dead”. In it, Joe correctly stated:

“Natural search engine optimization is still thriving. It is more difficult today than it was five or six years ago, but the core of search results are still free. Natural Search Engine Optimization being dead is a popular myth because of the standardization of methodology that is now used. Each of the major search engines has released guidelines for Webmasters that detail the dos and don’ts of Web site optimization. Since more of the online world is aware of successful optimization techniques, they do not work as well. In other words, there is more competition from knowledgeable optimizers armed with the same toolsets.”

Back to this in a minute. Also, in sorting through some old articles set aside, I ran across this one from Todd Friesen about Learning the New Rules of Search. It touched on the same topic, looking at what might happen when SEO’s no longer have access to the intelligence tools they use on their competitors, through some type of authentication requirement.

“A move like this, which would block our ability to do competitive research at that level, would be a setback, if not a crushing blow, to SEO as we know it. That got me thinking. Where would that leave all of us search engine optimizers? What research avenues would be left?”

Finally, in the last few weeks, I have talked to representatives from the usability teams at Google, Yahoo and Microsoft. I asked each what was the major challenge for search. The answer varied slightly, but it all went to providing better results, aligned to individual intent. (By the way, I’m currently working on a series of articles from these interviews..more about this to come later)

So, to go back to the orginal reason for the post. What is the future of SEO? Both Joe and Todd are looking at this assuming the current paradigm of one query, one page of 10 organic results holds. In fact, the entire SEO industry is hanging on this paradigm. Right now, link baiting/building, optimization, competitive intelligence and all the rest are aimed at securing a top spot in the organic results. But what happens when there no longer is a “top spot” because every result is personalized, based on your geographic origin, your past search history, your behavior or preferences you’ve shared with the engines. That’s where search is going, through a number of different initiatives, and if less transparency with access to tools would deal a “crushing blow to SEO” imagine what that would do.

Now, that doesn’t make organic any less important to the user. In fact, the increase in personal relevancy will make it more important than ever. So I still stand by my original thought that organic results, of some kind, will always be part of the results set presented. But from a tactical perspective, the disappearance of universal search results throws a King Kong sized monkey wrench in the SEO works. In Canada, we’re already dealing with this as Google experiments with re-ordering organic search results based on Geo-targeting of user IP’s. The same is true in the UK and other markets.

But how do you tactically deliver SEO services in this new environment? The word ranking ceases to have meaning. There will always be a hierarchy in the results, but it will be different for each person. The control of measuring progress by positions achieved will come to a crashing halt and with it, the SEO industry as we know it. If you thought SEO was a black box before, wait til you try it under these new rules.

Over 50% of CMOs aren’t looking for Big Agencies for Online

A new study has reaffirmed something I’m hearing more and more. Big agencies don’t get online.

Sapient, through Evalueserve, surveyed a number of CMO’s, and just over half of them believe that traditional, large ad agencies are “ill-suited to meet online marketing needs”. They believe that there’s too much invested in traditional models, and that agencies can’t think beyond these constraints.

The upshot? Fewer than 10% of those polled seek to partner with large agencies for online marketing. They instead look for partners with roots in technology, a high degree of creativity and traditional print expertise, or, even more common, to use multiple agencies.

It’s not that large agencies don’t have people capable of getting online. In many cases, they do. But they’re trapped in a rigid and bureaucratic structure that sucks the lifeblood out of the bold thinking and initiative essential for online. They spend more time fighting turf wars than they do providing value to clients, and it seems that the clients are getting tired of it.

Increasingly, large agencies are struggling to understand the shifting marketplace. They are fighting the idea of a participatory approach to branding, with a community of consumers at least as important in the process as the actual brand itself. They are far more comfortable with the more traditional, and much more profitable, command and controlled channel form of marketing that has been built over the last several decades. They’re struggling to win in a new game where they don’t know the rules, largely because they haven’t been written yet.

The big agencies are out there shopping right now. They’re looking to buy expertise needed. I wonder how successful this will be. It’s not just the expertise they’re lacking. It’s the environment needed to let their experts do their job. You can buy all the roses you want, but if you lock them in a dark basement, you’re not going to see much blooming.

Lee Odden’s Feed n Read List

In a brilliant example of win/win, Lee Odden has shared his OPML file for the 250 Must Read Blogs in the SEM space. Thanks for including me on the list Lee! As you may have noticed, I’m trying to be more diligent about posting this year (New Year’s Resolutions and all) so hopefully I can persuade you to remove the comment “posts about search marketing from time to time”.

But consider the win/win aspects of this:

  • Lee helps the community by sharing a great resource.
  • Those included on the list share the love by posting a link on their own blogs back to Lee’s post
  • Lee suddenly gets hundreds of links from the best blogs in the business
  • Lee’s authority goes up
  • Everybody wins!

For anyone looking at blog promotion, link baiting or just in how to align the online planets in your favor in general, this is a textbook example!

Most Shoppers Don’t “Shop Around,” at least Physically

A new study from the Grizzard Performance Group found that US Shoppers don’t have time to “shop around”, with 62% not bothering to compare prices at even two stores. However, they’re very open to saving money, right up to the time of purchase. It’s just that they don’t have the time.

This ties in with my previous post about real time inventory and e-shopping, currently being tested by a a few online services at malls and major chain stores. When we can quickly and conveniently check prices at a number of stores in our area through our handheld devices, trust me, shopping will change forever. And then, a whole new dimension of direct response marketing comes into play. Last minute pushes of discounts at the point of purchase, delivered through your mobile device. As the study by Grizzard indicates, consumers are very open to saving money on a comparable product, even if it wasn’t previously in your consideration set. So consider this. The shopping engine knows what you’re looking for, knows where you are, and knows what comparable products are in stock in the same store. The advertiser can purchase the right to push a message to you right at the point of purchase, offering you 15% off their product, or even offering an automated “match and beat” deal, where it automatically matches the price of whatever you’re buying, and takes a further 10% off. A store around the corner could do the same thing, making it worth your while to check out at least one more store. All these things could easily be handled by algorithms and pre-set pricing thresholds.

And what if we take the Priceline approach? You’re ready to buy, but before you do, you send an offer to stores in your area with what you’re willing to pay for a particular product. The store in question can then decide whether to accept your offer or not. It would be true consumer control. And the really ironic thing? It’s a whole bunch of sophisticated technology, but it brings us right back to old fashioned haggling over the price. Isn’t it fascinating that the more sophisticated the technology, the closer we get to how we used to shop a century ago?

The Sausage Manifesto: An Open Letter on Click Fraud

Jeffrey Rohrs has posted an open letter to the PPC Networks calling for a fresh approach to tackling the click fraud issue. He’s named it the Sausage Manifesto. In it, he outlines 11 things he’d like to see the networks do:

  1. Talk, Don’t Lecture
  2. Appreciate Our Unique Circumstances
  3. Invest in Proportion to the Problem
  4. Acknowledge that Tracking Alone Is Not the Answer
  5. Improve Click Quality Customer Service
  6. Build a Click Quality Education Resource Center
  7. Light a Fire Under the IAB
  8. Play Nice with Others
  9. Put Somebody in Jail
  10. Create a Click Fraud Perp Registry
  11. Put Your Data Where Your PR Is

I agree with all Jeffrey’s points, but to varying degrees. I think some of them are probably not completely fair to Google and Yahoo, but they do sum up the level of frustration with advertisers, so they have to be addressed and taken seriously by the engines. If we use Shuman Ghosemajumder and his team at Google for an example, I think they’re working on points 1, 3, 4 and 7.  But most advertisers aren’t aware of the extent of the effort and that, combined with a natural skepticism about any messaging coming from the Networks, who have so much at stake, are impeding much needed communication.

For that reason, I absolutely think it’s time to knock down the walls. I agree completely with points 5, 6, 8, 10 and 11, especially 11. The whole problem here is the lack of reliable data. We’re trying to peer over the walls. Google and Yahoo in particular have to be more forth coming. Nothing solves a problem faster than exposing it to the light of day. Information is the answer here, and getting the information into as many hands as possible. I appreciate Google’s efforts to police the problem, but this can’t be a siloed effort, it has to be a collaborative one. Having wrestled with the issue through SEMPO, I’ve seen first hand how access to data to even judge the scope of the problem can be a tremendous challenge.

Thanks to Jeff Rohr for crystallizing the thoughts of a lot of advertisers. Hopefully it acts as a catalyst to push forward solutions.

A Sign of Things to Come: eShopping at a Store near You

A small article in the Wall Street Journal (a subscription is needed to read the whole article) is a precursor of a big shake up to come. It’s something I’ve been predicting for sometime now, and while it will take awhile to gain traction, it will turn local retail upside down.

Three malls in California and one in Arizona have agreed to allow shoppers to check prices on actual inventory through text messages from their cell phones with a service called NearbyNow. According to their site, NearbyNow plans to add another 17 malls throughout the US to their network by April. Another service called Slifter is focusing on national chains like Best Buy, CompUSA and Foot Locker.

Here’s why this is revolutionary and why you’ll be hearing more.

  • For shopping, this represents discontinuous innovation. It’s a big win for the user, allowing them to shop smarter than ever before. Consumer demand will drive adoption of this new approach.
  • For retailers, this is scary as hell. By allowing their inventory to be captured realtime, they’re agreeing to be compared side by side with everyone else, including online retailers with no physical overhead to drive up prices. It completely levels the playing field.
  • As a number of technologies improve and converge, this will become substantially more useful and powerful. Mobile computing, GPS and search functionality will make this a must have for consumers.
  • It completely fuses the online and offline worlds, making the transition seamless.

This is one of those ideas you just know will take off, but there’s going to be some significant hurdles to overcome. These services are only as good as their success at signing up merchants. The more stores in the network, the more successful. If only a few are included, consumers will always wonder if there’s a better deal that isn’t part of the service, defeating the purpose. And a number of retailers will resist this trend til the bitter end. Ultimately, it will be consumer insistence that will force the laggards to join.

Another challenge will be the user interface. Right now, both services run on cell phones, meaning you have to deal with an awkward keypad and stripped down display. But this problem will rectify itself with advances in mobile technology.

In the world of shopping, this changes everything.

No Real Surprises in the Latest iCrossing Study

iCrossing released the results of a new study conducted by Harris Interactive just before the holidays. The study looked at the role of online in the CPG market. A media release outlines the key findings, including:

  • Consumers look for CPG’s online, with 39% of US adults confirming they’ve conducted a search for CPG’s.
  • Women do this more than man. Footwear and apparel lead the categories searched for.
  • Online CPG searches often result in offline sales. Much of this activity is looking for sales or special offers at traditional bricks and mortar retail locations.
  • Activity is spread pretty evenly over search engines, retailer websites and manufacturer’s sites. Shopping engines and consumer information sites have substantially less traffic.

There are a few notable take aways here that speak to the future use of online. Most CPG’s have been slow to move to online as a marketing channel. The more commoditized the product, the less the online research activity, or so traditional marketing wisdom has told us. Certainly, CPG’s have been very slow to enter the search arena, yet the iCrossing study tells us that there is a significant portion of the consumer population are turning online to research these every day purchases.

To be honest, I think the study is probably underreporting the frequency of this. At Enquiro, we’re steering away from self reported survey based vehicles as a sole vehicle to look at search behavior, because we find that people have trouble recalling how often they use search and what they use it for. It’s become second nature for us to turn to online, and that in turn usually means search. So in a survey like the iCrossing one, memory lapses usually mean overly conservative numbers.

Another notable trend that would influence the findings are the increasing spread of high speed internet access. The likelihood of this CPG online activity happening is directly related to how handy a computer with an internet connection is. The more ubiquitous access is, the more we’ll do a quick look up on everything. About the only purchases I make now that I don’t do some form of online research about are groceries. And as local search becomes more robust, that will probably change too.

I’ve been predicting another surge of advertising dollars migrating into search over the next year or two. As we understand more how universal online research truly is, and how a lot of major advertisers are completely missing this very important touchpoint, more budget will find it’s way into search. While there are no real surprises in the iCrossing study, it’s good that major advertisers are continually reminded that they’re missing a rather large boat.

Stepping into the Did It/Web Guerrilla/Searchengineland Fray

I came in this morning, and what did I find? Another tempest stirring up in the blogosphere! Danny Sullivan, Kevin Lee and Greg Boyser have all waded in, so what the hell, I’ll dive in too.

First, a little history. Did It President David Pasternack started the whole deal sometime ago when he took a swipe at SEO, calling for it’s imminent death. I’m not going to elaborate, but for those of you interested, here are links to the original article, and a follow up article.

Now, Kevin Lee from Did It has written a ClickZ column, adding some clarity, but also predicting organic results being pushed below the fold because sponsored ads are more relevant. I’m going to set aside for a moment the SEO spamming question that Kevin raises. Greg and Danny do a pretty passionate job of defending SEO.

I’d like to speak from another perspective, the search user. There are a couple things that should be considered here.

First of all, contrary to Kevin’s point, just paying for an ad doesn’t make it relevant. That’s because the vast majority of marketers don’t consider the intent of the search user. They assume that everyone is ready to buy right now. That assumption is at least 85% wrong. Go ahead, do a search for any popular consumer product. I’ll bet the ads you see are talking about lowest prices, free shipping, guarantees and other hot button items that are aimed at a purchaser. But study after study shows that search engines are used primarily for product research, not purchase. The problem is that marketers have a very biased set of metrics they use to measure return. They measure ROI based on purchase, so when they test, these types of ads tend to pull the numbers they’re looking for. But the metrics aren’t capturing the full story. The 85% of users that are researching are basically ignored. No value is assigned to them. Until PPC marketers figure this out, they’re not doing the user any favors.

Our research shows that a very interesting interaction takes place with the researcher versus the purchaser in that Golden Triangle real estate. Both users look at the top sponsored ads when they appear. They both look at the organic listings. Frankly, there’s not a lot of difference between the scan patterns. But it’s where they click that makes the difference. When they’re ready to buy, based on a recent eye tracking study, about 45% click on top sponsored, and about 55% clicked on the top 1 or 2 organic links. Almost a 50/50 split, FOR THOSE THAT ARE READY TO PURCHASE. But when we look at the other 85%, the ones doing research, EVERYONE OF THEM clicked on the organic link. And in the test, the same site appeared in both spots, so relevancy of the destination was equal. As long as users want organic links, organic optimization continues to be important.

Look, David Pasternack can ring the funeral bell for organic all he wants, but the fact is, it’s not his call. It’s the user’s. Yahoo has actually done exactly what he and Kevin are predicting. They’ve moved organic down the page, jamming more sponsored on the top. Based on Did It’s comments, this should be good for the user, right? It should be more relevant, pushing the “spam” down below the fold. Wrong. Google kicked Yahoo’s ass in user experience in our latest study by every metric we looked at. And they’re definitely winning in the big picture, including stock prices. The difference. About 14% of Yahoo’s screen real estate (at 1024 by 768 pixels) was reserved for top organic. 33% of Google’s real estate went for top organic. You want more proof? Ask, back in the Ask Jeeves days, pushed organic totally off the page, doing exactly what Kevin and David call for and filling the top with sponsored. Take a look at Ask now. Organic is back above the fold. Spend some time talking to Ask usability lead Michael Ferguson about how the absence of organic worked out for them.

And it’s not that sponsored links provide a bad experience. Our study proves Kevin somewhat right. Top sponsored links, for commercial queries, delivered the highest success rates. But those were in highly structured and commercially oriented scenarios. That doesn’t represent all searches. It’s not that we avoid sponsored links, but we do want a choice and we want relevance, ALIGNED TO OUR CURRENT INTENT. Google has recognized that to a much greater extent than their competitors, and they’re eating their lunch.

There’s a reason why 70% of users choose organic. We’ve done a number of studies over the past 3 years, and that number has remained fairly constant.  It can’t be because those results are filled with spam. I actually just chatted with Marissa Mayer at Google, and she continually emphasized the importance of organic on the page. It’s a cardinal rule there that at least one organic result will always appear at 800 by 600. It’s mandated by Larry and Sergey. And that’s because they know it’s important to the user. We want alternatives. And we will be the judge of relevancy. That’s why Google has stringent click through measures on their top sponsored ads. If they don’t get clicked, they don’t show. The top of the Golden Triangle is reserved for the most relevant results, period, and in more than 50% of the cases, those are organic (either through OneBox or traditional organic).

So we in this industry can debate sponsored versus organic. We can make predictions. We can post in blogs til the cows (or frogs) come home. But it’s not our call. It’s not even the engine’s call. It’s the user’s.

Google Pulls Back the Curtain on Quality Score – a Little

At the last few shows I’ve attended, an interesting theme emerged. Up to now, reverse engineering an algorithm was exclusively a preoccupation on the organic side. SEO’s would try to out wit and out guess Yahoo and Google’s black box. But with the introduction of quality score, that game suddenly moved to the sponsored side of the strategy table. Because the factors that went into the quality score weren’t disclosed, particularly by Google, it was a game of test and guess by advertisers. A lot of show attendees were expressing frustration that there wasn’t more transparency. Google has apparently heard the call, and yesterday issued a clarification.

Google’s advice?

  • Link to the page on your site that provides the most useful and accurate information about the product or service in your ad.
  • Ensure that your landing page is relevant to your keywords and your ad text.
  • Distinguish sponsored links from the rest of your site content.
  • Try to provide information without requiring users to register. Or, provide a preview of what users will get by registering.
  • In general, build pages that provide substantial and useful information to the end-user. If your ad does link to a page consisting of mostly ads or general search results (such as a directory or catalog page), provide additional information beyond what the user may have seen in your ad or on the page prior to clicking on your ad.
  • You should have unique content (should not be similar or nearly identical in appearance to another site). For more information, see our affiliate guidelines.

While a step forward, there’s still a lot hidden under the hood of this algorithm. Anytime you put algorithms in charge, it opens the door to reverse engineering, and you can bet the SEM community is going to launch a barrage of tests to try to determine the nuances that determine the quality of a landing page in the eyes of the quality score algorithm.

What this does do, however, is increase the complexity of the quality score substantially. There are now three seperate components, including user click through, ad quality and landing page quality. Each addition exponentially increases the complexity of the algorithm, making it a lot tougher to game. It harkens back to the original introduction of the Google PageRank algorithm, which went beyond on-the-page factors to introduce the whole concept of authority within the structure of the Web.

How important is the quality score? It’s vital. Moving up the ranks on the sponsored side is at least as important as on the algorithmic side, and if you can make the leap from the right rail to the top sponsored ads, you can expect a 3 to 10X increase in visibility and click throughs.

Our recent eye tracking study showed just how important relevancy is in these top spots. And Google has always been very aware of that importance. They have an obsession about providing relevancy above the fold, especially in the Golden Triangle, that is not matched by any of the other engines. I actually had a chance to chat with Marissa Mayer about this. The interview will be part of the Eye Tracking study (currently available, by the way, and you’ll get a free final version with Marissa’s interview when it’s available) but I’ll be including some tidbits in this blog as well.

Interview with Shuman Ghosemajumder about Click Fraud

Had a chance to chat with Shuman Ghosemajumder regarding click fraud. Shuman is Google’s point person on the click fraud issue. This follows up on the post Andy Beal made on MarketingPilgrim earlier this week. Most of what we chatted about was in my Search Insider column this week. However, not all of it made it into the column, as there is a cut off which I routinely ignore (thanks to MediaPost editor Phyllis Fine for keeping me in line).

Here’s some tidbits that didn’t make it into the column:

First of all, I wanted to take the media to task for crying the sky is falling around this issue. I know that’s what journalists do, but the portrayal of the click fraud issue has been very one sided to this point. That’s why I wrote the column. I think it’s important we get balancing viewpoints. In the absence of numbers universally regarded as accurate, one has to poll the extremes and guess that the true answer lies somewhere in the middle. Up to this point, all we’ve heard are the negative estimates, and these are based on some studies with methodolgy that’s questionable at best (i.e. the Outsell study)

Secondly, I believe it’s unfair that everyone seems to be taking aim at Google, and to a lesser extent, Yahoo on this issue. I know they’re easy targets, because the targets are so damned big, but when the real numbers finally do come out, I’d bet my 89 Mazda 626 (the car that just won’t die!) that it’s the 2nd and 3rd tier networks that are the hotbeds of click fraud.

I dealt with it briefly in the column, but one of the main sources of misrepresentation seems to be this question of what click fraud is. For me, the definition is pretty simple, fraudulent clicks that leave the advertiser financially impacted. But when it comes to most of the media portrayals, there are a number of clicks that get lumped together under the label “click fraud”, the majority of which don’t meet this definition. And Google’s point of contention with reports of click fraud that come from the media and various 3rd party fraud detection tools comes from this aggregation of questionable numbers. There’s no distinction made between actual fraud, the clicks that cost the advertiser, and attempted fraud, the ones that got caught. And often more benign clicks, i.e. multiple legitimate clicks coming from the same IP address, get mistakenly labelled as click fraud.

Another positive move by Google was the inclusion of invalid clicks in the advertiser’s reporting dashboard. Every move that Google makes towards greater transparency is a very positive one. And the best know Google evangelist for communication, Matt Cutts, indicated so on a blog post. By the way, Shuman also has a blog, where he goes into greater depth on this issue.

I can only imagine how frustrating this must be for Shuman and the Google Click Fraud team. They sit and listen to numbers be bandied about in the 15% plus range, knowing from first hand experience that the real number is likely much much lower (in the column, using assumptions that are probably on the high side, the actual amount of click fraud that an advertiser would have to challenge Google on is less than 0.18%). Yet, their tongues are tied, both by Google’s legal and corporate communications department.

Why is the media targeting click fraud and trying to scare the hell out of advertisers? In no other industry I can think of are reporters more prone to mix and match numbers without regard for accuracy. They do it, and get away with it, because there are no independent and reliable numbers to look at.

There are a number of reasons. Google is in the vanguard of disruptive change agents that are shaking the very ground of marketing. It’s somewhat defensive to look for an Achilles heel, and right now, click fraud seems to fit the bill. Google in particular is boldly stating they want to change everything. That scares people.

Part of it is that there is still a lot of people that would love to see Google be knocked down a few pegs. Much as we rever success, wildly successful companies or individuals generate jealousy and suspicion. Our society gets a nasty little thrill when the mighty fall.

But perhaps the biggest reason is the very strength of search and online marketing: it’s accountability. Nothing else is as measurable. So when something appears to be eating away at the cost effectiveness, we tend to go all forensic on it and analyze the hell out of it. Could you imagine the mainsteam press making a big deal out of a .18% hole in the accountability in television advertising, or radio, or print? Even a 10 to 15% hole? Of course not, because much bigger holes than that are accepted every day as being inherent in the channel. But search and online ad networks are apparently fair game.

Is click fraud happening? Absolutely. And if you switch the lens a bit, there are some sophisticated click fraud operations that are making a killing. In a response to my column, Chris Nielsen had this excellent observation:

The problem is not overt clicking on ads, competitors clicking on ads, or double-clicking on ads. The problem is with large-scale concerted efforts that are massive enough to to have enough variety of IP address, user agents, etc. and pose as “valid” user click activity.

Of course this activity varies some with the bid price of the clicks, but it’s really the old idea of stealing a penny from a million people. If anyone notices, who’s really going to care? The problem is that in some areas, there are hundreds or thousands of people stealing pennys, and it is noticible and it is a problem. The only real indication is the lack of bona fide conversions, and that’s hard to say for sure if it’s fraud or real factors with the marketing or web site.

But it comes down to which lens you look through. Do you look at those looking to profit from click fraud, some of them doing it very well? Or do you look at the scope of the problem over the big picture? The problem I have with the BusinessWeek report is that the reporting is trying to do both at the same time, and you can’t get a clear picture by doing so.

I just wanted to wrap up this post by mentioning some other initiatives on this front that Google is pursuing which didn’t make it into the original column. Obviously they’re working on proprietary techniques to filter out click fraud, but they’re also trying to attack the problem on an industry wide basis as well. They’re working with the IAB Click Measurement working group, in which SEMPO is also involved. And they’re calling for stringent and scientific independent auditing standards, so when we throw around terms like click fraud, we’re all dealing with a common reference framework. By the way, I also asked Shuman about impression fraud. We didn’t go into a lot of depth on the issue, but they feel they’re equally on top of that as well.