25th April 2008

Spam Monkeys

Google has penalized some sites for buying links and others for selling links. Personal blogs with no revenue stream have their rankings stripped while large brands carry on selling and buying links.

Why?

From the Navy Safety Center:

Start with a cage containing five monkeys. Inside the cage, hang a banana on a string and place a set of stairs under it. Before long, a monkey will go to the stairs and start to climb towards the banana. As soon as he touches the stairs, all of the other monkeys are sprayed with cold water. After a while, another monkey makes an attempt with the same result, and all the other monkeys are sprayed with cold water. Pretty soon the monkeys will try to prevent it.

Now, put away the cold water. Remove one monkey from the cage and replace it with a new one. The new monkey sees the banana and wants to climb the stairs. To his surprise and horror, all the other monkeys attack him. After another attempt and attack, he knows that if he tries to climb the stairs he will be attacked.

Next, remove another of the original five monkeys and replace it with a new one. The newcomer goes to the stairs and is attacked. The previous newcomer takes part in the punishment with enthusiasm! Likewise, replace a third original monkey with a new one, then a fourth, then the fifth.

Every time the newest monkey takes to the stairs, he is attacked. Most of the monkeys that are beating him have no idea why they were not permitted to climb the stairs or why they are participating in the beating of the newest monkey. After replacing all the original monkeys, none of the remaining monkeys have ever been sprayed with cold water. Nevertheless, no monkey ever again approaches the stairs to try for the banana.

Why not? Because as far as they know, that’s the way it’s always been done around there.

They don’t have to punish all the link buyers, not even the big ones, just get enough people talking about it and the rest will follow. If I were looking for monkeys I’d find the most vocal ones perhaps those involved in online forums, social media, and discussion groups. You wouldn’t want to waste your time going after trusted newspapers that sell links for $195 a year, specially when they offer:

Your search engine rankings will also improve by receiving a link on our sites!

While you’ll be less dependent on people having to search for your site, your search engine rankings will be improved for those that do.

“PageRank interprets a link from Page A (our sites) to Page B (your site) as a vote for Page B by Page A. PageRank then assesses a page’s importance by the number of votes it receives. PageRank also considers the importance of each page that casts a vote, as votes from some pages are considered to have greater value, thus giving the linked page greater value.” – Google Support site

“The best way to ensure that Google finds your site is to have pages on other relevant sites to link to yours.” – Google Support site

~Hat tip to Wingnut for the Monkey Quote, the link seller outing was my own doing.

If you liked this post please buy me a beer. Thanks.

posted in Google, Paid Links | 0 Comments

26th March 2008

Finally a penalty with some teeth for selling links

This just in from DaveN that he has an example and a confirmation from Google that they’ve actually done more than a visible PageRank reduction for a link seller.

It all started when Dave posted about a friend’s site that was no longer showing for it’s money keywords, in his words:

…because I know where they used to rank for t-shirt printing

The assumption was that they’ve done nothing spammy to deserve the penalty.

Matt Cutts stopped by to add:

“what do you do when you know you haven’t do anything wrong, but Google still gives you a penalty”

e-examine your assumptions? E.g. http://web.archive.org/web/20070817163057/http://www.indigoclothing.com/blog/ is a link that shows when they were selling links. Later versions of their site were selling even more links, e.g. “sexy underwear” links.

Google has been very clear about how we feel about selling/buying links. If indigoclothing.com has dropped their text link ads and remove the links that they sold, they could do a reconsideration request. According to the data I looked at, the site has never done a reconsideration request.

So now we can get on with the business of actually earning links now that Google is actually going to start reducing the rankings of sites selling links. The visible PageRank reduction caused quite a stir, but in the end all I kept reading about it was that peoples rankings and traffic didn’t change. Sort of a non-penalty.

A quick check of the web’s most notorious link seller shows them still indexed and still ranking. Google must be going after the small time TLA sites first before tackling the worst offenders. Google still recommends buying links from Yahoo! in their guidelines, but those are slow to update as the help for the Reconsideration Request still says it’s for deindexed sites:

If your site is not included in Google’s search results, and you believe that it does not violate our webmaster guidelines, you can ask Google to reconsider your site for inclusion in the index.

This is clearly not the case as shown by this very example.

If you liked this post please buy me a beer. Thanks.

posted in Paid Links, reconsideration request | 0 Comments

17th March 2008

MLSGB Penalty

I wante to patent the term MLSGB Penalty which is the My Link Scheme Got Busted Penalty..

The penalty presents itself as a sitewide deranking and no amount of on-page optimization or writing of reconsideration requests will fix it.

The problem is that your site which has built its reputation on crappy free directory listings, link exchanges, and automated text link purchases has been busted. Google has figured it out and systematically devalued the majority of your links. After all of those crap links are filtered what’s left is not much and the rankings you used to enjoy will not be found again until you can build your link profile back up to the level you were getting credit for, this time with real links. The MLSGB Penalty tends to be doled out in sectors by devaluing crap links in clusters.

Symptoms of this are:

  • Reconsideration requests go unanswered no matter if you’ve removed other offending schemes.
  • Sitewide PageRank drops
  • Same number of pages indexed but long-tail results are way down
  • Reduction in crawl rate
  • Ability to get new pages indexed is reduced
  • Not ranking for your own domain name
  • Interlinked domains suffer the same consequences
  • Seen in common sectors
  • Decrease is sudden, if not over night
  • Google’s site:, link:, and data in webmaster tools stays the same
  • No warning or letter from Google

Basically what we have here is that the site was ranking falsely before based on an improper link profile, now that the link profile has been updated the symptoms above appear. You can’t be reconsidered because in order to restore the rankings they’d have to give you credit for the links, which isn’t going to happen. The one key anecdotal piece evidence here is a drop in ranking for the domain name, which generally returns eventually. Google has indicated that a drop in ranking for a domain name is a sign of penalty, however if it returns eventually I believe it points to another cause. Most worthless link directories link to sites with the domain name as the anchor text, which is quite unnatural in today’s linking. It may have been all the rave back in 1996 when people actually did build link lists for humans but not anymore, even news services use keyword anchor text to help add to the story. When Google has figured out that a majority of your links came from directories and link exchanges and removes the credit, the domain ranking suffers because a key indicator, anchor text, has been removed. Eventually relevancy will win over and the site will rank again for the domain. Just like a newly launched site will skyrocket to the top with a few good links because Google wants to keep its index fresh with the latest trends, the same goes for the reverse. When a site looses a large percentage of links all of a sudden it follows that it should also be reduced in rank. These behaviors happen naturally in the wild, a new site splashes onto the scene and goes viral in days, or an old site shuts its doors or gets involved in a scandal causing people to no longer link to it. Aggressive false link building mimics that natural quick link growth actual popular sites enjoy catapulting it to the top, but even more aggressive filtering by Google also mimics a drop in popularity.

How can you recover? Surely not through the same methods that got the site into this mess to begin with. More aggressive link building in crap directories and link exchanges on ‘links.html’ pages surely won’t help and may even aggravate the situation with another round of deranking. Reconsideration Requests will go unanswered as there is nothing to reconsider just a low linked site ranking where it should. The only answer is to build links naturally at a pace the site deserves and if the content is so poor that it won’t get links no matter who you show it to, it may be time to start over.

I have no insight whether or not this is manual or automated but I tend to think its manual as the quickness of the onset of the penalty suggests. An automated method would slowly remove such links as they are found, whereas a manual review of a sites link profile would tend to be quick and a one-time event. I think this penalty may have other aberrations such as “going supplemental”, “the minus (insert number of the day here) penalty”, or even some of the recent uproar over paid links.

I don’t want to give the impression that I believe the bad links to the site are actually harming anything, just that they used to count for something and no longer do. So before you go out and sign up your competitor for a million bestlittlewebsitedirectoryintheworld.com links remember that they may enjoy that unnatural bump for a while, and with the extra traffic actually get a few real links.

If you liked this post please buy me a beer. Thanks.

posted in Google, SEO | 1 Comment

12th February 2008

Nightmare Scenerio: future of search

Search Logo

If you liked this post please buy me a beer. Thanks.

posted in SEO | 2 Comments

25th January 2008

Google’s “Scalable” Solution

I’m no stranger to Google’s reconsideration request. I’ve helped dozens if not hundreds of people scour their sites, identify possible violations, implement changes, and compose the reconsideration request. I don’t do this as a professional cause but as an extension of my efforts in helping webmasters in Google’s Webmaster Help Group. Perhaps its because I choose the sites I want to work with and only cater to the ones that I believe are acting in ignorance rather than more devious intentions, but my success rate is quite high. There’s never been a case I couldn’t solve, then again this is probably due to my selective choices and not my mad Google skills. Either way, I know of what I speak.

Which brings me to an interesting situation that I was alerted of in twitter, saw in Sphinn, and then saw unfold on Dazzlin Donna’s take on SEO news, tips and theories SEO Scoop Blog. If you take the time to read Donna’s post you’ll see that she was caught up in the paid links dragnet and lost some of her visible PageRank. After a while she decided to demonetize her blog and set it up to comply with Google’s guidelines regarding paid links. She’s not Yahoo! so her time and opinion in choosing which sites to review are not worthy of being compensated for if they contain an active link (Google’s opinion, not mine).  After cleaning up the site she submitted a reconsideration request to Google.  Time passed and yet her PageRank penalty persisted.  Five weeks passed and she has finally found some resolution, though not through Google’s reconsideration request, but through the only solution that will actually work.

From my outsiders point of view and without any inside knowledge, the situation unfolded like this.

  1. Sometime in late December a reconsideration request was filed.
  2. Five weeks passed…
  3. Donna posts her plight to her blog
  4. A twitter is sent out.
  5. The post is Sphunn.
  6. 20 people sphunn it.
  7. The Sphinn goes hot 2 hours later.
  8. Matt Cutts comments on her blog, scolding her for her non-scalable method of approaching the situation, but offers to help.
  9. Matt offers to look into another commenter’s site.
  10. Matt says that her disclosure policy could be the problem.
  11. Donna changes her policy and responds that she did so.
  12. Matt emails the Google employee charged with reviewing Donna’s request.  Apparently there is another post that is still passing PageRank that was paid for.
  13. Donna fixes the post and comments that she did so.
  14. Matt points out another violation.
  15. Donna fixes that violation.
  16. Matt praises his team and says that they will get to it soon.

I would not have thought of how obtuse this whole process was had it not been for Matt saying, “In general you want to go with the reconsideration request approach rather than invoking me (that’s not scalable :)”  [my emphasis] Obviously this process is not scalable at all.  Here we have someone who’s worked on fixing her site, made some substantial changes, submitted a request for review, and apparently missed some things.  What she missed was exactly the same problem that she already admitted guilt to in the reconsideration request, but rather than offering any help Google files the request in the circular file and ignores the problem.

Since the majority of site owners don’t know Matt Cutts, know how to use social sites to get attention to their blog, don’t have blogs for that matter, and if they did probably wouldn’t get Matt to write six comments on their blog and send an email on their behalf, this is not a scalable solution.

A scalable solution would be the following:

  1. Site owner fixes site and submits a reconsideration request.
  2. Google reviews the site and finds some outstanding violations.
  3. Google sends a message back in the site owners  webmaster’s tools message center saying, “We have received and reviewed your request for consideration.  Unfortunately at this time we are unable to act on your request due to continued possible violations of our Webmaster Guidelines.  Please feel free to review the Webmaster’s Guidelines, make any changes that you find appropriate and resubmit your reconsideration request”
  4. Site owner digs deeper and sends in request.
  5. Google responds with another note, “We have received and reviewed your request for consideration.  It appears that your site is now within our guidelines.”

Notice that I didn’t even say that Google had to specifically say what violation they had.  I didn’t even specify whether or not a penalty has ever existed or has been lifted.  What I did do is “COMMUNICATE“.   Letting the site owner at least know that they are being heard.  Google’s response can be an automated one with only two possibilities. I’m sure their is a radio button somewhere on a computer somewhere that a Google employee is clicking when they review a reconsideration request.  It wouldn’t be too much to program one of two auto-responses depending on the status of that button.  That would be a scalable solution.

Their communication efforts in the help groups and their webmasters blog have been quite admirable lately, but there still is a disjoint between your average webmasters and those who know how to get to Matt Cutts, and that is just not right.  Not right at all.  I’ve heard many people say and write that one thing you should look for on an SEO’s resume is whether or not they know any search engine engineers, this situation just adds  that, and that is just not right.  Not right at all.

Having Matt Cutts be the voice of Google out there writing on his own blog,  commenting on people’s sites, and occasionally penning something on the official webmaster’s blog is great and wonderful for the community that watches that sort of thing.  I just believe that those people are a small subset of the actual webmaster population and the majority should not be at a disadvantage because they don’t subscribe to the right feeds.

If you liked this post please buy me a beer. Thanks.

posted in Google, Matt Cutts, Paid Links, reconsideration request | 12 Comments

21st January 2008

How much is my blog worth?

I was reading Sebastian’s latest rant about a scraper and spammer named Veronica Domb so I followed his link to the scraper site. Other than it being a complete copy of one of Sebastian’s posts, I saw a little button saying that the scraper blog was worth $10,161.72!

Since I actually write the stuff on my blog and not just republish feeds I thought I’d see what this little button thinks of me.

 


My blog is worth $38,388.72.
How much is your blog worth?

 

 

What load of crap!

Besides being totally inaccurate, it does auto-magically create a bunch of links to the button author, which I don’t appreciate. Those links would explain why his site is worth $6,220,101.72 (in its flawed opinion). The only thing consistent with that calculation is that all the values end in 0.72. So if anyone would like to buy me a beer for $38,388.72 I’ll transfer the domain, export the databases, zip the server contents, and email it all to you in about 30 seconds. We’ll be having steak that night.

Oh yeah, did I mention Veronica Domb yet?

If you liked this post please buy me a beer. Thanks.

posted in SEO | 2 Comments

17th December 2007

Pearls of Wisdom from Google

I found an interview with Matt Cutts on Sphinn that I found quite revealing. Normally these types of interviews are very bland and boring where the interviewer gets to build up some sort of street-cred by getting a big name search rock star to talk, and the interviewee gets to spew the normal company spin. This one was a bit different, at least in the fact that I took from it some pearls that can be used in forums to dispel frequently addressed concerns.

Leaving the obligatory social interaction out of it, the highlights of the interview as I see fit are as follows:

  1. Syndicating Content - When syndicating content to be published on multiple sources be sure to include a link within the content to the original source of the content. This will help transfer PageRank the syndicated content may get from external links to the original source. When Google is deciding what story to return in the results when there are many copies they apparently, ” a lot of the times it helps to know which one came first; which one has higher PageRank” On the other side of the story, if you are stealing syndicating content and want it to rank higher than the original then don’t include a link to the original source and get more PageRank to yours than theirs.
  2. Supplemental Results - There are a couple of undocumented methods for finding your supplemental page count still working. At least one data center is now actually searching the supplemental black hole for 100% of it’s queries, with, “hopefully at more in the future”
  3. Link Quality - “a link is a link, is a link; wherever that link’s worth is, that is the worth that we give it” .edu links do not count more than a .com link based on any specific weighting in the ranking algorithm. It just happens that many .edu links are naturally better than your average easy to get .com link. Additionally social bookmarking links follow the same guidelines and are not devalued based on their social network status, if they are weak they are weak on their own, without any help from a calculation.
  4. Link Count - The old adage of 100 links per page is a bit outdated and a good example is DHTML throwout menus when many links could be seen on one page. Matt notes that a page with 5000 links would have it’s PageRank so diluted when it came to distribution that the links wouldn’t pass much. The question I think wasn’t answered here was in regards to the DHTML menu structure is that navigation like that tends to be site wide and Google is quite good at determining what part of pages is the template or site wide stuff, vs the actual content. My question would then be is PageRank flow just a simple division of PageRank by link count, or does more weighting go to the actual page content and less to sitewide navigation. Obviously a page with 1000 links (like a sitemap) isn’t user friendly as the designer should have provided a logical tree for the user to find the information, rather than have to read 1000 links and figure it out for them selves. Bottom line is that 100 links per page isn’t a hard and fast number, but keeping it reasonable still applies.
  5. NOFOLLOW passing anchor text - In it’s early days there were some rare, and bug-like, instances where the anchor text of a nofollowed link was used in the search results. Those bugs have been killed. Right now, “At least for Google, we have taken a very clear stance that those links are not even used for discovery; they are not used for PageRank; they are not used for anchor text in any way. Anybody can go and do various experiments to verify that.”
  6. Predatory Link Buying - Buying links for your competitor in hopes of hurting them is more than likely going to help them as Google is most attacking the link sellers. Kind of goes without saying, but I bring it up only because the original Webmaster Guidelines on the issue only addressed buyers and not sellers.

There’s a lot more in the interview.

If you liked this post please buy me a beer. Thanks.

posted in Google, SEO | 8 Comments

11th December 2007

Of paid links debate

Earlier tonight I left a comment on Sphinn, “This is all becoming borderline Sisyphean,” regarding the debate that has permeated the SEO crowd.  I’ve looked for uproar beyond the professional search engine manipulation machine and just cannot find any.  Anyway I find the whole fiasco quite humorous to witness.  I have opinions on the matter, but none that are going to make a difference.

Avoiding the obvious wiki reference I found this definition to help clarify my position:

Of or relating to an endless and ineffective task.
This one comes straight out of Greek myth. Sisyphus was a king of Corinth, a son of Aeolus (the ruler of the winds, hence our word aeolian for something produced by or borne on the wind). In later legend he was the father of Odysseus or Ulysses. His name actually meant “crafty” in Greek: he was noted for his deception and he’s the equivalent in Greek folklore of the master trickster who turns up in many folk beliefs, such as Coyote in American Indian mythology. He even managed to cheat Death the first time around, surviving the experience to live to a ripe old age. In Greek legend Sisyphus was punished in Hades for his misdeeds in life by being condemned eternally to roll a heavy stone up a hill. As he neared the top, the stone rolled down again, so that his labor was everlasting and futile. The word first appeared in English in the middle of the seventeenth century. It isn’t used much these days because so few people understand the reference to classical literature.

The parallels that one can draw from that story are uncanny.  It also answers the question, beyond the obvious “not to help the spammers angle” why Google doesn’t notify everyone of their penalties, their cause, their existence, nor their cure.  The announcements of paid links have accomplished the same task, paralyzing the community in an endless and ineffective task of one-sided debate.

If you liked this post please buy me a beer. Thanks.

posted in Paid Links | 4 Comments

10th December 2007

Found on GWHG today

This was found today on the Google Webmaster Help Group. A site which is selling it’s SEO services is banned/removed from Google’s indexed. When pushed the poster admits that he doesn’t really know any SEO and is just “outsourcing” the services. Apparently he’s got a few of these sites or at least has scraped some, all with the fine keyword stuffed bottom navigation (classic), clip art images, no external links, you name it.

I don’t want to out the guy as he’s got plenty of troubles all ready, but this is just classic. From the FAQ of the site:


Why is [site name] not ranked high on the search engines?

[site name] web site is intentionally not optimized for search engines because our services are for companies needing high traffic exposure and awareness. The less traffic we receive the better because we focus on qualified and selected clients that will actually benefit from High Rank optimization.

You just can’t make this stuff up.

If you liked this post please buy me a beer. Thanks.

posted in GWHG, SEO | 4 Comments

26th October 2007

Googleblog linking to bad neighborhoods

In what many are calling just a warning volley across the bow of many link-selling sites Google has initiated a penalty of sorts by reducing some sites’ visual PageRank score, one such site is Search Engine Roundtable which provides us proof that their rankings and traffic have not been affected.

Clearly a reduction in PageRank for a site selling links is a signal that Google feels that they are breaking the rules as written in the webmaster guidelines. Regarding link selling they state,

Links purchased for advertising should be designated as such. This can be done in several ways, such as:

  • Adding a rel=”nofollow” attribute to the <a> tag
  • Redirecting the links to an intermediate page that is blocked from search engines with a robots.txt file

Which is a practice that Search Engine Roundtable is not ready to take part in as made evident by Barry Schwartz (rustybrick) in the above linked article,

On a personal note, I trust my sponsors, I value their sponsorships and I couldn’t do what I do without their financial support. Some sponsors can’t afford huge sponsorships, so they sponsor in their ways. It is what enables this site and many other sites to function and operate on a daily basis. I turn down sponsors all the time because they are simply not relevant or useful to my reader. I hand select them and for them to be on my site, means I trust them. Why nofollow someone you trust and want to thank? Is that a slap in their face? Will I have to and will they continue to sponsor? Time will tell.

So we have a site that outwardly sells links, does not want to conform to the webmaster guidelines by marking paid links in the manner in which Google desires, and has been hit with a PageRank reduction. Clearly a signal that the site could be considered not only a rule breaker but a bad neighborhood to be associated with.

In Google’s webmaster’s guidelines it clearly states,

In particular, avoid links to web spammers or “bad neighborhoods” on the web, as your own ranking may be affected adversely by those links.

I’ve written before on the difficulties of discerning (complete with Matt Cutts’ email address!) what a bad neighborhood is even asked for clarification on the matter.

I find it quite ironic then that Google’s very own official blog links to said site which has admittedly broken the rules and publicly been admonished. ( screen shot ) Are they not taking their own advice and linking to a bad neighborhood? This is not the first time Google has talked out of both sides of their mouth. Even during this last wave of anti-link-selling assaults the largest link seller in the land has gotten off scot-free with out any sort of PageRank deduction. Matt Cutts has even come out and said that they are allowed to sell links because they review the links before they publish them and not every one makes it into the directory. If that is the standard to be followed perhaps Barry Schwartz and others like him to accept advertising dollars should just charge for the chance of being listed on his site, add an element of uncertainty to the equation. Maybe then he will get his PageRank back while making more money as he could oversell the advertisements 10 fold. To me that seems highly unethical but for some reason is the only method of linking endorsed by Google without the use of machine only readable declaration of a paid link.

If reducing one’s PageRank for selling links is really a penalty, I would expect Google to do the right thing for their own blogs ranking and not link to such terribly bad neighborhoods (t.i.c.) and will be watching the site as an indicator.

Before anyone get’s upset at me, I’m not calling Barry Schwartz a spammer or even personally think he did anything wrong. The PageRank degrading has been widely published and I am just drawing the connection between what Google says and what they do. I follow Barry on Twitter, his own Cartoon Barry, Search Engine Roundtable, and his contributions on Search Engine Land. He is considered a leader in the industry sector and I value his opinions and expertise.  There’s at least 4 links in this article to his properties that are genuine followed editorial links of endorsement, not like the link to the Yahoo! directory which is nofollowed due to their blatant breaking of the rules.

If you liked this post please buy me a beer. Thanks.

posted in Google, Paid Links | 1 Comment

24th October 2007

Digital Point Members put on Suicide Watch

noose.jpg

THIS IS AN EMERGENCY SEO/WEBMASTER BROADCAST.

Due to Google’s apparent assault on Paid Links resulting in some sites’ PageRank being reduced, Digital Point forum members will have to be guarded 24 hours a day for the foreseeable future. If you are near one please remove their belt, shoelaces, and anything that could be fashioned into a sharp weapon. They should also be moved to the lowest floor in the building and all windows should be boarded up.

This is not a drill. I repeat. This is not a drill.

If you liked this post please buy me a beer. Thanks.

posted in Paid Links | 2 Comments

24th October 2007

Rants: paid links and penalties

It’s my party and I’ll cry if I want to.  I’ve been reading a lot of ranting lately on Sphinn and the blogs, which got me into a ranting mood.  Let the games begin.

Even though Google has a ton of official blogs, discussion groups, webmaster guidelines, and press releases available they decide they would be best served to send Deep Throat down to the parking ramp with Danny Sullivan doing his best Carl Bernstein impression to break the news that penalties are now given to link sellers.  I can only guess they’d choose this chicken-shit cowardly approach because it has some plausible deniability if it really hit the fan.  Beyond their poor choice of using unnamed sources I have a couple other issues bugging me.

As soon as one is assimilated into the collective the first thing they teach the new drones is the gospel of “Don’t Worry About PageRank“.  You’ll see is spewed from every orifice of any Googler giving a speech, writing a blog, answering a question in a forum, or just plain pontificating from on high.  It’s the canned response for any and all questions regarding the green bar; its effect, its acquisition, its retention, its loss,  its very existence.  They are all told to say things like, “worry less about PageRank and more about creating unique and compelling content [and tools].” So if this PageRank is nothing to worry about, then why would docking some college newspaper’s PageRank be a suitable punishment?  If PageRank is no big deal and not to be worried about as much as content, why would they choose this as their punitive reaction?  Maybe one should worry about PageRank just a little bit.  You can’t have it both ways, either it’s not worth worrying about or it is worth worrying about and something we should all fear loosing.   This leads me to another thought on the matter, that it’s not punishment but rather an adjustment, more on that later.

It’s clear that to battle the evil doers that sell and purchase links some sort of punishment must be doled out.  They can’t have an outright ban on all site that sell a link, as Google would soon become a joke.  If someone is searching for Stanford’s Newspaper they’d better find it.  If not, then Google looses it’s relevancy.  Sure it wouldn’t matter much if they just destroyed some nice lady in Colorado who’s buying baby food with the money she makes from her site, but there would be plenty of high profile cases that would just make them look silly.  We as webmasters, marketers, SEO’s or just plain anyone who has any idea how the inner workings of search work have to step back from the scene for a moment.  The VAST majority of Google’s users, customers, and shareholders don’t give lick about paid links, hidden text, or cloaking.  They just know that when they search for something they expect to see it.  If Google banned Stanford for selling links and someone who wasn’t in the know was told that was the reason, the response would be a great big, “So what?  The point is that while selling links goes against Google’s webmaster guidelines, not listing the site selling the links goes against Google core principles of returning the most relevant results.

That principle has it’s limits.  In the case of a site known for fascilitating the selling of links, it’s so well known that when you use the Google tool bar to search for its name you’ll get it listed as a suggestion as soon as you type [text-l] in the field.  If you continue the query and fill the whole [text-link-ads] you will not find the site listed.  In this case, Google has decided that returning the most relevant results are not quite worth as much as punishing the offender.  So I am quite confused when that distinction is made.  Is it just academic institutions that get this exemption? Or if Matt Drudge started selling links would he too get to be listed for his name and his site?  I find it utterly priceless that Google is taking the moral high ground on this text-link-ad selling problem by not returning the site for its own name, but they are more than happy to take their money to show their ads in the results.  In this case the most relevant result is required to pay for their position.  This reminds me of a little story I read on the Stanford web site, ” For example, we noticed a major search engine would not return a large airline’s homepage when the airline’s name was given as a query. It so happened that the airline had placed an expensive ad, linked to the query that was its name. A better search engine would not have required this ad, and possibly resulted in the loss of the revenue from the airline to the search engine. In general, it could be argued from the consumer point of view that the better the search engine is, the fewer advertisements will be needed for the consumer to find what they want. This of course erodes the advertising supported business model of the existing search engines. However, there will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.”  No, it’s not an exact comparison or even a pretty close metaphor, but the idea that the most relevant result has to pay for its position is true in both cases.   Larry and Sergey knew that was wrong way back then, oh my how their little project has strayed.

I’m going to go out on a limb here and postulate that Google cannot detect 100% of the paid links 100% of the time.  I’ve deduced this solely based on their behavior.   1) they still encourage people to tattle on their competitors and do their job for them 2) They have to manually penalize sites by removing PageRank or knocking them down a few hundred notches in the results, and 3) If my buddy calls me tonight and tells me he’ll buy me a shot and beer if I link to him tomorrow, no where in that process is Google involved.  If they were able to detect paid links there wouldn’t be a need for penalties of any kind, they would just re-rank the index as if said links didn’t exist.   The fact that Deep Throat and Danny had to have that clandestine meeting is proof enough to me that Google’s ability to detect paid links is completely flawed.  By admitting that penalties for selling links exists, they are admitting that they cannot handle them algorithmically, which as you’ve heard before just isn’t scalable.  Sure there isn’t a shortage of 3rd world countries with people willing to work for $1 a day hand checking sites, but at some point the web will become so large that even that isn’t scalable for a company with billions and billions to spend.

I don’t want to just hammer on Google, I give them a lot of credit, they still are the best option available for sending free traffic to a site that isn’t going to go viral on youtube.   Perhaps Google’s inability to detect and devalue paid links isn’t all that flawed, all paid endorsements are not irrelevant.  That is what we are after, relevancy.   If you want Bill Clinton to speak at your college’s commencement be prepared to pay him handsomely for the honor.  That does not make his speech to the leaders of tomorrow any less relevant.   To get Jeff Gordon to use your motor oil and put a little sticker on his car it’s going to cost you millions, but his endorsement would mean a lot more than the man on the street telling you what to buy.  Then again if Bill Clinton told us what oil to buy and Jeff Gordon wanted to tell us how to work in the global economy no one (should) listen to them either.  The point is that both of these men are experts in their field who demand a high amount of compensation for their limited time.  The fact that they are paid does not render their opinion any less relevant.  The same could be said of links.   If Stanford links to an academic the link should carry a lot of weight, then again if they link to britney-spears-mesothelioma-nude-lawyer.info it shouldn’t be considered an endorsement on that subject either.

In that sense the drones saying, “Don’t worry about PageRank” are right. PageRank in it’s purest form, the sum of the weighted links to a page shouldn’t be worried about.  The relevancy of the links should, be them paid endorsements or pure out-of-the-goodness-of-their-hearts editorial links.

One final note on paid links.  This includes some other webmaster guideline no-nos here as well, like hidden text.   We can all easily prove that Google’s ability to detect either a paid link or hidden text is limited.  Create a new page, buy a link see if it get’s indexed or create a page with some obscure hidden text, see if you can find it on Google.  Even if they could detect 100% of the hidden text and paid links within a month of it’s publishing, that month would be plenty of time for some people to make use of it.  With domains being pennies now adays the true black-hatter doesn’t even care if a domain is banned, penalized, or blown-up completely.  By the time that month is over they’ve moved on to hundred or a thousand other sites.  Who’s really getting caught up in this dragnet is the “honest” webmaster’s who think they are acting the way they should.  They are trying to build a site for the long-haul and really want to produce a good product but are fed so much bad information that they truly think they are doing the right thing.  It’s the center of all the anger I have with Google right now, utter lack of communication with the real webmaster.  Daily many webmasters approach the Google webmaster help group saying things like, “I’ve exchanged tons of links, bought links and yet I lost my ranking” Not because they are trying to be sneaky but because they feel that is what they SHOULD do.  They don’t read searchengineland or listen to Rand’s youtube video of the week because they are busy running their sites.  It’s the actual honest webmaster’s who don’t have the right information in front of them that are getting hurt, while the black-hats slip through the cracks only to have Google help them by removing legitimate sites by the thousands.

I want to rant about PageRank funnelling and Google’s Green attitude, but that will have to wait for another post.  I’m tired.

If you liked this post please buy me a beer. Thanks.

posted in Google, SEO | 2 Comments

25th September 2007

Googlebot gave up

Feedback LoopThere’s been some rumblings lately around the fact that the DMOZ home page was removed from the index. I don’t pay too much attention to the DMOZ, but in this case it was interesting. I started to follow various threads in the webmastering/SEO community diligently as I’ve seen this “lost my homepage” behavior many times in GWHG. I even made an appeal on behalf of the unfortunate webmasters which was ignored.

Matt Cutts, the true ambassador to the webmaster, came through and answered the question, even taking time from electronic cat gadgets and their pedometers to do so.

Hey all, I dug into this a little bit with the help of a couple crawl folks. It looks like when Googlebot tried to fetch http://www.dmoz.org/, we got a 301 redirect back to http://www.dmoz.org/ . It looks like that self-loop has been going on for several days. We were last able to fetch the root page successfully on Sept. 10th, but from that point on DMOZ was returning these 301-to-itself pages, and after a few days Googlebot gave up on trying to fetch the url.

This makes sense, as Googlebot hit the page it would get a 301 response saying that the new page was the page it hit. When that information got to the normal process that handles 301s it probably just faulted out. Since no other information on a page loads after a 301 (normally) they would have to remove the page as they’d have no data for it.

Here’s the odd thing

When I first heard of this, several days ago, i visited the DMOZ site, and viewed it just fine. Depending on your browser, you can’t view a page that redirects to itself, as this example I’ve set up. Internet Explorer will just sit there and spin, Firefox will eventually give you an error message, and using an online tool will let you know that there is an error.

Pure Conjecture

Matt Cutts has been doing this a long time and probably the best at speaking around issues when he needs to (protecting secrets, towing the company line, etc) but never has there ever been any appearance of being anything less than truthful, so I will by default dispel the idea that he was giving us bad information. So how can I not see a 301 redirect, no one else mentions that the page won’t load ANYWHERE in all the discussions, but yet Googlebot sees the behavior?

  1. All things considered, the simplest explanation is usually the best, perhaps the 301 redirect was briefly shown only when Googlebot happened to visit the site, but not long enough for anyone to take note of it.
  2. They somehow managed to return a 301 response code, but not the redirect. This is something I tried to simulate on many platforms but could not. The browsers and tools I used all seemed to expect the redirect location and either defaulted to one or erred out. Google on the other hand doesn’t actually CRAWL anything, they just hit the page and return back with whatever it saw. I don’t know enough about how the interwebby works to really say if this is a possibility or not, it is after all pure conjecture.
  3. They were cloaking their 301 only showing it to Googlebot (or other bots for that matter) and not to regular users with a browser or not from Google’s IP range.
  4. Perhaps the 301 was referrer based, and when there was no referrer it showed the redirect. Googlebot, since she runs on a predetermined schedule of URLs to crawl would not show a referrer.

Any other ideas that I am too simple to see?

If you liked this post please buy me a beer. Thanks.

posted in Google, Matt Cutts, SEO, Webmastering | 5 Comments

28th August 2007

Paid links: A scalable solution

Google has always been smart in respect to building solutions based on scalability. From the onset they always wondered what would happen if they had to grow the solution at hand by 10 fold or even greater. Scalability in their algorithm is so entrenched as its philosophy that they even openly admit that sites that are submitted via a spam report are not removed or penalized. They rather use that data as information to judge their algorithm against.

What amazes me regarding their battle with paid links how non-scalable the solution is:

  • Adding a rel=”nofollow” attribute to the <a> tag
  • Redirecting the links to an intermediate page that is blocked from search engines with a robots.txt file
  • [it used to say something about javascript but they took that out]

They even take it a step further, which is obviously also a step back in their fight against spam, when they ask for people to submit sites that sell links, supposedly for some hand-to-hand combat.

So what’s a more scalable solution, Google tweaking it’s system to identify paid links on its own or having millions and millions of webmaster’s modify their billions and billions of pages available on the web? Obviously its much easier for Google if we all just bend over and do their job for them, but then again how serious are they about this? Sure it’s available in the guidelines, at conferences, and if you read Matt Cutts blog, but that probably reaches a very small percentage of the real content creators out there. All the pros will know about it, but the VAST majority of indexed content managers out there are going to miss the message.

Let’s go back for a second to review why they think Paid Links are bad. What set Google apart from the rest of “search engines” at the time was that they not only looked at the content on the page but also used the academic model of references in literature to vouch for the authority a page is on the subject. At the time of that original theory the web was young and innocent and pretty much not exploited nearly as much. So Google’s rankings are based largely on the links to a page/site and since most people want to rank higher so they get more traffic the obvious optimization procedure is to get more links. Had they ranked sites based on the use of purple text all websites would be using purple text today.

Back when the original ideas for Google came about most of the links out there were actually votes for other sites. It was when “surfing” the web actually meant bouncing around from site to site based on the links of those sites. You didn’t Google something you surfed for it. In 1584 when Google came up with this idea the barriers to get online were much higher than they are now from registration and hosting to easy content generation it’s gotten much easier to get your site online today, back then it was more academic institutions, geek squads, and corporations that had the resources to publish sites.

Well the times they are a changing. Now you can buy a domain for pennies, hosting is next to free, and writing content has never been easier. There are so many millions of new links created every day that they have lost their value due to the sheer volume of links available. HOWEVER, there are some sites that have some value, traffic, authority, PageRank, and links from those sites tend to be worth something, and BOOM an economy of link selling is born.

Not straying too far from their original founders who borrowed the reference system used in academic papers as a judge of quality, Google wants to borrow from the older established media sources that must disclose paid endorsements. What’s different however is that most of those media outlets are regulated by authorities. Being that Google is the only game in town when it comes to actual search traffic they are the defacto authority to regulate the masses.

So how can Google get everyone on board, let me repeat that EVERYONE, not just the 0.0001% of the publishers that read Matt’s blog, or the 10,000 subscribers to seoMOZ, but EVERYONE. If Google wants to regulate the web then they need to start regulating it and not just observing it, its going to be painful but if they really want to monitor all the links on the web it will have to be done.

  1. The first thing to do is throw out all of the links known till this point. They are polluted, we have no way of knowing the intention of any of the links since they exist pre-regulation.
  2. In order to have the links count they have to be registered, verified, and monitored by Google so all websites will have to be removed from the index.
  3. After verifying ownership in your webmaster tools account, Google will crawl the site. They can then show you a list of all the external links on your site. You then select what type a link it is: Regular Voting link, Paid Link, non-endorsed user generated link, etc. After selecting the link attribute you will have to digitally sign an agreement attesting to the authenticity of your claim, enter the captcha, and submit. Repeat for the rest of the links on your site.
  4. After the links are verified and attested to Google can then add them in as votes or non-votes into the index.

Now we’ve got something with some teeth in it. In order to be included in Google’s index you have to have agreed to their terms and have signed a legally binding contract that they can go back on.

  • We no longer have to worry about hidden links as they won’t be verified.
  • Links will only be bought and sold for traffic.
  • You can code your links any way you’d like.
  • User submitted link directories are all but dead.
  • Sitewide links will probably disappear due to the sheer labor required to insert them.
  • Sneaky little plug-in and theme developers that drop links all over the place will be wasting their time as the site owners probably won’t vouch for them.
  • Automatic text link building systems will grind to a halt as whenever the links change on a page the page will drop out of the index waiting to be verified.
  • As the publisher has to be verified by state issued credentials, large false link networks built up by SEO’s will have little value as Google will be able to see all of them as owned by the same person.
  • Comment spamming will disappear as people will just turn off their comments.

Now until that is instituted and since you’ve read to the end of this story and know about Google’s stance on paid link you are morally bound to nofollow all of your paid links and only buy nofollowed links. Granted your competitors who didn’t go to SES San Jose or read Matt Cutts blog probably aren’t doing that, but that’s your problem not Google’s.

The only flaw in the system is that some people may actually LIE and say that a link that they got paid for is actually a regular link. Oh my. Well at least that’s a sin of ccommission and not a sin of omission like the millions of people currently not nofollowing their paid links.

If you liked this post please buy me a beer. Thanks.

posted in Paid Links | 2 Comments

16th August 2007

TIFKAS

no supplemental

I’m proud to announce: TIFKAS

T - The

I - Index

F - Formerly

K - Known

A - As

S - Supplemental

Inspired by Prince when he went by The Artist Formerly Known As Prince - TAFKAP

tafkap.jpg

If you liked this post please buy me a beer. Thanks.

posted in SEO | 1 Comment

  • Please Support

  • Marquette University

  • Sponsored