A week ago, Google ran an update to its search algorithm which is known as Penguin. This is likely to be run more often and with greater complexity. Publicly Google says this update is to deal with “black hat webspam.” Embarrassingly the example Google gives in its own blogpost on the update shows that cheating the system can still work. That said, it’s still not something I would recommend to clients.

The Effect of the Penguin Update

The Penguin update appears to target a variety of sites that suffer from genuine spam or hacking as well as sites that appear to have an ‘unnatural’ link profile.  Google claims this has changed the way 3.1% of sites worldwide rank, but in some countries, as many as 12% sites are affected.  It is also clear that sites which have engaged in high quality syndication of original articles to other places on the internet have also been affected, which seems a bit unfair.  Google has never liked this anyway, as evidenced in this video from their webmaster series. The difficulty is always in getting a machine to distinguish between good, bad or plain awful writing.

So if you’ve been hit by the Penguin update, your analytics will look something like the chart below, with a change immediately after 24th April 2012.  If your site has changes earlier than this, especially around 19th April, your site might have been hit by the latest Panda update which works in a different way.  Note that the site shown below has only dropped a few places on front page rankings for many of its major keywords - a demonstration of just how important above the fold positioning in Google is.

Google’s Example and How the Cheats Still Prosper

Google’s own blog on the update gives an example of the sort of web spam it is targetting.  Here’s an image from that blog post, which shows an article which is probably machine generated or ‘spun’.  You can see that way the link saying ‘Pay Day Loan’ has no relationship to the article itself.

The site hosting this article is no longer indexed by Google, but it is indexed on Bing and so it is possible to find the original site and track which site the ‘Pay Day Loan’ text links to. Unsurprisingly, it is a company selling pay day loans, called checkintocash.com, which is engaged in massive amounts of link building. A search on MajesticSEO, a specialist search engine for tracking links, shows that this website has a massive 350,000 inbound links.

What is interesting is that this massive exercise in link building in a spammy way is still working and checkintocash.com does not appear to have been affected by the Penguin update.   A search for [pay day loans] on Google.com shows it is ranked third, above the fold.  Cheating – or beating – the system still seems to work.

So Should You ‘Cheat’ the System?

Asking whether you should ‘cheat’ (or perhaps ‘beat’) Google’s system is one which many webmasters constantly consider. My general advice is that if you are running a grown up business in a grown up way and wish to grow a brand, then avoid it.  If you are a fly-by-night marketer who does not mind if one of your domains vanishes or drops from the Google index after six months or a year of stellar performance, then that might be different.

There is a very fine line between “White Hat” and “Grey Hat”  SEO, and assuming we are discussing practices which are quite legal (rather than, say, hacking other peoples sites) there are cases for going either way.  My clients are usually given “white hat” advice only, but we keep a careful eye on everything else and experiment with it too.

The reality is that Google is running their business and you are running yours.  You probably have a symbiotic relationship with Google for search marketing.  However, Google is not interested in your individual site and if it falls due to an algorithmic chance by design or fault, you have very few ways of quick recovery.  Your overall marketing needs to think carefully about your own customers (mailing lists?), other marketing channels (Facebook, Amazon, Ebay etc) and a business plan which is not reliant on Google alone.

Further reading

The following blog posts from other publishers might be of interest to anyone who wants to read further about the Penguin update:

Google Launches Penguin Update

Penguin Recovery Tips

Unnatural Inbound Links Heavily Targeted

Keyword research is market research for the digital age. We have never had more data to focus tightly on the markets we are selling to. It is an essential precursor to building a web site and creating laser-targeted web pages properly to achieve success in either a Search Engine Optimisation (SEO) or online advertising campaign. Here I look at what is needed for SEO – Getting to the top of Google.

Keywords are words or groups of words which people use to search on Google and other search engines.

Google releases data to us through its Adwords Keyword Tool and we can find other useful data for SEO from a number of other sources, such as Majestic SEO for analysing links. However, specialist software (such as Market Samurai), expertise and an insight into the market targeted by a particular business are needed to show whether a web site and web page can achieve meaningful results in a search engine optimisation campaign.

There is a fine balance to be made between the amount of traffic, the competitiveness of the marketplace and a suitable budget.

On average, a top organic search listing will achieve 42% of the click-throughs for a search. The seventh listing will achieve 1%. You can see the gains can be impressive, but proper keyword research must be carried out.

As an example, Google tells us the UK monthly traffic flowing for the following search terms:

• [Hat] 2,740,000
• [Wedding hat] 40,500
• [Black top hat] 1,300

Other research shows that [hat] is far too competitive a term to focus on as there is little chance of ranking for it without very deep pockets. It is probably only a suitable term for a very large retailer to target, being very generalist.

The competition for [wedding hat] is lower and a focused seo campaign for a specialist shop would be worthwhile.

Targeting [black top hat] would be relatively easy with a well designed web site and a good link building campaign. Even though the search numbers are far smaller, the searchers are very focused and traditional silk top hats can cost £2,000 in London’s West End. At the same time, there is high competition for advertising on the term, driving the Google Adwords price up to over £0.60 per click, meaning this would be an ideal term for a specialist retailer to focus their SEO efforts.

Here are just some of the factors a keyword researcher needs to look at to understand how successful a SEO campaign might be and how much might need to be spent on it:

• The number of other websites containing the search term on their pages, in their URLs, meta titles and descriptions.
• The number of links to competing pages and competing domains.
• The authority of links to competing pages and what anchor text those links are using.
• The PageRank of competing websites in the top search results.
• The size of websites in the top search results.
• The age of the domains ranking in the top search results.
• The age and authority of the domain which will be the focus of the campaign.

A sensible piece of research will analyse tens of thousands of pieces of data for a small business; will generate previously unconsidered keywords; and will help a business focus. Software will help gather this data, but ultimately insight and common sense will make sense of it.

Without keyword research, it is impossible to run either a sensible or an economical search engine optimisation campaign. If you are building a new website, this should also be the starting point.

Everyone knows Google has a very complicated algorithm to assess the quality and relevance of a website for a particular search. Few people realise that Google also has a team of quality raters who are trained to manually review web pages.

In the last month or so, a recent copy of Google’s General Guidelines for rating URLs was inadvertently posted on the internet. Google’s lawyers have made pretty certain that this confidential document has been removed, but I had a chance to read the rather turgid 125 page document before it was taken down.

When are manual reviews likely to happen?

It seems certain that search queries and websites will have a manual review as a research exercise before an algorithmic change is discussed and applied. In addition, manual reviews are likely to be applied to websites where someone has submitted a spam request to Google. Google may then apply a manual penalty to the site or remove it from the index altogether until the problem is fixed, at which stage a reconsideration request can be made to Google to have the site re-indexed. This penalty may be due to the fact that page contains malware, or ‘may be spam’.

Google will also apply manual penalties to sites that are cheating the system, like the case of US company J C Penney, which was buying text links on a large scale reported here in the New York Times.

What is uncertain is whether Google are applying manual reviews to high ranking websites for high traffic keywords as a matter of course. They could be.

What do the Guidelines Say?

Google’s manual reviewers are rating a webpage according to the following scale of relevance:

Vital
Useful
Relevant
Slightly Relevant
Off-Topic or Useless
Unrateable

In addition, pages are flagged as:

Not Spam
Maybe Spam
Spam
Pornography
Malicious

Any flag other than ‘Not spam’ is going to cause problems for your rankings.

For some types of business and website owners, there is a very subjective difference between ‘not spam’ and ‘maybe spam’ especially with sites designed to earn advertising, commissions, or affiliate sites. There is a fine line between these categories whether the site is human reviewed or algorithmically ranked.

As ever, the key thing is to ensure your own website has high quality original content which is relevant and useful to the user. If in doubt, seek some advice.