I've been reading an article today that is a statistical analysis of the correlation between keyword difficulty and the size of the domain. Basically, this analysis asks the question of whether bigger sites do better for more competative keywords.

To do this, the team looked at around 30 different keyowrds with differing levels of competition. They defined 'competition' as:

Keyword Difficulty = Keyword Search Volume (Exact match) x Google competing pages

essentially, the more results there are for an exact match in the google serps, the more competative the phrase is likely to be, and also the more times the phrase is searched for, the more competative it is. In actuality, this algorithm should include a logarithmic factor, but for a crude analysis, the formula stands.

The keywords were grouped into high, medium and low competative results. Then for each keyword, they retrieved the top 10 results in google, and for each of these 10 results, they checked the number of indexed pages useing the site: prefix in the google search box.

From this, they computed the average number of indexed pages for each keyword and made a correlation plot of the keyword competition vs average number indexed pages (after stripping out outlier results that can skew data). The researchers found that there was a 78% correlation between the competitiveness of the keyword and the number of indexed urls.

In the conclusion, the study does recognise that other factors associated with high ranking can also influence domain size, for example, older domains tend to have more pages, and also domain age has a positive influence on page ranking. Also older domains have had time to organically grow backlink and trust rank.

Another factor is that large domains have a higher number of internal links so have and advantage over smaller domains in that respect.

One thing that the study does not mention is the fact that high ranking sites for competative keywords are more likely to have been deliberately manipulated by SEO activities to target those keywords, and that lower competative keywords do not need a big budget SEO campaign for better ranking, and therefore, cheaper smaller sites have a much better chance of gettign to the top of google for long tail keywords. Sites with big budgets for SEO also have big budgets for onsite development, and therefore have greater numbers of pages.

The conslusion from this study, although not without it's flaws, is that bigger sites do better in search engines for competative keywords, but smaller sites do better for more short tail keywords. This site speclialisation effect means that if you have a site targeted at a niche keyword, you do not have to spend as much time adding content to get high rankings for that phrase, but if you want your site to get more traffic, then you should put a significant amount of effort into adding new and rich content for your site. Sounds obvious really doesnt it.