Links for 2005-10-02

Tom, the claim it’s #1 may or may not be perfectly true, but Wikipedia does benefit from the way PageRank worked–at least the original algorithm. Only Google knows how it works today, of course.
cf. http://www.webworkshop.net/pagerank.html
I curbed my enthusiasm though. I admit, I got a little carried away with myself.

Tom,
There is a lot more to the story behind PageRank than what the Google toolbar suggests. For instance, Google selects certain sites as authorities. It’s impossible that Wikipedia will be gain the highest PageRank, but it is increasingly gaining the highest relevance on the page results. You can try it for yourself:
http://ericgalloway.blogspot.com/2005/07/do-20-of-wikipedia-articles-have-1.html
As I have been told, I’m (unsurprisingly) not exactly the first person to make this obvious claim about the relationship between the PageRank algorithm and Wikipedia.
http://www.nivi.com/blog/article/wikipedia-defeats-google-sometimes/
http://ericgalloway.blogspot.com/2005/09/does-yahoo-search-rank-wikipedia.html
Eric Galloway in particular has done a lot of investigation on this topic.
Regarding fairness, there’s this argument
http://www.qwikly.com/2005/07/wikipedia-and-google-work-together.html
Google’s PageRank algorithm is objective in and of itself, and in that sense ‘fair’, but it is not ‘in and of itself’. Rather, it is in a feedback cycle with the entire World Wide Web, and as a result it is only as ‘fair’ as the world of human endeavours.
Now, I first noticed the correlation between wiki size and PageRank in the early days of wikidom, and later discovered the underlying mathematical reason why. These days I think about how this relates to my more pressing problem of WikiSpam, and why wikis are hugely disruptive to Google’s quality.
If you recall, not so long ago, Google was increasingly returning spam links as relevant links. The strategy was to build huge link farms. Google altered their search engine to counter. One technique they use is to identify ‘cookie cutter’ websites in the hopes this will encourage high quality content to be rated highly.
It’s a natural economic relationship, really, and one that will continue to be fought. As long as Google uses a statistical model to deduce quality rather than human intellect, people will look for ways to bias those statistics in their own favour. Google will look for a way to punish those people, and other people will look for a way to punish and extort Google.
The search results are never going to be fair. What is more important to me is “are they useful?”
Google is not in the business of fair. They are in the business of selling search services and advertising, and they succed by answering questions usefully enough that people will go to them first for quick answers. If you want fair, you should use a multiple search engine approach, plus manual classification systems, plus personal research to find a more balanced picture–which is a process that is implicitly incorporated in the way Wikipedia works, mind you. (Wikipedians frequently do actual non-Internet library research. Shocking!)
However, I think fair is overrated when it comes to search engines because it’s unsettleable. Because the scope of the Web is too large to understand, you’ll never be able to predict, know, or decide whether a given ordered result set is ‘fair’. In particular, there are no objective criteria to measure. Sure, searching for ‘IBM’ should return http://www.ibm.com, but what about ‘Tom Coates’? Why are you first and not http://www.math.harvard.edu/~tomc/? You benefit because the structure of the PageRank algorithm likes your big content-full website with lots of blog hits, but that doesn’t mean that you are more relevant to my search query if I was looking for more of Dr. Coates’ papers.
Similarly, why am I a more relevant “Sunir” than Sunir Kapoor? I’m not as powerful as he is, but I do write a lot on wikis.