The product is also available as a Britannica Profiles Hollywood, a CD-ROM for Windows and Macintosh computers.

Interviewer: Back to the accuracy of the Wikipedia postings. Because it is much more dynamic than other encyclopedias that are out there, it could be more accurate in some areas.

Wales: That is absolutely true. There are quite a few good examples of that. There is a small scandal going on in Germany. One of the questions on the German version of ‘Who Wants to be a Millionaire’ was wrong. The show had referenced an answer on the German version of Brittanica, which was wrong. It was wrong on Wikipedia as well, but we were able to update it immediately.

Piercing insight aside embedded in the “question” aside (it could be more accurate, it could also be less accurate, for the very same reason), one should point out that (a) Britannica is spelled B-r-i-t-a-n-n-i-c-a–it’s just not hard to get it right and that (b) while there are versions of Britannica in Korean, French, Chinese, Japanese and a large number of print translations, there is no “German version of Britannica”. Wales no doubt was refering to Brockhaus.

Shame on you, Nathan Kaiser, for failing to do the least bit of copy-editing or fact checking.

While the URS is given fairly high marks, both reviews ding Britannica for splitting the different encyclopedias (”Encyclopedia Britannica”, “Student Library”, and “Elementary Library”) into different areas of the application (in screenshots, the three colored tabs across the top)–a complaint we’ve heard from user interface consultants as well. Oddly, this has been a feature of the design since 2003, but this is the first year we’ve heard significant complaints about that. In any event, we hear you. I’ve seen prototypes for the 2007 edition that attempt to address the problem in a couple of ways.

The CNet reivew is also a bit harsh on Britannica’s customer service, which suffered some system failures during the recent headquarters move–not that that justifies the problem CNet encountered, but it may help explain it. I’ve talked to the head of customer service personally and I guarantee you they are ready and willing to prove their mettle.

Vandalism is a problem, Wikipedians are quick to assert, but one that is solved by constant vigilance–Wikipedians are watching recent changes “like hawks”.

“Yes, vandalism is common on Wikipedia,” we read in the recent collaboratively edited press release, “but Wikipedia heals quickly.” After all, “IBM researchers found that most vandalism on Wikipedia was reverted in less than five minutes.”

We see this statement frequently repeated at Wikipedia and elsewhere.

Most vandalism on Wikipedia is reverted in less than five minutes. Let us assume, for the moment, that that statement is true. Does it imply that vandalism is a solved problem for wikipedia? Well, no. Suppose that 99 out of every 100 articles that get vandalized are reverted within 24 hours. Then there is more vandalism in Wikipedia today than there was yesterday. Without knowing the rate that un-corrected vandalism is added to Wikipedia, it is entirely possible that the percentage of vandalized articles is greater today than it was yesterday. The rate at which most vandalism is reverted isn’t the right question to ask, we should be concerned with whether the amount of vandalism is shrinking or growing.

But it gets worse than that. Most vandalism on Wikipedia is reverted in less than five minutes. Is that a meaningful thing to say? In order to know that most vandalism is reverted within minutes, wouldn’t we need to identify all vandalism, at least for a representative sample of Wikipedia articles? At best what we really mean is that most known vandalism is reverted in less than five minutes. Unknown vandalism is, well, unknown.

But wait–there’s more. Most vandalism on Wikipedia is reverted in less than five minutes. Did IBM researchers actually say that? Well, no. As far as I can see, the article to which everyone links seems to have only one paragraph on vandalism, which reads as follows:

“As publicly editable sites, Wikis are vulnerable to vandalism. We’ve examined many pages on Wikipedia that treat controversial topics, and have discovered that most have, in fact, been vandalized at some point in their history. But we’ve also found that vandalism is usually repaired extremely quickly–so quickly that most users will never see its effects. The pictures below tell the story.”

The “pictures below” are:

“Visualizing every saved version of the page on “abortion”, with each version getting equal space. The vertical black interruptions indicate times when a visitor has deleted most of the page.”

and

“Same page on “abortion”, but here horizontal spacing corresponds to time, so that rapid-fire changes show up almost on top of each other. Because vandalism is repaired so quickly, it does not show up in this view of the visualization”

Wait a minute. The IBM tool visualizes (a) the number of lines in the article and (b) who created those lines. It doesn’t give any insight at all into the content of those lines. It seems that they’ve defined “vandalism” as “deleting most of the page”, and that in articles they’ve examined this is usually repaired “extremely quickly”. Wikipedian’s don’t even enumerate “deleting most of the page” on their list of common types of vandalism.

Where’s the “most vandalism” part? Or even the “five minutes” part? What IBM researchers really say is that for the controversial articles they have examined, page-wipes are restored quickly.

It seems that this “IBM researchers found most vandalism on Wikipedia is reverted in less than five minutes” line is a complete myth: IBM researchers didn’t actually make that claim, it’s not a meaninful claim to make, and it doesn’t really tell us anything at all about the volume of vandalism within Wikipedia.

[Via Gizmodo.]

This sparked several uncharacteristicly self-critical responses from Wikipedians:

Although the raw numbers [of editors] are large, the number of articles is even larger, and so there are not enough editors to go around. […] Where are all the subject-matter experts?

We’d like to think that it’s inevitable we’ll asymptotically approach high quality, as Tony defended with [[Eventualism]]. But I think it’s too simplistc.

In my view, wikipedia has to undergo a paradigm change if it really wants to succeed in creating a good encyclopedia. […] We shouldn’t give up the principle of open editing but we should make clear now from the beginning that we seek good writers and knowledgeable people, not anyone. Yes, anyone can edit an article. But not anyone should edit any article.

If Robert Henry [sic] is right (and judging by a number of fine articles now laying in ruins I suspect he is), then WP, should it desire to get finer control on article quality, needs to modify its “completely open” model a little bit.

[Via Andrew Orlowski at the Register]