December 14, 2005
'Pedia Still Astonishingly Awesome
Only eight serious errors, such as misinterpretations of important concepts, were detected in the pairs of articles reviewed, four from each encyclopaedia. But reviewers also found many factual errors, omissions or misleading statements: 162 and 123 in Wikipedia and Britannica, respectively.
I’m pretty darn awed by that.
If you’ve been watching Romenesko’s letters this week, you might have caught Karen Heyman’s letter about Wikipedia’s problems. A snippet:
Unless you already know a field, you can have no idea that an apparently definitive entry presents only one side of an ongoing fight between specialists. That it may be changed, and changed back again, hardly helps matters. This, btw, is the best explanation as to why simply sitting back and saying, “It’s okay now, it’s changed,” ultimately would not have worked for Seigenthaler. Chances are high that later somebody would have come along to “fix” the correction.
Wikipedia is a fantastic idea, a wonderful service, with entries that often reflect great effort and care. Unfortunately, inevitably, as it’s grown, the flaws built into its original design have become more obvious. Egalitarian editing may be a noble goal, but the reality is that if Wikipedia is to truly fulfill its promise, it needs a way to vet contributors, to let users know whether an entry on neuroscience was written and edited by a senior professor, a student who just took Psych 101, or a layperson who’s paraphrasing an old issue of Scientific American. Certainly prankster Brian Chase’s initial belief that Wikipedia was a joke site says a great deal about how some of its entries appear to the general public. If Seigenthaler’s complaint actually leads to more accountability, far from hurting Wikipedia, he may ultimately have saved it.
I’ll cross-post my reply to Ms. Heyman below:
If Karen Heyman’s letter were a wiki, I’d edit the beginning of her concluding paragraph to read: “The newspaper is a fantastic idea, a wonderful service, with articles that often reflect great effort and care. Unfortunately, inevitably, as it’s grown, the flaws built into its original design have become more obvious.”
Why is it that people, especially journalists, always seem to propose fixing Wikipedia’s problems by dispatching more editors to the site? News organizations are rife with editors, yet I’d trust Wikipedia’s coverage of most complex science and tech issues at least as much as your average news report. Which is to say, cum serious grano salis.
Wikipedia’s problem is not that it has become more (or more obviously) flawed as it’s grown, as Heyman says. But it has become such an incomparably useful resource for so many that its flaws are more regrettable. Look back two or three years, and you’d find it informative and accurate in spots, but plainly lacking in key areas. Today, it’s astonishingly comprehensive, with standards of accuracy and balance that very often match or exceed media that are far more established, and a regard for transparency that would put any news outlet I know to shame. That growth couldn’t have happened if Wales and co. had a system in place to vet contributors. Such a system couldn’t scale.
All that is not to dispute the problems that Heyman points out. But Wikipedia is a strange new beast, and old-school measures aren’t going to cut it.
One greater problem that the Seigenthaler incident brings to light is our collective lack of basic media literacy. A prankster like Brian Chase has tons of options to introduce false information into an official-looking public record. It’s a very simple matter to create a fake CNN mockup and disguise a link to it. If Chase’s friend could be shocked by a random sentence on a Web page featuring the words “edit this page” in bold type across the top, chances are it wouldn’t have taken much to fool him.
We should continue thinking of ways to improve Wikipedia; it’s a phenomenal public resource. But we must turn our attention to educating people how to evaluate all sources of information as the amount of it proliferates.
By the way, don’t miss this tidbit from the Nature article:
Wales also plans to introduce a ‘stable’ version of each entry. Once an article reaches a specific quality threshold it will be tagged as stable. Further edits will be made to a separate ‘live’ version that would replace the stable version when deemed to be a significant improvement. One method for determining that threshold, where users rate article quality, will be trialled early next year.
This is a fantastic idea that Wales has been discussing for a while now. I imagine it will work in much the same way the featured article process does today.
When editing Wikipedia becomes more mainstream, especially within the academic community, it will be such an amazing project. A solitary tear runs down my cheek, just thinking about it.