The murmur of the snarkmatrix…

August § The Common Test / 2016-02-16 21:04:46
Robin § Unforgotten / 2016-01-08 21:19:16
MsFitNZ § Towards A Theory of Secondary Literacy / 2015-11-03 21:23:21
Jon Schultz § Bless the toolmakers / 2015-05-04 18:39:56
Jon Schultz § Bless the toolmakers / 2015-05-04 16:32:50
Matt § A leaky rocketship / 2014-11-05 01:49:12
Greg Linch § A leaky rocketship / 2014-11-04 18:05:52
Robin § A leaky rocketship / 2014-11-04 05:11:02
P. Renaud § A leaky rocketship / 2014-11-04 04:13:09
Jay H § Matching cuts / 2014-10-02 02:41:13

The Journal of Consilience
 / 

Ah: The awesome Public Library of Science just launched PLoS ONE, an open-access scientific journal without boundaries:

PLoS ONE features reports of primary research from all disciplines within science and medicine. By not excluding papers on the basis of subject area, PLoS ONE facilitates the discovery of the connections between papers whether within or between disciplines.

But this is perhaps the most important distinction:

Too often a journal’s decision to publish a paper is dominated by subjective criteria, which can be frustrating and delay the publication of your work. PLoS ONE will publish all papers that are judged to be rigorous and technically sound. Judgments about the importance of any particular paper are then made after publication.

Read that last sentence a few times. That’s kinda the genius of the entire internet, isn’t it? Publish first, filter second!

(Via David Weinberger.)

December 21, 2006 / Uncategorized

8 comments

Whoa — I’m all for the free exchange of ideas, but doesn’t a medical and scientific journal abdicate its authority by abdicating the peer review process? I wouldn’t want my doctor to be prescribing drugs or performing experimental surgery based on the Wikipedia version of The New England Journal of Medicine.

Science and medicine are fundamentally different from other kinds of intellectual work — all they have is the scientific method, which is the basis of peer review (and vice versa). This is particularly true for experimental scientific research — you can wiki/open-source an encyclopedia because it’s about collectively assembling information from other sources, rather than trying to establish facts that aren’t hitherto known.

I guess I just wonder just how the filtration process is going to work, and how practitioners are expected to read/use the information they find there.

(Note: this comment was written in collaboration w/Sylvia, my inside source for the medical library biz.)

Dan says…

Based solely on Robin’s excerpts, I’d say that there still appears to be some kind of peer review. Someone has to determine if the paper is based on work that is “rigorous and technically sound.” The innovation here is more that there is no editor saying: this work isn’t good enough for Nature, try the Journal of Absurd and Generally Questionable Things instead.

And I think it’s even a mistake to hold peer review on too high a pedestal: not only can peer reviewed articles still turn out to be wrong, there are some serious experimental disciplines where the slowness of peer review has led to the creation of extensive pre-print networks whereby important work is circulated informally to the entire research community (or worse, the the research community that is lucky enough to be on the inside).

Yes, they still have peer review — I’m not sure how they define ‘peer’ in this sense, though — so you can’t just get anything published.

But, there’s a larger point — couldn’t peer review take place in a more open environment? Who’s to say you couldn’t make a Wikiscience project that evaluated contributions not on their references and ‘neutral point of view’ but on their rigorousness, replicability, & whatever else peer reviewers consider?

If Dan’s characterization is right, I’m more or less satisfied. I don’t really have much of a beef with an open journal for experimental science so much as medicine. I’m also going on the authority of my spouse, who reports that doctors consulting medical research using the web tend to be lazy and are prone to wind up with outdated and/or inaccurate research. (Google Scholar, whose algorithm tilts towards older artices, exacerbates this problem.) Also — and I admit I have a shaky grasp on what their processes are going to be — the notion of doctors/researchers/companies Google-bombing articles to promote medicines or technologies they have an interest in creeps me out a little bit.

And Dan shoud know that his comment was something of a low blow, since I’m occasionally called upon to write book reviews for JAGQT… but no offense taken. 😉

Dan says…

It’s really all about how you look at it. I mean, you could interpret my comment as a slam against JAGQT, but that would change if I said: “But, hey, at least we didn’t say you should publish in that American rag, Science.

Tim’s comment points at a real tension in our discussion of journals. What are they for? Are they primarily a means by which a small profession of experts shares the results of research and develops programs for further research? Or are they the authoritative pronouncements of experts, imbued with real power in the world?

Obviously, they inhabit both definitions to different extents. Medical journals are a great example of one that carries disproportionate weight as a means of disseminating useful information. I wonder to what extent medical researchers think that is a good thing. My guess is that medical, engineering, and agricultural journals are the most likely to have actual audiences looking for TRUTH, and that other disciplines can get away with being less useful.

At one level, this is why we have experts: to tell other people what is worthwhile to read, and to make distinctions between these two kinds of journal articles. Can an automated system that relies on user ratings or such a thing do the same kind of work? Maybe.

I remember when MIT OpenCourseware came out, Robin and I spent a lot of time thinking about how experts might place their seal of approval on info being made available to the whole, wide world.

One thing that open systems can do, besides expose The Wisdom of the Crowd(tm), is actually help you FIND experts.

I mean, although Wikipedia is aggressively formatted and maintained by a core group of a couple thousand people, most of the content comes from drive-bys — random people seeing a topic they know something about and jumping in.

There’s been a lot of talk lately about the value of networks as resource coordination tools (Sunstein, Benkler). There’s a lot people out there with some special skill or knowledge; there’s a lot of tasks out there that require special skill or knowledge. Bound by geography and formal or social constraints (e.g. you are not a scientist, you cannot publish science) we can’t connect the dots. Open systems, designed and deployed wisely, help us begin to make the connections.

To me, the promise of PLoS ONE is that it may encourage people to publish results that really just aren’t that great, particularly negative results.

Robin, I don’t know if we’ve already talked about this, but Nature did a trial of open peer review this year.

What I would like to see, in addition to more publishing of negative results, is an open approach to data. In my journal of the future, you will have to submit all your raw data along with your manuscript, and if your paper gets published then so do all the data. People will hate me for it at first, but eventually the wisdom of my benevolent socialist data dictatorship will be recognized.

Alf Eaton, who runs Hubmed and generally has a lot of good things to say at the intersection of biology, tech, and pop culture, has a whole different side of appreciation for ONE.

The snarkmatrix awaits you

Below, you can use basic HTML tags and/or Markdown syntax.