Newsgroups: comp.sources.d
Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!think.com!mintaka!bloom-picayune.mit.edu!athena.mit.edu!jik
From: jik@athena.mit.edu (Jonathan I. Kamens)
Subject: Re: v01INF1: Status - Status of comp.sources.reviewed
Message-ID: <1991Apr14.232818.15851@athena.mit.edu>
Sender: news@athena.mit.edu (News system)
Organization: Massachusetts Institute of Technology
References: <scs.671464069@hela.iti.org> <16390:Apr1305:56:2091@kramden.acf.nyu.edu> <nolan.671566523@helios> <6787:Apr1420:38:3991@kramden.acf.nyu.edu>
Date: Sun, 14 Apr 91 23:28:18 GMT
Lines: 152

In article <6787:Apr1420:38:3991@kramden.acf.nyu.edu>, brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes:
|> Face it: this has absolutely nothing to do with journal publication.
|> The editor is the author's sole contact; reviewers simply do not contact
|> authors. Yet there is not a hint of the traditional anonymity in these
|> guidelines.

  If a journal reviewer decides that there is something ambiguous or unclear
in the paper he is reviewing, he may indicate that in his review, and the
journal may decide that the ambiguity is severe enough that is has to be
corrected (or, of course, they may decide that the paper should be rejected
outright; but that isn't really relevant to the current discussion).  At that
point, the journal will forward the reviewer's comments back to the author,
with all names removed (the anonymity that Dan is defending so vociferously),
and the author has the opportunity to respond to the reviewer's comments if he
feels they are incorrect, or to make corrections if he does not.  (Note that
you can replace "he" with "he/she" in the entire previous paragraph, and the
rest of this posting where appropriate, if you are easily offended by such
things.)

  The way we currently think we'll be doing things in c.s.r, the moderator
will give no indication to the submitter of who the reviewers are.  They will
remain anonymous, unless they choose to contact the submitter directly.  If
they do not want to reveal themselves, and they need to contact the submitter
to ask questions, they can do so through the moderator.  Finally, any comments
that the reviewers have that they think might help people will be included
when the submission is finally posted to c.s.r.

  Mr. Bernstein appears to have three different problems (that I can see) with
this proposed scheme.  First of all, he objects to the idea of the reviewers
ever contacting the submitter at all, anonymous or not.  Second, he objects to
the possibility of unanonymous communication between a reviewer and submitter.
Third, he objects to the idea of reviewers' comments being published when the
submission is posted.

  I see nothing wrong with reviewers contacting the submitter of a package. 
It seems no different to me from the forwarding of reviewers' comments back to
the author in the case of a real journal, and I *know* that this happens, at
least in some cases, because my sister is a biologist who has submitted
several papers to several different journals and has, in several different
cases, been given copies of reviewers' comments (with names reviewed) and
asked to respond to them.  The only difference, it seems, is that the c.s.r
process will be iterative and responsive (i.e. a dialogue between the
reviewers and the submitter), while the journal process is usually one-step
(reviewer gets article, reviewer read article, reviewer writes evaluation,
author gets copy of evaluation).  This strikes me as an improvement over the
journal review process, not a flaw.

  Now, as for the question of anonymity.  Dan may turn out to be right -- we
may end up deciding that it was a mistake to ever consider allowing reviewers
to contact submitters and reveal their identities.  However, thus far the only
reason Dan has given for espousing this belief is that that's not the way real
journals do things.  In my opinion, that's not a very persuasive argument.  If
he would tell us why real journals do reviews anonymously, and then explain
why he thinks those reasons apply equally in the case of c.s.r, I might be
more convinced.

  It seems to me that there are several reasons for anonymity in real
journals.  First of all, you may end up reviewing an article written by a
colleague, and you may not want that colleague to find out that it was you who
panned his article and caused it to be rejected.  Second, the colleague whose
article you review today may end up reviewing one of your articles tomorrow,
and if you give him a bad review today, he may do the same to you tomorrow as
revenge (or, on the other hand, if you give him a good review, he may feel
obligated to give you a good review even if you don't deserve it).  The world
of science is very competitive, and it seems to me that these and all the
other reasons for anonymity are attempts to keep that competitiveness from
contaminating the review process.

  I believe that the review process which we will undertake in c.s.r is *not*
competitive, and was not meant to be.  We do not have 100 programmers out
there, all over the world, working on software in the same ground-breaking
field, and competing with each other to be the first to publish their programs
on the net.  For the most part, when a submitter sends us something, he will
probably be the only person working on anything like that, or one of the view,
and it is very unlikely that he will have to feel a need to compete with other
people working on similar projects.  Furthermore, our criticisms will, for the
most part, be of the "you have to fix these problems before this can be
posted" variety, rather than of the "this program is utterly brain-damaged and
we see no reason to post it to this newsgroup or any other variety" variety.

  What all this comes down to is that, in the majority of cases, there is *no
need* for anonymity in the c.s.r reviewing process.  In cases where there *is*
a need for anonymity (for example in the latter case in the last sentence of
the previous paragraph), it has been guaranteed, because reviewers need never
contact the submitter directly.

  If Dan has some reason why he believes that anonymity in the c.s.r review
process will be harmful despite what I have said above, I would like to hear
it.  If his only reason remains, "That's not how real journals do it!" then,
like I have said, I remain unconvinced.

|> > There are currently two or
|> > three alternatives to comp.sources, c.s.r is just another option.  
|> 
|> That's one of its biggest problems.

  I've never understood this attitude.  Why is giving people more options a
problem?

|> Yes. I invite you---any of you---to explain what comp.sources.reviewed
|> would have done for my latest package, volume24/yabbawhap in c.s.unix.
|> Rich made no changes to it, so it's easy competition, right? Surely you
|> can point out *some* advantage in your precious ``reviewing'' process.
|> All I see is that c.s.reviewed would take weeks where Rich took days.

  First of all, Dan, as I am sure you are aware, arguments of this sort from
the specific to the general hold very little water.  You cannot argue that
because one package would not have benefitted from c.s.r, no packages can
benefit from c.s.r.  The fact that you are attempting to do so seems to me to
cast doubts on your ability to come up with better arguments against c.s.r.

  In any case, the c.s.r reviewing process may have found typos in your
documentation.  It may have found unanticipated OS dependencies that needed to
be removed.  It may have found bugs you were not aware of.  It may have found
any number of problems that neither you or I can anticipate.  Since you seem
to think you know so much about the review process in real journals, you must
know that it is not intended only for the reviewers to find the problems that
the author knows about; it is intended for the reviewers to find any problems
that may be there, including ones that the author may not have anticipated. 
Therefore, to answer your question -- the c.s.r review process may have found
problems that you did not anticipate.

  Now, you can argue, "But there were none!" but that's like saying that the
journal review process is unnecessary because all submitters of papers to
journals can guarantee to the journal in advance that their papers have no
problems in them.

  I will give you two counter-examples in response to your one example.  The
"chemtab" package, as originally posted to comp.sources.unix, had so many
problems that I could not install it on my system when I got it.  The
configuration options didn't, and the compile-time options didn't.  I have
talked to Rich Salz privately about this, and he has told me, "Chemtab was a
mistake."  C.s.r would not, in my opinion, have allowed that mistake to
happen.  After all, I could very well have been one of the reviewers, in which
case I would have caught the problems *before* it was posted, in the review
phase, rather than *after* it was posted, which is what happened when it was
posted to c.s.u..

  Another example is my "delete" package.  When the whole comp.sources.unix
flame war was happening in news.admin, I expounded at length about the
problems I had with my package and comp.sources.unix.  I have talked about
that with Rich Salz too, and he has acknowledged that he made mistakes with my
package and apologized (an apology I accepted willingly; I think Rich is a
great guy, and I certainly don't think that anything he did was intentional or
typical).  I truly believe that *none* of the problems I had with my package
in c.s.u would have occurred had I posted it to c.s.r.

-- 
Jonathan Kamens			              USnail:
MIT Project Athena				11 Ashford Terrace
jik@Athena.MIT.EDU				Allston, MA  02134
Office: 617-253-8085			      Home: 617-782-0710
