Re: hold on a minute....

["John G. Neuhoff" <jneuhoff@xxxxxxxxxxxxxxx>]

Bob,

I am not convinced that the previous posters were suggesting that the thoughts
of a wise and prestigious researcher are suitable replacements for peer review
and experimentation.  If they were, then I agree with your post whole
wholeheartedly.  Experimentation is essential.  It allows us to draw
data-based conclusions of cause and effect.  Peer review has the effect of
increasing our confidence in the validity of these conclusions.

However as I understand it, the project proposed by Al Bregman is not attempt
to draw any conclusions.  The value of this type of project is heuristic.  I
am puzzled as to what the reviewer might review.  Would a reviewer be prepared
to say with confidence that a demonstration would not generate an important
and interesting experiment from which one could draw data-based conclusions at
some time in the future? I think that those reviewing the grant proposal need
to make their funding decision based on how likely they believe it is that
this project will spur others on to meaningful research projects.  I do not
believe this type project requires review.  A web site of this sort would not
really be considered "in the scientific literature" , and the issue of
individual differences can be sorted out in the future experiments.

With that said, it is not a productive move to go back to the grant reviewers
and say that we believe you are wrong. Thus, to get the project funded, I like
Stephen McAdams’ idea for online review from the entire auditory community
much like the Amazon system for reviewing books.

-John Neuhoff


Bob Carlyon wrote:

> Dear all,
>
> As one from the anally retentive ASA school I was a bit concerned about
> some of the comments regarding al bregman's suggestions. So here's my view
>
>  Demos are a great way of giving an audience a subjective feel for a
> phenomenon. But they are no substitute for experiment, nor for peer review.
> So I can see a value for making some demos avalable on the web, but only as
> spurs for those with the time and inclination to do the experiments and
> write them up. It is these papers, which should be published in reviewed
> journals, that would then deserve to be cited.
>
> Onbe reason for my comments is that, although some demos are heard the same
> way by everybody, there are bound to be inter-subject differences -
> especially in the less anally-retentive areas that Al works in (and to
> which I personally am somewhat belatedly heading). I think it would be an
> enormously retrograde step to bypass the review process and have people
> referring to a "demonstration" that is only perceived by a subset of
> listeners, or that has inadequate controls for co-varying aspects of the
> stimulus. Of course, experiments with the latter weakness sometimes slips
> through in refereed journals, but there at least you have the CHANCE that a
> couple of expert reviewers will spot a fatal flaw. I was particularly
> concerned by the statements that because Al is so distinguished he doesn't
> NEED a reviewer. Yes he's distinguished, no that doesn't mean he doesn't
> need, like everyone else, thorough external appraisal of his work before it
> enters the scientific literature. I think we got past accepting the
> "sayings of wise men" in lieu of rigorous experimental investigation some
> time ago
>

I have moved!  Please note my new contact info.
_______________________________________
John G. Neuhoff
Department of Psychology
The College of Wooster
Wooster OH 44691

Voice: 330-263-2475
FAX:   520-244-5577

http://pages.wooster.edu/jneuhoff/index.htm

Alternate email: jneuhoff@mediaone.net