Subject: Re: hold on a minute.... From: "John G. Neuhoff" <jneuhoff(at)acs.wooster.edu> Date: Mon, 18 Sep 2000 09:47:38 -0400Bob, I am not convinced that the previous posters were suggesting that the tho= ughts of a wise and prestigious researcher are suitable replacements for peer r= eview and experimentation. If they were, then I agree with your post whole wholeheartedly. Experimentation is essential. It allows us to draw data-based conclusions of cause and effect. Peer review has the effect o= f increasing our confidence in the validity of these conclusions. However as I understand it, the project proposed by Al Bregman is not att= empt to draw any conclusions. The value of this type of project is heuristic.= I am puzzled as to what the reviewer might review. Would a reviewer be pre= pared to say with confidence that a demonstration would not generate an importa= nt and interesting experiment from which one could draw data-based conclusio= ns at some time in the future? I think that those reviewing the grant proposal = need to make their funding decision based on how likely they believe it is tha= t this project will spur others on to meaningful research projects. I do n= ot believe this type project requires review. A web site of this sort would= not really be considered "in the scientific literature" , and the issue of individual differences can be sorted out in the future experiments. With that said, it is not a productive move to go back to the grant revie= wers and say that we believe you are wrong. Thus, to get the project funded, I= like Stephen McAdams=92 idea for online review from the entire auditory commun= ity much like the Amazon system for reviewing books. -John Neuhoff Bob Carlyon wrote: > Dear all, > > As one from the anally retentive ASA school I was a bit concerned about > some of the comments regarding al bregman's suggestions. So here's my v= iew > > Demos are a great way of giving an audience a subjective feel for a > phenomenon. But they are no substitute for experiment, nor for peer rev= iew. > So I can see a value for making some demos avalable on the web, but onl= y as > spurs for those with the time and inclination to do the experiments and > write them up. It is these papers, which should be published in reviewe= d > journals, that would then deserve to be cited. > > Onbe reason for my comments is that, although some demos are heard the = same > way by everybody, there are bound to be inter-subject differences - > especially in the less anally-retentive areas that Al works in (and to > which I personally am somewhat belatedly heading). I think it would be = an > enormously retrograde step to bypass the review process and have people > referring to a "demonstration" that is only perceived by a subset of > listeners, or that has inadequate controls for co-varying aspects of th= e > stimulus. Of course, experiments with the latter weakness sometimes sli= ps > through in refereed journals, but there at least you have the CHANCE th= at a > couple of expert reviewers will spot a fatal flaw. I was particularly > concerned by the statements that because Al is so distinguished he does= n't > NEED a reviewer. Yes he's distinguished, no that doesn't mean he doesn'= t > need, like everyone else, thorough external appraisal of his work befor= e it > enters the scientific literature. I think we got past accepting the > "sayings of wise men" in lieu of rigorous experimental investigation so= me > time ago > I have moved! Please note my new contact info. _______________________________________ John G. Neuhoff Department of Psychology The College of Wooster Wooster OH 44691 Voice: 330-263-2475 FAX: 520-244-5577 http://pages.wooster.edu/jneuhoff/index.htm Alternate email: jneuhoff(at)mediaone.net