Five years on, are there any updates on experience
using Mechanical Turk and such for sound perception
experiments?
I've never conducted psychoacoustic experiments myself
(other than informal ones on myself), but now I think I have
some modeling ideas that need to be tuned and tested with
corresponding experimental data. Is MTurk the way to go?
If it is, are IRB approvals still needed? I don't even know
if that applies to me; probably my company has corresponding
approval requirements.
I'm interested in things like SNR thresholds for binaural
detection and localization of different types of signals and
noises -- 2AFC tests whose relative results across conditions
would hopefully not be strongly dependent on level or
headphone quality. Are there good MTurk task structures that
motivate people to do a good job on these, e.g. by making
their space quieter, paying attention, getting more pay as the
task gets harder, or just getting to do more similar tasks,
etc.? Can the pay depend on performance? Or just cut them
off when the SNR has been lowered to threshold, so that people
with lower thresholds stay on and get paid longer?
If anyone in academia has a good setup for human
experiments and an interest in collaborating on binaural model
improvements, I'd love to discuss that, too, either privately
or on the list.