Re: [AUDITORY] online data collection for auditory experiments

[Samuel Mehr <sam@xxxxxxxxxxxxxxx>]

Also Pushkin, which plays nicely with jsPsych https://link.springer.com/article/10.3758/s13428-018-1155-z (this is how our site runs, except for the Musical IQ test, which embeds a Shiny server in Pushkin so that we can run Peter's psychTestR). The most we've ever paid for hosting on AWS is I think ~US$400 for one month, but that was under very high traffic. Our costs are more typically US$100/month, and the current per-participant rate is about a tenth of a cent.

--

Samuel Mehr

Department of Psychology

Harvard University

Be a citizen scientist at themusiclab.org!

On Fri, Apr 10, 2020 at 7:55 PM Peter Harrison <p.m.c.harrison@xxxxxxxxxx> wrote:

Hi Felix,

 

This is a great list! For R-Shiny, you could add shinyapps.io (https://www.shinyapps.io/) as a hosting solution, and psychTestR (https://pmcharrison.github.io/psychTestR/) as a framework for designing experiments in R-Shiny.

 

Best wishes,

Peter

 

From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> on behalf of Sarah Creel <000001009817b89f-dmarc-request@xxxxxxxxxxxxxxx>
Reply to: Sarah Creel <screel@xxxxxxxx>
Date: Friday, 10 April 2020 at 06:01
To: "AUDITORY@xxxxxxxxxxxxxxx" <AUDITORY@xxxxxxxxxxxxxxx>
Subject: Re: [AUDITORY] online data collection for auditory experiments

 

I would also add Finding Five (https://www.findingfive.com/) to this list. --Sarah

 

On Thu, Apr 9, 2020 at 2:05 AM Felix Dobrowohl <F.Dobrowohl@xxxxxxxxxxxxxxxxxxxx> wrote:

Hi Jessica and List,

 

We’ve been compiling a resource of online experiment platforms for behavioural auditory research which might be of use for some of you.

Note, experiment builders which are listed as hosting by Pavlovia can potentially be hosted elsewhere.

 

Have a good long weekend everyone,

 

Cheers,

Felix

 

Integrated Service providers 

Name	GUI	hosting	Cost	Comment	URL	Tutorials/Templates
Qualtrics	yes	provided	~2500$/year	Survey builder with integrated audio/video support. Limitations for randomisation. For surveys only.	Qualtrics.com	Qualtrics Tutorials
Gorilla.sc	yes	provided	1.08 USD per participant	Online experiment builder with drag & drop GUI but limitations for randomisation among others.	Gorilla.sc	gorilla.sc tutorials
Testable	Yes	provided	Mixed payment. Potentially free	Very simple experiment builder for basic behavioural experiments. Automatic credit granting via SONA may not be possible.	Testable.org	Testable tutorials
Inquisit Web	No	provided	200 USD/month	Scriptable experiment platform, which requires a client installation from the participant, but in turn provides the potentially best control and precision	Inquisit Web Link	Inquisit templates
Labvanced	Yes	provided	1080 EUR/quarter	Versatile interface and large amount of tutorials and support online	Labvanced.com	Labvanced Video tutorials

 
Experiment builder software (free) 

Name	GUI	Coding language	hosting	Comment	URL	Tutorials,  Templates
PsychoPy3	Yes	Python	Pavlovia	Similar in structure to e-prime, pavlovia implemented	PsychoPy.org	Link
PsychoJS	No, works in conjunction with PsychoPy	_javascript_	Pavlovia	_javascript_ library for PsychoPy experiments	psychoJS	Link
JsPsych	No	_javascript_	Pavlovia	_javascript_ library	JSPsych.org	Link
PsyToolkit	No	own	provided	Basic script-based experiment builder with its own scripting language	PsyToolkit.org	Link
NodeGame	No	_javascript_	self	_javascript_ based experiment builder with option for multi-participant designs	NodeGame.org	Link
Tatool	Yes	_javascript_	self	Simple experiment builder, supporting both a GUI or _javascript_.	Tatool-Web.com	Link
LionessLab	Yes		self	A simple online experiment builder that allows for interactive experiments	Lioness-Lab.org	Link
O-tree	No	Python	self	Python based experiment builder with option for multi-participant designs	OTree.org	Link
LabJS	Yes	HTML, _javascript_	Pavlovia	User friendly HTML experiment builder with some GUI functionality	LabJS	Link
OpenSesame	Yes	Python, _javascript_	self	Online experiment builder with minimal GUI, using both _javascript_ and python	OpenSesame	Link
R-Shiny	No	R	self	R package to build interactive web apps from R Studio, extendable with CSS, html and _javascript_	Shiny Rstudio	Link

  

Hosting platforms 

Name	Cost	Comment	URL
Pavlovia	1500 pounds/year	Hosting platform by the University of Nottingham, designed for PsychoPy/PsychJS but also hosts LabJS and jsPsych	Pavlovia.org
Amazon Web Services	variable	Web hosting service by Amazon	AWS
Firebase	Storage: 0.03$/GB  Traffic: 0.15$/GB	Web hosting service by Google	firebase.google.com
Heroku	variable		Heroku.com

 

Containers (runtime environments required to run experimental code on servers) 

Name	Cost	Comment	URL
JATOS	Free, open source		JATOS.org
Experiment factory	Free, open source		ExpFactory.github.io

 

 

Felix Dobrowohl | Senior Technical Officer - MARCS Institute

Western Sydney University – Bldg 5.G.57 Bankstown Campus

P: +61 02 9772 6723

E: F.Dobrowohl@xxxxxxxxxxxxxxxxxxxx

 

westernsydney.edu.au

 

From: AUDITORY - Research in Auditory Perception [mailto:AUDITORY@xxxxxxxxxxxxxxx] On Behalf Of Jessica Louise Slater, Ms
Sent: Thursday, 9 April 2020 12:07 AM
To: AUDITORY@xxxxxxxxxxxxxxx
Subject: [AUDITORY] online data collection for auditory experiments

 

Dear list,

 

Like many labs, we are currently evaluating options for running auditory experiments online. I wondered if others would be willing to share their experiences with online data collection, or resources they have found helpful - particularly pertaining to the challenges of auditory stimulus presentation (e.g. timing precision)? We currently have adaptive tasks running in Matlab so are especially interested to hear about experiences using Matlab-based solutions, but any insights very welcome. 

 

Best,

 

Jessica    

 

 

--

Sarah C. Creel
Professor, Cognitive Science
UC San Diego
9500 Gilman Drive
La Jolla, CA 92093-0515
http://quote.ucsd.edu/creel