FRESH: fNIRS REproducibility Study Hub

What happens when you get a fresh set of eyes on the same data set?

Project Overview

Functional Near-Infrared Spectroscopy (fNIRS) is an increasingly popular neuroimaging technique [0] with a diverse range of analysis procedures [1, 5, 6]. Inspired by projects in other research domains [2, 3, 4, 7, 8], this project aims to explore the range of analysis techniques utilised by the fNIRS community, and quantify the variation in conclusions that are drawn when many independent researchers analyse the same dataset. We believe this to be particularly important for the fNIRS community due to the diverse range of analysis procedures, young nature of the technique relative to fMRI, EEG, MEG, and the increasing adoption across a wide range of neuroscience and commercial applications.

Participants in this project will be provided two fNIRS datasets to analyse as they deem most appropriate. The data will be provided with a brief description of the experimental design and research question. Participants will return their results via a structured form to the steering committee who will summarise the findings and produce a journal article in which the contribution of all participants will be recognised. This is not a competition to get the best results, it is a community effort to understand the variability in fNIRS processing.

For up to date details, please view the OpenfNIRS forum: https://openfnirs.org/community/postid/1347/

Registration

To register your interest for this study please complete these steps:

Add your institution if it is not in the list below using this form.
Add each participant using this form.
Register your team using this form. (A team can be a single participant or have many people in the team)

Once you have completed these steps we will send you a confirmation email in the following weeks. You can modify your team and individual details at any time by contacting committee@fresh-fnirs.org. We will reconfirm each team after you submit your results.

Registering does not oblige you to commit to entering the study. It just allows us to contact you and provide you with further information as it becomes available. You can remove yourself at any time.

Timeline

Due to the scale of this project a strict timeline will be adhered to. Participants will have 4 months to analyse the data and submit their results.

October 3rd 2022: Data is released and result submission opens.
February 3rd 2023: Result submission closes.

Steering Committee

Robert Luke: Postdoctoral Research Fellow at Macquarie University, Australia

Rickson C. Mesquita: Associate Professor at the University of Campinas, Brazil

Meryem Ayşe Yücel: Associate Professor at the Neurophotonics Center, Boston University, USA

Alexander von Lühmann: Head of Intelligent Biomedical Sensing (IBS) Lab at BIFOLD, Technische Universität (TU) Berlin, Germany

Expectations

Participants are expected to follow the following guidelines to ensure the smooth running of the project:

Teams must submit their results within the prescribed time frame (4 months).
Multiple teams from the same institution may register for the project.
You can register as an independent, you do not need to be part of an academic institution.
Teams may consist of one or more individual participants.
There must be a significant contribution from each member of the team
This is not a competition. We will not be ranking the performance of participants. The paper will not link the authors/teams to specific results.
Do not discuss your analysis approach, strategy, or outcomes with other teams.
You attempt to address all hypotheses. However, if your usual analysis approach does not allow you to answer certain questions, you can skip them and note this in your response.
You may use the https://openfnirs.org/community forum to discuss practical issues such as loading data.
All participants must conduct themselves in a professional manner and be considerate to other participants and the coordinating team.
Have fun!

Registered Institutions

References

[0] Yücel, Meryem A., et al. “Functional near infrared spectroscopy: enabling routine functional brain imaging.” Current opinion in biomedical engineering 4 (2017): 78-86.
[1] Yücel, Meryem A., et al. “Best practices for fNIRS publications.” Neurophotonics 8.1 (2021): 012101.
[2] Botvinik-Nezer, Rotem, et al. “Variability in the analysis of a single neuroimaging dataset by many teams.” Nature 582.7810 (2020): 84-88.
[3] Silberzahn, Raphael, et al. “Many analysts, one data set: Making transparent how variations in analytic choices affect results.” Advances in Methods and Practices in Psychological Science 1.3 (2018): 337-356.
[4] https://www.eegmanypipelines.org
[5] Pfeifer, Mischa D., Felix Scholkmann, and Rob Labruyère. “Signal processing in functional near-infrared spectroscopy (fNIRS): methodological differences lead to different statistical results.” Frontiers in human neuroscience 11 (2018): 641.
[6] Pinti, Paola, et al. “Current status and issues regarding pre-processing of fNIRS neuroimaging data: an investigation of diverse signal filtering methods within a general linear model framework.” Frontiers in human neuroscience 12 (2019): 505.
[7] Camerer, Colin F., et al. “Evaluating replicability of laboratory experiments in economics.” Science 351.6280 (2016): 1433-1436.
[8] Dreber, Anna, et al. “Using prediction markets to estimate the reproducibility of scientific research.” Proceedings of the National Academy of Sciences 112.50 (2015): 15343-15347.