Computational Reproducibility in Behavior Research Methods
Assessing computational reproducibility in Behavior Research Methods appears online today. Behavior Research Methods ( BRM ) is a journal dedicated to the methodologies, techniques, and tools utilized in psychological research. We recently examined computational reproducibility in BRM after changes to submission policies. Specifically, a new policy was introduced for all new submissions on January 1st, 2020, and subsequently made public as part of an editorial: “BRM requires the information to be easily available in a repository or in an appendix” ( Brysbaert et al., 2021, p. 2 ). This project involved a large team that painstakingly sampled a variety of research assets (N=200) before and after those policy changes. We double-coded what worked and what didn't. We even timed how long it took to get data, code, or stimulus materials up and running. So what did we find? The good news is that things are improving. The decay rate is also slower fo...