Posts

Computational Reproducibility in Behavior Research Methods

Image
Assessing computational reproducibility in Behavior Research Methods appears online today.  Behavior Research Methods ( BRM ) is a journal  dedicated to the methodologies, techniques, and tools utilized in psychological research.  We recently examined computational reproducibility in BRM after changes to submission policies. Specifically, a  new policy was introduced for all new submissions on January 1st, 2020, and subsequently made public as part of an editorial: “BRM requires the information to be easily available in a repository or in an appendix” ( Brysbaert et al., 2021, p. 2 ). This project involved a large team that painstakingly sampled a variety of research assets (N=200) before and after those policy changes.  We double-coded what worked and what didn't. We even timed how long it took to get data, code, or stimulus materials up and running.  So what did we find?  The good news is that things are improving. The decay rate is also slower for articles published after cha
Post a Comment
Read more

Academic References for Promotion + some advice

For those unfamiliar with academic promotions (e.g., moving from assistant to associate or to full professor) the process remains labor intensive. It frequently involves lengthy documents where a candidate explains why they are worthy of promotion* alongside assessments from line managers, Deans, and senior academics from other institutions. Once all of these documents are amassed, committees discuss what should happen next. These meetings happen a few times a year.  It's all rather stressful.  As an external person, I’m starting to get more reference requests and these tend to occur around the same time of year. Whenever I feel able to provide something useful, I will prioritize these because they are essential to the candidate and without them, the promotion process can drag on even longer.  All one can do is make an assessment of someone's contributions while keeping in mind the institutional criteria. That said, I still feel universities probably place too much emphasis on
Post a Comment
Read more

Lab Values will only take us so far

The systems that are meant to support research are far from perfect and I'm not just talking about the speed of peer review or a lack of data sharing . By this, I mean how we ensure the accountability and integrity of researchers themselves.   We are currently developing a Lab Values document to agree on what is important to us as a group. This will help define the ways we want to work (see here for a great example). I expect that much of this will align with the values of our institution.  But what about when those values are ignored or deliberately misconstrued?  It is safer to assume that this will happen eventually. Sad to say I've observed many issues closer to home and further away. From individuals being abusive or misogynistic to grandiose authorship claims and data fabrication on an industrial scale.  Such behaviors are likely to hurt someone's career and there are, of course, well-publicized cases where individuals have faced serious consequences. However, by and
Post a Comment
Read more

Exploring aspects of Research Culture: The Open Research Agenda

I've been asked to join a panel discussion about Open Research at the University of Bath . This will be part of a  week, commencing June 20 th , to celebrate and explore aspects of Research Culture at the University.  Before this, I will give an overview of my views on the open research agenda and what it means for research. This is roughly what I will say:  * * * Open and transparent practices benefit individual researchers, groups and institutions. This is beyond doubt. While there is no one size fits all, any attempt to improve the accessibility of scientific papers, data, code or materials is a step in the right direction.  Even small changes appear to have significant positive impacts in the long term. For example,  simply putting papers on pre-print servers appears to reduce the likelihood of retraction .  Open research practices including the pre-registration of analysis plans can also act as a counterweight whereby the scientific enterprise in its current form can allow for
Post a Comment
Read more

Favourite Youtube Technology Channels

As someone who is interested in technology (old and new), here are a few of my go-to YouTube channels.  Broadly speaking, these cover current and historical trends in technologies.  I think what I enjoy about these channels is that the history of technology is yoked to the histories of people and cultures.  AudioPilz  The AudioPilz channel, home of shows like Bad Gear and Better Gear (but mostly Bad Gear;). We all love synthesizers, grooveboxes and drum machines but some of them are hated too. Let's take a closer look. Brace yourself for jams, video mashups, memes, real education, electronic music production enlightenment and all kinds of madness. Andrew Robinson  Journalist & Reviewer covering audio, home theatre, tech & design. Cathodic Ray Dude (CRD)  If I had to pick a theme, I'd say I try to make videos about things we tend not to think can be interesting. Vintage video gear is a big interest, but I've logged far more hours in the computer cockpit. Gaming Histo
Post a Comment
Read more

Digital Detox Bull**it

Following the pandemic, a monstrous number of Digital Detox self-help books are hitting the market.  Personally, I think there is a place for these kinds of books provided they aren't attempting to dish out behavioural science based entirely on a few thoughts inside the author's head.  Unfortunately, books promoting Digital Detoxes do exactly that and more.  First, they assume that technology alone is the problem*.   Technology may very well be the problem, but which bit? Many collective years have been spent studying the literature that attempts to understand links between between general technology use and harm. Alongside several milestone studies, the idea that general technology use has any tangible impact on well-being is pretty much dead in the water. Turns out that's true for large swathes of psychology and behavioural science. Bit annoying, but better to know than not know.     The whole argument can be turned on its head - a lack of access poses a greater risk to i
Post a Comment
Read more

Reflections on Interdisciplinary Research (2013-2020)

I've always found interdisciplinary research rewarding. It also fits with my view that, at least from an applied perspective, no single discipline has all the answers. Probably also says something about my attention span.  Anyway, I'm going to share three things that early-career interdisciplinary researchers in the UK (or who are thinking about working in the UK) might find useful. Somewhat timely as we come to the end of another REF cycle. For those outside the UK, the REF is used to quantify the quality of research outputs across institutions every 7 years or so. It determines how much funding is allocated to each institution by the UK Government.  The below is therefore based on my own and colleagues experiences while conducting interdisciplinary research at research intensive institutions across the UK. 1. Take time to appreciate disciplinary hierarchies because it may impact where you can work in the future. Publication expectations placed on researchers in terms of the R
Post a Comment
Read more