Posts

When Comfort Replaces Curiosity

There will come a point where a survey can only take you so far. This is especially true when it contains moderation or mediation models. These problems are serious yet are routinely ignored and remain widespread. It's one of those classic cases where the statistical machinery is more powerful than the data it is being applied to. The models answer questions that a single survey will never be equipped to ask. None of this means surveys are useless, but it does mean we should be much more honest about what they can and cannot tell us. What makes this more interesting is that researchers often seem more content to try to drag more out of a survey or psychometric instrument, rather than switch up their methods.  I remain struck by how rarely researchers move beyond their comfort zone, either methodologically or by stepping away from their home discipline. When this happens, it is rarely done with any real conviction. Largely because it involves questioning many methodological and phil...
Post a Comment
Read more

Is the narrative CV dead already?

The narrative CV was supposed to be the great equaliser. A chance to move beyond the blunt metrics of h-indices, publication counts, and journal impact factors. A chance for researchers to tell a story. This would help bring to life the messy, non-linear, deeply human realities of a career in academia. On paper, the idea was noble. In practice, I suspect something else has happened entirely. I say suspect, because I am not aware of any research that has assessed whether the narrative CV has improved or hindered progress to date. The Promise When UKRI and other major funders began rolling out narrative CVs, the rationale was compelling. Traditional CVs had long been criticised for rewarding a narrow definition of academic success: more papers, bigger grants, higher-ranked journals. This format systematically disadvantaged early career researchers, those who had taken career breaks, researchers working in applied or interdisciplinary fields, and anyone whose contributions didn't fit ...
Post a Comment
Read more

The Publication Vortex

When academics leave academia, the reasons mirror most career transitions: the misalignment between personal values and organizational culture becomes too great. But academia has a particular way of manifesting this misalignment. It promises freedom and autonomy, the ability to pursue questions that matter to you, while simultaneously creating conditions where that freedom feels increasingly illusory. I'm not talking about workload modelling or administrative creep, though those matter. I'm talking about something more fundamental: academia's unique relationship with control, or rather, the illusion of it. The Predictable and Unpredictable in Academic Life Here's what we can predict in academia: the pressure to publish will continue to exist. The volume of output will continue to grow at a seemingly impossible rate. Quality will remain variable as a result. What we cannot predict, and what many struggle to accept, is nearly everything else. Which journals will accept yo...
Post a Comment
Read more

Computational Reproducibility in Behavior Research Methods

Image
Assessing computational reproducibility in Behavior Research Methods appears online today.  Behavior Research Methods ( BRM ) is a journal  dedicated to the methodologies, techniques, and tools utilized in psychological research.  We recently examined computational reproducibility in BRM after changes to submission policies. Specifically, a  new policy was introduced for all new submissions on January 1st, 2020, and subsequently made public as part of an editorial: “BRM requires the information to be easily available in a repository or in an appendix” ( Brysbaert et al., 2021, p. 2 ). This project involved a large team that painstakingly sampled a variety of research assets (N=200) before and after those policy changes.  We double-coded what worked and what didn't. We even timed how long it took to get data, code, or stimulus materials up and running.  So what did we find?  The good news is that things are improving. The decay rate is also slower fo...
Post a Comment
Read more

Academic References for Promotion + some advice

For those unfamiliar with academic promotions (e.g., moving from assistant to associate or to full professor) the process remains labor intensive. It frequently involves lengthy documents where a candidate explains why they are worthy of promotion* alongside assessments from line managers, Deans, and senior academics from other institutions. Once all of these documents are amassed, committees discuss what should happen next. These meetings happen a few times a year.  It's all rather stressful.  As an external person, I’m starting to get more reference requests and these tend to occur around the same time of year. Whenever I feel able to provide something useful, I will prioritize these because they are essential to the candidate and without them, the promotion process can drag on even longer.  All one can do is make an assessment of someone's contributions while keeping in mind the institutional criteria. That said, I still feel universities probably place too much emphas...
Post a Comment
Read more

Lab Values will only take us so far

The systems that are meant to support research are far from perfect and I'm not just talking about the speed of peer review or a lack of data sharing . By this, I mean how we ensure the accountability and integrity of researchers themselves.   We are currently developing a Lab Values document to agree on what is important to us as a group. This will help define the ways we want to work (see here for a great example). I expect that much of this will align with the values of our institution.  But what about when those values are ignored or deliberately misconstrued?  It is safer to assume that this will happen eventually. Sad to say I've observed many issues closer to home and further away. From individuals being abusive or misogynistic to grandiose authorship claims and data fabrication on an industrial scale.  Such behaviors are likely to hurt someone's career and there are, of course, well-publicized cases where individuals have faced serious consequences. How...
Post a Comment
Read more

Exploring aspects of Research Culture: The Open Research Agenda

I've been asked to join a panel discussion about Open Research at the University of Bath . This will be part of a  week, commencing June 20 th , to celebrate and explore aspects of Research Culture at the University.  Before this, I will give an overview of my views on the open research agenda and what it means for research. This is roughly what I will say:  * * * Open and transparent practices benefit individual researchers, groups and institutions. This is beyond doubt. While there is no one size fits all, any attempt to improve the accessibility of scientific papers, data, code or materials is a step in the right direction.  Even small changes appear to have significant positive impacts in the long term. For example,  simply putting papers on pre-print servers appears to reduce the likelihood of retraction .  Open research practices including the pre-registration of analysis plans can also act as a counterweight whereby the scientific enterpris...
Post a Comment
Read more