Misc

There will come a point where a survey can only take you so far. This is especially true when it contains moderation or mediation models. These problems are serious yet are routinely ignored and remain widespread. It's one of those classic cases where the statistical machinery is more powerful than the data it is being applied to. The models answer questions that a single survey will never be equipped to ask. None of this means surveys are useless, but it does mean we should be much more honest about what they can and cannot tell us. What makes this more interesting is that researchers often seem more content to try to drag more out of a survey or psychometric instrument, rather than switch up their methods.  I remain struck by how rarely researchers move beyond their comfort zone, either methodologically or by stepping away from their home discipline. When this happens, it is rarely done with any real conviction. Largely because it involves questioning many methodological and phil...

The narrative CV was supposed to be the great equaliser. A chance to move beyond the blunt metrics of h-indices, publication counts, and journal impact factors. A chance for researchers to tell a story. This would help bring to life the messy, non-linear, deeply human realities of a career in academia. On paper, the idea was noble. In practice, I suspect something else has happened entirely. I say suspect, because I am not aware of any research that has assessed whether the narrative CV has improved or hindered progress to date. The Promise When UKRI and other major funders began rolling out narrative CVs, the rationale was compelling. Traditional CVs had long been criticised for rewarding a narrow definition of academic success: more papers, bigger grants, higher-ranked journals. This format systematically disadvantaged early career researchers, those who had taken career breaks, researchers working in applied or interdisciplinary fields, and anyone whose contributions didn't fit ...

When academics leave academia, the reasons mirror most career transitions: the misalignment between personal values and organizational culture becomes too great. But academia has a particular way of manifesting this misalignment. It promises freedom and autonomy, the ability to pursue questions that matter to you, while simultaneously creating conditions where that freedom feels increasingly illusory. I'm not talking about workload modelling or administrative creep, though those matter. I'm talking about something more fundamental: academia's unique relationship with control, or rather, the illusion of it. The Predictable and Unpredictable in Academic Life Here's what we can predict in academia: the pressure to publish will continue to exist. The volume of output will continue to grow at a seemingly impossible rate. Quality will remain variable as a result. What we cannot predict, and what many struggle to accept, is nearly everything else. Which journals will accept yo...

Assessing computational reproducibility in Behavior Research Methods appears online today.  Behavior Research Methods ( BRM ) is a journal  dedicated to the methodologies, techniques, and tools utilized in psychological research.  We recently examined computational reproducibility in BRM after changes to submission policies. Specifically, a  new policy was introduced for all new submissions on January 1st, 2020, and subsequently made public as part of an editorial: “BRM requires the information to be easily available in a repository or in an appendix” ( Brysbaert et al., 2021, p. 2 ). This project involved a large team that painstakingly sampled a variety of research assets (N=200) before and after those policy changes.  We double-coded what worked and what didn't. We even timed how long it took to get data, code, or stimulus materials up and running.  So what did we find?  The good news is that things are improving. The decay rate is also slower fo...

For those unfamiliar with academic promotions (e.g., moving from assistant to associate or to full professor) the process remains labor intensive. It frequently involves lengthy documents where a candidate explains why they are worthy of promotion* alongside assessments from line managers, Deans, and senior academics from other institutions. Once all of these documents are amassed, committees discuss what should happen next. These meetings happen a few times a year.  It's all rather stressful.  As an external person, I’m starting to get more reference requests and these tend to occur around the same time of year. Whenever I feel able to provide something useful, I will prioritize these because they are essential to the candidate and without them, the promotion process can drag on even longer.  All one can do is make an assessment of someone's contributions while keeping in mind the institutional criteria. That said, I still feel universities probably place too much emphas...

The systems that are meant to support research are far from perfect and I'm not just talking about the speed of peer review or a lack of data sharing . By this, I mean how we ensure the accountability and integrity of researchers themselves.   We are currently developing a Lab Values document to agree on what is important to us as a group. This will help define the ways we want to work (see here for a great example). I expect that much of this will align with the values of our institution.  But what about when those values are ignored or deliberately misconstrued?  It is safer to assume that this will happen eventually. Sad to say I've observed many issues closer to home and further away. From individuals being abusive or misogynistic to grandiose authorship claims and data fabrication on an industrial scale.  Such behaviors are likely to hurt someone's career and there are, of course, well-publicized cases where individuals have faced serious consequences. How...

I've been asked to join a panel discussion about Open Research at the University of Bath . This will be part of a  week, commencing June 20 th , to celebrate and explore aspects of Research Culture at the University.  Before this, I will give an overview of my views on the open research agenda and what it means for research. This is roughly what I will say:  * * * Open and transparent practices benefit individual researchers, groups and institutions. This is beyond doubt. While there is no one size fits all, any attempt to improve the accessibility of scientific papers, data, code or materials is a step in the right direction.  Even small changes appear to have significant positive impacts in the long term. For example,  simply putting papers on pre-print servers appears to reduce the likelihood of retraction .  Open research practices including the pre-registration of analysis plans can also act as a counterweight whereby the scientific enterpris...

As someone who is interested in technology (old and new), here are a few of my go-to YouTube channels.  Broadly speaking, these cover current and historical trends in technologies.  I think what I enjoy about these channels is that the history of technology is yoked to the histories of people and cultures.  AudioPilz  The AudioPilz channel, home of shows like Bad Gear and Better Gear (but mostly Bad Gear;). We all love synthesizers, grooveboxes and drum machines but some of them are hated too. Let's take a closer look. Brace yourself for jams, video mashups, memes, real education, electronic music production enlightenment and all kinds of madness. Andrew Robinson  Journalist & Reviewer covering audio, home theatre, tech & design. Cathodic Ray Dude (CRD)  If I had to pick a theme, I'd say I try to make videos about things we tend not to think can be interesting. Vintage video gear is a big interest, but I've logged far more hours in the computer cock...

Following the pandemic, a monstrous number of Digital Detox self-help books are hitting the market.  Personally, I think there is a place for these kinds of books provided they aren't attempting to dish out behavioural science based entirely on a few thoughts inside the author's head.  Unfortunately, books promoting Digital Detoxes do exactly that and more.  First, they assume that technology alone is the problem*.   Technology may very well be the problem, but which bit? Many collective years have been spent studying the literature that attempts to understand links between between general technology use and harm. Alongside several milestone studies, the idea that general technology use has any tangible impact on well-being is pretty much dead in the water. Turns out that's true for large swathes of psychology and behavioural science. Bit annoying, but better to know than not know.     The whole argument can be turned on its head - a lack of acces...

I've always found interdisciplinary research rewarding. It also fits with my view that, at least from an applied perspective, no single discipline has all the answers. Probably also says something about my attention span.  Anyway, I'm going to share three things that early-career interdisciplinary researchers in the UK (or who are thinking about working in the UK) might find useful. Somewhat timely as we come to the end of another REF cycle. For those outside the UK, the REF is used to quantify the quality of research outputs across institutions every 7 years or so. It determines how much funding is allocated to each institution by the UK Government.  The below is therefore based on my own and colleagues experiences while conducting interdisciplinary research at research intensive institutions across the UK. 1. Take time to appreciate disciplinary hierarchies because it may impact where you can work in the future. Publication expectations placed on researchers in terms of the ...

Charging $50 to submit a manuscript.  This is another baffling move from a journal that was already heading in the wrong direction for three reasons: (i) Editorials are all over the place. See a recent letter in response to one from last year.  (ii) The editor-in-chief is not an editor. They don't handle manuscripts or communicate with reviewers. Many reviews are definitely not worth $50 (good, bad or neutral)!  (iii) An increasing number of papers have obvious issues with basic reporting, which the journal clearly doesn't care about.  In some areas of finance and accounting, it is common to have submission fees, which are sometimes refunded following acceptance. However, this is not the norm for psychology (nor should it be).  It looks like the publisher has implemented this on the quiet (kudos to Lee Hadlington for noticing) and is presumably an attempt to handle submission loads.  The number of submissions shouldn't come as a huge surprise.  First...

[see updates at the end of this article] I disagree with a fair chunk of Griffiths and co's work theoretically and methodologically. That's science. But Griffithsgate  goes beyond that and raises some uncomfortable questions about editorial bias and the very real consequences of carelessly applied research (see  Dorothy Bishop's  blog and Tom Chiver's  article in  Unheard ). In saying that, it is tricky to separate procedures from science because the rushed nature of the work means that it is riddled with contradictions. Like a political party trying to avoid the opposition, it is almost impossible to debate a moving target. For example:   Write about why data should be open , but don't share your own when requested .  Write about students and issues concerning plagiarism , then do the very same thing . That's all just procedural remember and long before getting to the actual science. I've previously written about the problems of publishing on an ...

I tweeted about a dilemma earlier this week. It's a familiar tale.  In the last few months, I've reviewed 3 papers for the same journal. I am also a co-author of a paper that is under review at the same outlet.   To be completely transparent, our paper has been reviewed as far as I can tell, but it has now spent more time on a desk than with reviewers. A colleague emailed politely asking if the paper would be sent out for review in Feb after it sat for a month with no activity. I emailed again asking for an update on our paper earlier this month. The journal office claims to forward emails on, but we receive zero response from any editor.  What do you do? The problem with not reviewing is that I am not helping authors who deserve to have their paper reviewed in a timely fashion. On a side note, the very same journal also has a habit of giving reviewers a set number of days to complete a review and then cancelling the review before the due date.  And then they wo...

Writing academic books has become somewhat less fashionable in psychology, but I’ve always been encouraged to do things I genuinely want to do rather than be completely guided by the REF, TEF or KEF etc. That advice has always stuck with me.  The book is now almost finished pending some minor edits and a bit of copyediting. I am aware that a few early versions of the manuscript have gone out to some folk who might say something nice for the back cover. Pretty much all the content is new and, I hope, as up-to-date as a book can be. Some of it naturally pulls ideas from a handful of recent papers. On a side note, it's been an interesting experience to wrestle permissions from publishers so I can re-use portions of text or figures from my own papers! Publication is penned for later this year ( update September 2020: you can buy it now) , but in the meantime here are three general things that have stuck with me throughout the course of putting it together. ...

We recently ran a little study to see what happens when you prevent a small group of students from using their smartphone for 24 hours. Participants were instructed to place their smartphone in a secure evidence bag. So what happened? Well not much really, they missed their smartphone.  As part of this study, we also asked participants to complete the Smartphone 'Addiction' Inventory (SPAI). Interestingly, a few participants who dropped out later in the study had fractionally higher SPAI scores. This may indicate that smartphone ‘addicts’ were unable to fully participate in the study and so discontinued, thus affecting our findings. However, this is unlikely given that smartphone addiction scales  do not align favourably with objective behaviour . It is worth noting that while this small number of participants were slightly more anxious at time 1 they were also on average, a bit happier. It's increasingly difficult to know exactly what these 'addiction...

We recently collected more smartphone usage data to test if pen and paper scales could predict behaviour ( they didn't ). However, in the process we managed to replicate some of our previous results from 2015. Specifically, the average number of smartphone pick-ups per day remains remarkably similar across both samples despite using different software and smartphone operating systems to quantify these behaviours. These results therefore cast some doubt over the idea that Android and iPhone users differ in their usage behaviours ( we previously observed some demographic and personality differences between these two groups ). Mean number of pick-ups from 2015 sample: 84.68 ( SD =55.23) . Mean number of pick-ups from 2018 sample: 85.44 ( SD =53.34) . It's worth remembering that our results in 2015 were already comparable with data collected by others in 2009 ! The idea that people are using their phones more doesn't really hold up to scrutiny.  ...