Let’s go beyond “the effect of”: reappraising the impact of ordinary activities on cognition [Review]

Review of the manuscript, “Let’s go beyond ‘the effect of’: reappraising the impact of ordinary activities on cognition”, for Psicológica. This article discusses five issues that must be taken into account in order to make an unbiased, well-grounded, and nuanced assessment of the evidence that is proposed to make judgments about the benefits of certain activities on cognitive abilities. The article is well written, well organized and well argued. I find it a very good contribution for experimental psychologists. The main problem I find with it is that I'm not sure that it fits well into the scope of Psicológica and if the readers who could benefit the most from reading it are the audience of this journal. But that's not an assessment I should make in my role as a reviewer. It will be the executive editor who must assess whether it falls within that scope. As far as the content itself is concerned, I have a few issues I'm going to raise, as contributions aimed at possible improvements to the manuscript. Below I do detail also some errors and more specific aspects. Main points: In my opinion, too little attention is given to potential conflicts of interest associated with the economic benefits of businesses related to these alleged effects in a meta-analysis. The benefits can go beyond the most obvious: https://robertocolom.wordpress.com/2020/11/30/sobre-los-conflictos-deintereses-en-psicologia-por-juan-botella/ I would mention this issue in several places, and not only in the conclusions. Communicating something more than the dichotomous decision on significance seems today already a must. The obvious alternative among scientists is to report the effect size. The authors propose to extend some old known alternatives. But previous attempts at its implementation have been unsuccessful. The BESD, for example, is almost 40 years old, but it hasn't quite taken root. Perhaps because indeed they do not facilitate the interpretability of the results as much as it might seem. The same goes for others that are mentioned. As an example, it is said "...60.3% of individuals above the mean of the other population...". This magnitude may seem large, but indeed it is not, since the "floor" (or no effect) would be 50%. It can also be misleading. Something happens with the alternative formats. I suggest including a couple of sentences acknowledging that these indices have a long history and yet have not been successful. Perhaps it is time to come up with other, more friendly ones. One of the most relevant issues for meta-analysts is the heterogeneity of the true effects under study. It is often forgotten that confidence intervals refer to the “mean effect”, but when fitting a random effects model (routinely assumed in psychology) you are assuming that the true effect is not always the same. Therefore, it should be made explicit that the effect of these type of interventions can be very heterogeneous, and when the range is moderately large, it must be reported that the genuine effect will sometimes be small, null, or even that the intervention may be detrimental (Higgins, Thompson, & Spiegelhalter, 2009; Inhout et al, 2016; Riley, Higgins, & Deeks, 2011). This point should also be discussed in the article. Minor points: Page 2, line 8 of the second paragraph. Put “e.g.” before the author “Mehr”, since the cited article only refers to musical training and is therefore only one example of the possible ones. In figure 1, the letters “A”…”E” are used to refer to the five aspects to be highlighted, while numbers are used in the text. The consistency in these formal aspects facilitates an organized reading; please, use numbers here too. In both figures the font is too small and difficult to read. Please, enlarge it. Page 5, second paragraph, line 5. I think "amount of hours" fits better than "number of hours". Page 7, line 10 from bottom. I think there is some mistake here, or perhaps I have not understood what is intended to say. I think you should put “…be inflated as a result of…” instead of “…be inflated or the result of…”. References Higgins, J. P., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re‐evaluation of random‐effects meta‐analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1), 137-159. IntHout, J., Ioannidis, J. P., Rovers, M. M., & Goeman, J. J. (2016). Plea for routinely presenting prediction intervals in meta-analysis. BMJ open, 6(7), e010247. Riley, R. D., Higgins, J. P., & Deeks, J. J. (2011). Interpretation of random effects meta-analyses. BMJ, 342.