Preregistration of Experiment "Response priming with metacontrast-masked number stimuli" (student project)

1 University of Kaiserslautern-Landau (RPTU) Center for Cognitive Science, Visual Attention and Awareness Laboratory Head: Prof. Dr. Thomas Schmidt, Dipl.-Psych., FPsyS Preregistration on zenodo.org Date: Tuesday, December 9, 2025 Authors: Thomas Schmidt, Jessica Ruby, Diana Hubert, Aikaterini Siskou, Benjamin Fox, Marco Elbert Preregistration: Masked perception of numerical stimuli General approach We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study (“21 word solution”). We do not report conditions, variables, analyses, or participants selectively without transparently revealing the reasoning behind the selection. We avoid the use of analysis techniques with excessive and unwarranted user-degrees of freedom. In our style of experimentation, we subscribe to the psychophysical tradition of “small-N / large-Ni designs” (Smith & Little, 2018) that relies on the careful analysis of individual observers instead of extensive averaging across many observers. We therefore prefer to draw statistical power from a large number of trial repetitions rather than the number of participants (Arend & Schäfer, 2019; Baker et al., 2020). We follow the principle that research articles must be methodologically transparent without requiring additional knowledge of supplementary materials or preregistration documents. We share data with natural persons for scientific purposes in accordance with German data protection laws (Landesdatenschutzgesetz Rheinland-Pfalz). 2 Novel or preexisting data? No data have been collected for this study yet. Purpose of the experiment, major hypotheses and expectations We measure response priming effects in response times and error rates. On each trial, participants are presented with a brief prime display (a number between 1 and 9, excluding 5) followed by a target display (another number from the same set). Trials are congruent if prime and target afford the same response, and incongruent otherwise. Participants are instructed to respond as quickly and accurately as possible to the target by pressing one of two response keys (Target ID task). In another task (Prime ID), they are asked to discriminate the prime as accurately as possible without time pressure. The prime-target SOA is varied in three steps (30, 60, 90 ms). All stimuli are composed of hexagonal elements that are masked by a hexagonal grid of lines (“honeycombs”). This way, we hope to achieve metacontrast masking of the primes. We measure full priming functions where response times and error rates are plotted as a function of prime-target congruency (incongruent vs. congruent conditions) and primetarget SOA (varied parametrically in three steps: 30 ms, 60 ms, 90 ms). We also measure full masking functions of prime discrimination accuracy as a function of SOA. In addition, we want to examine response times for effects of numerical distance between prime and target. We are looking only at numerical distances of three (1-4, 6-9, 3-6, 4-7), since other combinations have more congruent combinations than incongruent ones. For example, the numerical distance of two only has one incongruent combination (4-6) and many more congruent ones (1-3, 2-4, 6-8,…). Our hypothesis is that the same distance leads to more similar RTs within response categories than between response categories. 3 Our major question is whether response priming is affected by using honeycombs for masking numbers. We aim to demonstrate that response priming effects remain largely stable, even when masking functions differ substantially in qualitative terms. Design, independent variables, and dependent variables; variables that are not part of the analysis plan We apply a completely crossed three-factorial repeated-measures design with factors of SOA (3 levels) and prime-target congruency (2 levels). Each participant participates in every condition over a course of two sessions. All analyses follow this same factorial structure. Dependent variables are mean response times and error rates. Apart from the analyzing the mean error rates, we apply hazard analysis and conditional accuracy functions where physical time is subdivided into time bins and the hazard rate of responding as well as accuracy of responses are plotted as a function of time (see our tutorial paper by Panis, F. Schmidt, Wolkersdorfer, & Schmidt, 2020). There are no further dependent variables. Apart from the major independent variables, there are additional variables that are needed for counterbalancing purposes and the avoidance of experimental artifacts. These are the concrete prime identity and target identity (apart from the analysis of distance-3 effects, see above) as well as whether a target is larger or smaller than five on any given trial. Those balancing variables are not part of the analysis plan and are averaged across. The same holds for all participant variables, like gender, age, handedness, or other characteristics. Criteria for data trimming, discarding of data or participants, or selective analyses We rarely have to exclude participants from analysis after data are collected. We only do so if a participant clearly does not follow instructions. Indicators of that are very high 4 error rates (close to chance level) on the target identification task or monotonic use of the same response key. Apart from that, sometimes participants’ data cannot be used because of equipment malfunction, mistaken instructions, or other unforeseen events. In such cases, we try to recover as much of the valid data as we can. However, when a participant decides to abort an experiment prematurely we discard all data from that person. Finally, we exclude trials with problems in experimental timing, such as skipped monitor frames as measured by the timing feedback from the Psychophysics Toolbox. In experiments involving color discrimination, we screen participants for color deficiencies by using Ishihara plates. A frequently used strategy for data analysis in unconscious perception research is to selectively analyze trials or participants with certain performance outcomes (e.g., trials with low visibility ratings or participants not exceeding certain prime discrimination criteria). We are strongly opposed to such practices, which lead to strong statistical distortions and artifacts like regression to the mean, and we never use them. Planned analyses, variable transformations The major analysis strictly follows the three-factorial structure of the independent variables. We apply two-factorial repeated-measures analysis of variances (ANOVA; factors Congruency, SOA) on response times as well as error rates. Practice blocks are discarded. Error rates p are transformed into logits according to the formula logit = log [p/(1-p)], after replacing perfect error rates of 0 and 1 with values of 1/2r and 1 - 1/2r, where r is the number of stimulus repetitions per participant. In other words, half of a response is added to or subtracted from any perfect score to prevent division by zero. In fully crossed repeated-measures designs, we adjust standard error bars for intersubject variability (Loftus & Masson, 1994). Because the statistical models for repeated measures only use the interaction between the participant factor and the effect of interest as an error term, the main effect of the participant factor does not enter the error variance and can be 5 discarded. Error bars should reflect the resulting increase in power to aid the graphical interpretation of the data. We use a very simple adjustment method (Bakeman & McArthur, 1996) where each participant’s data pattern is vertically shifted by an additive constant until all individual means are equal to the grand mean (ipsative data). This way, the intersubject variance is zero while all interactions remain intact. Then standard errors are calculated across the adjusted participants as usual. Statistical decision criteria and multiple tests We generally report statistical tests as conventionally significant at a false-positive risk of α = .05 because many readers use that criterion. However, internally we are more conservative and regard p-values between .01 and .05 as only mild evidence against the null hypothesis and would not base strong conclusions on such results. GreenhouseGeisser correction is used for all tests irrespective of the outcome of a formal sphericity test. Corrections for type-I error accumulation are handled in the following way. We follow the custom of not correcting the set of F tests within a single ANOVA model for multiple tests (usually three main effects and four interactions). We also follow the custom of not correcting F tests across the ANOVAS of different dependent variables. However, when we conduct k multiple tests within the frame of an analysis (e.g., analyzing priming effects separately for each SOA), we use a Bonferroni correction of α’ = 1 - (1 - α)k. Foreseeable follow-up analyses In many experiments, discoveries in the data lead to follow-up analyses. Furthermore, reviewers often request additional analyses. Examples would be learning effects (timecourses of effects across blocks or sessions), sequential effects (e.g., response times following error trials), or delta plots. We always clearly specify which analyses are planned (follow from the design) and which ones are post-hoc (based on discoveries). 6 Not every aspect of data analysis can be preplanned. For instance, it is always an executive decision to choose a good bin size for a histogram, a color code for a heat-map, or the minimum number of trials that allow for plotting a data point in a conditionalaccuracy or hazard diagram. Unexpected problems may lead to unbalanced designs or unusual distributions of data, forcing analyzers to adjust statistical methods. Advances in methodological knowledge and creative development of methods may even lead to the replacement of one technique by a superior one. We believe that this is generally a sign of scientific progress, not of researcher degrees of freedom running wild. We are transparent about such issues and developments in our own analysis strategies and clearly identify them in our publications. Sample size determination In multi-factor repeated-measures designs, statistical power can be calculated if all effect sizes can be predicted along with their respective error variances. In practice, however, too many terms are unknown for a meaningful power analysis. Because the number of trials per participant and condition is about as important for power as the number of participants (Arend & Schäfer, 2019; Baker et al., 2020; Smith & Little, 2018), we control measurement precision at the level of individual participants in single tasks and stimulus conditions (Biafora & Schmidt, 2019; Lakens, 2024). For each task, we calculate precision as s/√r (Eisenhart, 1962), where s is a single participant's standard deviation in a given cell of the basic design (e.g., Consistency x SOA) and r is the number of repeated measures in each cell and subject. We assume standard deviations of SD = 60 ms for response times (based on our own benchmark data) and the maximal possible standard deviation of .5 for error and accuracy rates. For instance, when r = 100, we can expect a precision of 60/√100 = 6 ms in individual response times and 0.5/√100 = 0.05 (five percentage points) in error/accuracy rates. This means that an uncorrected 95% confidence interval calculated within a single observer would be able to resolve mean differences of two precision units, ≈ 2s/√r. 7 In the present experiments, r = 128, and we assume SD(RT) = 60 ms, max. SD(Error) = 0.5. Therefore we expect a precision of 5.3 ms in response times and 4.4 percentage points in accuracy scores of each individual observer and condition. Precision thus exceeds our previous recommendations for response priming studies (r = 60, F. Schmidt et al., 2011). Based on these considerations, we measure 10 observers in all experimental conditions and 128 trials per condition and observer. In our publications we show the data patterns of all individual observers. 8 Literature: Arend, M. G., & Schäfer, T. (2019). Statistical power in two-level models: A tutorial based on Monte Carlo simulation. Psychological Methods, 24(1), 1–19. https://doi.org/10.1037/met0000195 Bakeman, R., & McArthur, D. (1996). Picturing repeated measures: Comments on Loftus, Morrison, and others. Behavior Research Methods, Instruments, & Computers, 28(4), 584-579. Baker, D. H., Vilidaite, G., Lygo, F. A., Smith, A. K., Flack, T. R., Gouws, A. D., & Andrews, T. J. (2020). Power contours: Optimising sample size and precision in experimental psychology and human neuroscience. http://dx.doi.org/10.1037/met0000337 Biafora, M., & Schmidt, T. (2019). Induced dissociations: Opposite time courses of priming and masking induced by custom-made mask-contrast functions. Attention, Perception, & Psychophysics, 82, 1333–1354. https://doi.org/10.3758/s13414-019-01822-4 Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorials in Quantitative Methods for Psychology, 1, 42–45. https://doi.org/10.20982/tqmp.01.1.p042 Eisenhart, C. (1962). Realistic evaluation of the precision and accuracy of instrument calibration systems. In H. H. Ku (Ed.), Precision Measurement and Calibration (1969), (pp. 21–48). Washington, D.C.: National Bureau of Standards. Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1). https://doi.org/10.1525/collabra.33267 Loftus, G. R., & Masson, M. E. J. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin & Review, 1, 476-490. Panis, S., Schmidt, F., Wolkersdorfer, M. P., & Schmidt, T. (2020). Analyzing response times and other types of time-to-event data using event history analysis: A tool for mental chronometry and cognitive psychophysiology. I-Perception, 11, 1-24. doi: 10.1177/2041669520978673 Schmidt, F., Haberkamp, A., & Schmidt, T. (2011). Dos and don’ts in response priming research. Advances in Cognitive Psychology, 7, 120–131. https://doi.org/10.2478/v10053-008-0092-2 9 Schmidt, T. (2000). Visual perception without awareness: Priming responses by color. In T. Metzinger (Ed.), Neural correlates of consciousness (pp. 157-179). Cambridge: MIT Press. https://doi.org/10.7551/mitpress/4928.003.0014 Schmidt, T. (2002). The finger in flight: Real-time motor control by visually masked color stimuli. Psychological Science, 13, 112-118. https://doi.org/10.1111/14679280.00421 Schmidt, T., Niehaus, S., & Nagel, A. (2006). Primes and targets in rapid chases: Tracing sequential waves of motor activation. Behavioral Neuroscience, 120(5), 1005–1016. https://doi.org/10.1037/0735-7044.120.5.1005 Schmidt, T., & Seydell, A. (2008). Visual attention amplifies response priming of pointing movements to color targets. Perception & Psychophysics, 70, 443-455. doi: 10.3758/PP.70.3.443 Smith, P. L., & Little, D. R. (2018). Small is beautiful: In defense of the small-N design. Psychonomic Bulletin & Review, 25(6), 2083–2101. https://doi.org/10.3758/s13423-018-1451-8 Vorberg, D., Mattler, U., Heinecke, A., Schmidt, T., & Schwarzbach, J. (2003). Different time courses for visual perception and action priming. Proceedings of the National Academy of Sciences USA, 100(10), 6275–6280. https://doi.org/10.1073/pnas.0931489100