Open Review Tracker
← Back to Manuscript Reviews

The influence of cognitive load on the anchoring effect in facial attractiveness

Manuscript Reviews

Recommended citation (APA 7)

Röseler, L. (2026, February 19). Review: The influence of cognitive load on the anchoring effect in facial attractiveness [Peer review]. Open Review Tracker. https://lroesele.zivgitlabpages.uni-muenster.de/reviews/manuscript-reviews/review-jddm-anchoringcognitiveload.html (accessed September 16, 2026).

Manuscript title: The influence of cognitive load on the anchoring effect in facial attractiveness
Invitation to Review date: 19.02.2026
Review submission date: 23.02.2026
Review type: original submission / revision / other

I have reviewed the manuscript “The influence of cognitive load on the anchoring effect in facial attractiveness”. The authors describe a study that featured attractiveness ratings in the presence of anchors and high or low cognitive load. I have read the entire paper once. I spent 2.5 hours on the review.

Evaluation: I was very happy to learn about ongoing research in the area of anchoring and assimilation and contrast. I think the article has the potential to be published at JDDM but think that several larger issues need to be addressed first: I did not understand the relevance of system 1 and system 2 or assimilation and contrast. I think the study would equally well with a focus on the insufficient adjustment model or as a conceptual replication of Epley and Gilovich’s cognitive load study. Moreover, though possibly in accordance with your journal’s policy, the article does not meet basic open science standards. Data, code, and materials should all be shared openly, and software and versions should be cited. I have listed remarks including recommendations below and I hope that they help the authors improve the manuscript.

Thank you very much for inviting me to review this manuscript.
Lukas Röseler

PS: I am aware of the fact that a lot of the studies that I cite below include me as an author. I encourage the editor to chime in if they think that this is self-promotion and if the authors should cite other findings instead. Given that I am working on a permanent contract, I can ensure the editor that I do not need citations of my work to promote my career.

Recommendation: Revision required

----------------------------------------------------

All my remarks in no specific order

  1. I think the System 1 and 2 distinction comes from Kahneman’s “Thinking fast and slow” but do not know how relevant it is at the moment; I suggest that you consider stronger embedding this argument in the dual-processing model literature (e.g., https://doi.org/10.1177/1745691612460685). I think your introduction would also benefit from arguments that explain the automaticity of anchoring (e.g., how strategies to counteract anchors often fail).
  2. The abstract starts out with discussing anchors and system 1 vs 2 and ends on concluding that facial attractiveness judgments are system 1 judgments. I was a bit surprised given that the attractiveness ratings seemed more like an arbitrary medium to test the hypothesis. I suggest you clarify why you chose facial attractiveness judgments – if there is a special reason for using those that is. Interestingly, attractiveness ratings do not seem common for anchoring effects (at least, we don’t have them in the OpAQ dataset https://metaanalyses.shinyapps.io/OpAQ/)
  3. Inconsistent use of terminology: You sometimes say anchor effect and sometimes anchoring effect, I recommend that you use the same term throughout the manuscript
  4. Given that the 1974 paper by K&T includes a lot of biases, I recommend that you add the page number to the second citation (p. 1128?) even though APA guidelines do not require it in the absence of a direct quote
  5. I had to read this sentence a couple of times until I understood it: “Put simply, assimilative effects are from the previous response given whereas contrasting effects result from the previous stimulus/face.” Maybe it is just me, but if not, I suggest adding another sentence or reformulating it around something like “A respondent’s previous response exerts an assimilative effect on the target stimulus whereas the previously presented stimulus exerts a contrasting effect on the target stimulus.”
    Thank you also for giving more fuel to my notion that assimilation and contrast are everywhere (https://doi.org/10.31234/osf.io/krwcn). Although I stopped doing research in that area, I’d be very interested in your thoughts on finding common ground for many different A&C theories (though not necessarily as part of your paper). In this case, and in light with the “overlap model” we are citing, the possibility to determine differences very quickly visually may be responsible for contrast and the abstract notion of numbers may be responsible for assimilation.
  6. You argue that “individuals … adjust less from the anchor when their mental resources are depleted” 🡪 There are in fact two opposing findings and views on this by Banker et al. (2017) versus Francis et al. (2018), arguing that depletion leads to more versus less anchoring. We did a registered report on this, which produced a null finding: https://doi.org/10.1016/j.jesp.2019.103942 I suggest that you mention this discussion in the section on cognitive load. I see how it is strange to say “the hypothesis that I am testing is probably wrong” but I think it is stranger to selectively cite supporting evidence (though I am of course assuming that you were probably just not aware of these findings).
  7. I was surprised to see you cite our 2024 replication of Epley and Gilovich in the section before cognitive load but not at the point where you cite their findings. I suggest that you cite the multiple failures to replicate Epley and Gilovich’s findings alongside their findings: https://www.ojs.jcr-econ.org/index.php/jcre/article/view/30 Note that we also list other failures to replicate them and explain how the decade-lasting arguments between Mussweiler and Englich (Selective Accessibility Model) versus Epley and Gilovich and Simmons, LeBoeuf, etc. (Insufficient Adjustment Model) have produced a wealth of inconsistent findings that all turn to null-findings in replications and that the Insufficient Adjustment Model is probably wrong (https://doi.org/10.15626/MP.2024.4137).
  8. “The current study aims to explore the effect of external anchors on judgements of facial attractiveness to see if social perceptions are equally as vulnerable to this bias as other decisions.” 🡪 I think that to actually test this, you would need to include stimuli for other decisions alongside social perceptions. I suggest writing “if social perceptions are also vulnerable to this bias.”
  9. “Furthermore, we will investigate whether cognitive load exacerbates any anchor effects, which would suggest that System 2 acts to adjustment away from the anchor that biases System 1.”
    1. As above noted, I think a lot of information about this theory is missing. I do not even think that you are testing this theory. For example, what would results look like that disconfirm that System 2 is counteracting anchoring? What results pattern would suggest that System 1 is counteracting anchoring? I think it would have been easier to come up with a theory of anchoring that is independent of the dual processing models or simply test an assimilation-contrast-theory. I do not suggest changing the theory or hypothesis a posteriori because that would be HARKing. I recommend that you discuss these limitations. If you decide to drop the theory part due to the inconsistency, I would expect you to transparently state that this is an exploratory study.
  10. Methods: The Gorilla Experiment Builder figure is a bit difficult to read. I recommend you replace it with a figure that only contains crucial information (e.g., not the labels in the bottom right corner).
  11. Open Materials: Thank you very much for sharing the Gorilla files with licenses. Please also include a readme file explaining the different files and upload everything in a non-commercial repository that supports long-term archival, ideally an OSF project that you register (= create a permanent and frozen version) after acceptance or a Zenodo upload.

Methods section

  1. In the 2-back task, did you exclude letters that resemble numbers (most importantly “O”)? Maybe add a sentence about that.
  2. Can you please add details on how the anchors varied? Were they drawn from a uniform distribution that is between 16 and 35?
  3. Just a sidenote: I find it interesting that you used slider scales. We had one study with slider scales that produced a contrast effect (high anchor led to lower responses), which led us to the overlap model with concrete/perceived versus abstract/thought anchors. The study is number 2 of this article (https://doi.org/10.15626/MP.2024.4137).
  4. Design: “a one-way …”: I think that one-way ANOVA refers only to a one-factorial model: https://en.wikipedia.org/wiki/One-way_analysis_of_variance. Since you have anchor and cognitive load as factors plus the DV, it should be called a two-factorial anova.
  5. Please cite all software that you used (e.g., Gorilla, G*Power, SPSS) including their versions.
  6. Power analysis:
    1. Please report the power analysis so that it is reproducible. Ideally, provide the power protocol that is generated by G*Power. To me, it is not clear what type of power analysis you conducted (a priori, a posteriori, sensitivity) nor could I reproduce the values that you report.
    2. I also recommend that you add that the f = 0.66 is for the interaction effect of cognitive load * anchor (right?).
    3. Given the lack of details from the power analysis, I do not understand how you arrived at 81. Please clarify whether this is simply the number of people that you managed to recruit or whether you planned to recruit exactly 81 (or more than X) participants.
  7. Please add information about participant payment (e.g., participants were not paid).
  8. If you have not done so, you should provide more details for the ethics number, though this is something I would expect the journal to verify due to the anonymized manuscript.
  9. Please explain that the study was not preregistered, or, if it was, provide a link to the preregistration (e.g., OSF can provide anonymous links). If the study was preregistered, you also should discuss all deviations from the preregistration and even say so if there was no deviation (which is almost never the case).

Results

  1. Just a small remark: I find it a bit odd to test for outliers when the DV is restricted (0-100).
  2. Table 1 and 2: Can you please add sample sizes to the groups (e.., in the heading or note)?
  3. I recommend also reporting confidence intervals for all crucial effect sizes as recommended by the APA’s Journal Article Reporting Standards. Note that these should be one-tailed 90% Cis for ANOVAs, since F-tests are always one-tailed.
  4. The unit is missing for the Tukey HSD test, please add it (“M = “ or delta?).
  5. Please openly share your data and code so that others can reproduce your results. If you used the SPSS GUI and not the Syntax, you can access the output file and view the code snippets or open the .jrnl file to trace the exact steps that produced the results. Shared data should also have an open license, be hosted on a non-commercial repository with a long-term archival policy, and adhere to FAIR principles (findable, accessible, interoperable, reusable).
  6. “There was some evidence that cognitive load may also influence attractiveness ratings, as there was a small and marginal effect such that increased load increased ratings.” I am not up-to-date with these discussions, but I think “marginal significance” and “close to significance” are not used anymore. Given that the effect was only significant after removing outliers, I would also argue that it is negligible.
  7. “attractiveness perception is vulnerable to external anchors” I would be more careful and say “attractiveness ratings”. For perception, I would expect a paradigm that tests if anchors could lead people to choose different persons to ask for a date or some effect in a juxtaposition of “anchored faces”.
  8. “In the case of judging attractiveness, however, there is no correct answer as people just give their subjective opinions. The fact that the anchor effect still works in such cases is noteworthy.” I disagree. There have been lots of studies asking people to estimate something that does not have a true value (e.g., judgments of how long somebody should go to jail [https://doi.org/10.1177/0146167205282152]; see also “preference” in Table 3, p. 22 in our meta-analysis [https://doi.org/10.31234/osf.io/wf2tn] or the corresponding data [https://metaanalyses.shinyapps.io/OpAQ / https://doi.org/10.5334/jopd.67 ], and in Schley and Weingarten’s Meta-Analysis [https://doi.org/10.31234/osf.io/teq7k_v2]).
  9. “So perhaps different methods of administering an anchor may influence the type of effect generated“: We do know that continuous versus dichotomous anchors both work and have large effect sizes (see meta-analyses above). I recommend that you report the Cohen’s dz for anchors and compare it with that from Kramer and Pustelnik.
  10. Again, the non-significant effect of cognitive load is most likely not due to the different types but due to E&G’s finding not being replicable.
  11. “This is evolutionarily plausible since making mating decisions has been an adaptive concern for far longer than language has existed, indeed far longer than all of human history. It makes sense, therefore, that humans still largely use an ancient, modular system when judging attractiveness. “ 🡪 I am not an expert on evolutionary biology but I am aware that not everything that we observe is due to survival of the fittest. For example, culture can lead to selection mechanisms that have little to do with survival and there are statistical effects that can increase the prevalence of characteristics independent of their relevance for survival (e.g., https://en.wikipedia.org/wiki/Genetic_drift). I thus do not think that this section is needed in the discussion.

Limitations

  1. I suggest that you report the number of correct responses in the cognitive load task to get a grasp of how difficult it was.
  2. I suggest that you check the effect of the order of the cognitive load treatment. Maybe if CL came first, participants were also depleted in second task.

Further comments

  1. Some references do not adhere to the APA format, e.g., DeBruine, Lisa (spelled-out first name) and some are missing DOIs.
  2. Relevance of assimilation vs contrast for the hypotheses: I do not really understand what the role of this theoretical space is for the present article. I had expected you to return to it in the discussion but you did not. I suggest you clarify this.

Methodological Checks (see also https://www.opennessinitiative.org/the-initiative/)

  1. Preregistration
    1. Existence: No
    2. Specificity: NA
    3. Inclusion of Analysis Plan (i.e., R-code): NA
    4. Discussion of all deviations: NA
  2. Open Data
    1. Existence: No
    2. Code sheet: No
    3. Raw AND Processed data: No
  3. Open Code
    1. Existence: No
    2. Open Source Software: Partly (G*Power vs. Gorilla and IBM SPSS) 🡪 for future studies, I recommend open source software such as R, Python, JASP, or JAMOVI
    3. Version control or disclosure of versions: No
    4. Well-commented: NA
  4. Methods
    1. Incentivization of participants: Undisclosed
  5. Results
    1. Statcheck results: 4 hits for F-Tests, all are consistent
    2. Indicators of QRPs: No
  6. Open Materials
    1. Analysis code: No
    2. Stimuli or questionnaires: Yes, though on a commercial platform; I recommend uploading them to a non-commercial repository with a policy for long-term archival. APA requires data to be available for five years after publication (e.g., https://www.apa.org/pubs/journals/resources/data-sharing, https://fairsharing.org/5519)
    3. Open source software: No (see above)
  7. 21 word solution or anything similar (e.g., file-drawer statement): No; I recommend adding a statement disclosing whether further studies have been conducted to test a similar studies and if so, why they are not part of the report
  8. Replicability
    1. FLoRA Annotator results: No hits
  9. Pre-print
    1. [does not apply due to DOA journal]

Disclaimers

  1. I did not conduct a reproducibility check since neither data nor code were available.
  2. I did not check for deviations from the preregistrations in detail since the study did not report about any preregistration.
  3. Publication of this review: I am uploading all my reviews to a public repository. This is to keep track of them and to support open and transparent research assessment: OSF reviews project

Recommendations for the journal

  1. I recommend that the journal conducts reproducibility checks. For example, see Meta-Psychology (“Statistical Reproducibility”, https://open.lnu.se/index.php/metapsychology/about or https://codecheck.org.uk).
  2. I recommend that, after acceptance, the authors create a frozen version of the OSF project to prevent data from being altered or deleted or archive all materials via the journal.

PRO Initiative Statements

  1. I request that the authors add [or retain] a statement to the paper confirming whether, for all experiments, they have reported all measures, conditions, data exclusions, and how they determined their sample sizes. The authors should, of course, add any additional text to ensure the statement is accurate. This is the standard reviewer disclosure request endorsed by the Center for Open Science [see http://osf.io/project/hadz3]. I include it in every review.
  2. I also suggest preregistering all future studies (e.g. on osf.io). Note, on osf.io, one can anonymize the document so blind reviewers can see it.
  3. I only ever recommend accepting a paper if I have access to the full survey materials (in absence of exceptional circumstances). I recommend posting them for reviewers and readers if the authors have not already done so.
Download originalWord document