ReproAI Run Report

ABCD Anchoring studies — "Anchoring-Based Causal Design (ABCD): Estimating the Effects of Beliefs"

Engine: line1-core 0.4.7 · Rules: 2026.06.27 · Audit date: 2026-07-09
Package audited: 2508.01677v1.pdf (Sulitzeanu-Kenan, Mandel & Rinott, Aug 2025) + Replication dataset/ (4 Stata .do files, 4 raw Qualtrics CSVs, 1 .dta helper) — confirmed via OSF API to be a complete local mirror of osf.io/s4ke6
Mode: static + independent R reimplementation. Stata is unavailable on this host and was not installed at the user's request, so the authors' own .do files were never executed. Everything reported as "reproduced" below was regenerated by a from-scratch R 4.6.0 port of each .do file's logic, run against the same shipped CSV/.dta data — this is not a byte-for-byte Stata reproduction.
Originals: untouched (extracted/); all R code is ReproAI's own, in exec_check/R/

Bottom line

	Total	P0 blocker	P1 high	P2 med	P3 low
Findings	9	0	4	2	3

Despite the tooling gap, the empirical core of this paper reproduces essentially perfectly. Every hard number targeted across Tables 2, 3, 4, the Round-3 decay statistics, and the Supplementary Information's polynomial anchor-optimization equations — roughly 60 individual coefficients/SEs/F-stats/N's — matched the manuscript, most to 3+ significant digits. Two of the four shipped .do files contain an undefined-variable reference and cannot run in Stata as posted; fixing the evident typo in each case reproduces the manuscript exactly, so the results themselves are genuine, but the posted scripts are broken. A separate, more serious gap: no code at all is shipped for the Supplementary Information's polynomial anchor-value analysis, which is the actual numeric basis for the paper's central design recommendation.

Independent reimplementation (replaces "live execution")

#	ReproAI script	Ports	Exhibits	Exit
1	`round1_reimplementation.R`	Round_1_Analysis.do	Table 2, part of Table 4	✅ OK
2	`round2_reimplementation.R`	Round_2_Analysis.do	Table 3, part of Table 4	✅ OK
3	`round3_decay_reimplementation.R`	Anchor_Decay_Analysis.do	Figure 3 / p.24 decay stats	✅ OK
4	`round4_multianchor_reimplementation.R`	(no author code exists)	Supp. Info §V polynomial fits	✅ OK

Full console log: exec_check/R/_reproai_console.log. Two lossless fixes were required to make the scripts runnable (documented below and in reproai_reports/advisory_plan.json); both are one-line variable-name corrections, not reinterpretations.

Numerical reproducibility — manuscript vs. R reimplementation

Table 2 (Round 1, N=1012 econ / 1003 donation) — all reported cells ✅

Cell	Manuscript	Reimplementation
OLS: nat. unemployment (ordinal)	−0.013***	−0.01288	✅
OLS: nat. unemployment (%)	0.612***	0.6119	✅
OLS: household finances (ordinal)	−0.013***	−0.01273	✅
OLS: personal unemployment (%)	0.260***	0.2604	✅
OLS: logged donation	0.369***	0.3687	✅
IV: logged donation	0.283** (0.093)	0.2828 (0.0928)	✅
Anchoring effect, recession	3.49 pp*, F=4.96	3.488 pp, F=4.956	✅
Anchoring effect, donation	0.744***, F=321.33	0.7441, F=321.3	✅
Descriptives (all 5 vars, mean+SD)	see table	identical to 3 decimals	✅

Table 3 (Round 2, N=1257 econ / 1232 donation, two-instrument 2SLS) — all reported cells ✅

Cell	Manuscript	Reimplementation
OLS (5 vars)	−0.014/0.609/−0.013/0.224/0.438	−0.0135/0.6092/−0.0133/0.2245/0.4383	✅
IV: nat. unemployment (ordinal/%)	−0.0004(.003) / 0.537***(.084)	−0.00044(.0034) / 0.5368(.0843)	✅
IV: household finances (ordinal/%)	0.003(.005) / 0.229*(.092)	0.00325(.0047) / 0.2288(.0916)	✅
IV: logged donation	0.194*(.098)	0.1943(.0982)	✅
Anchoring effect, recession	16.09 pp***, F=50.06	16.092 pp, F=50.06	✅
Anchoring effect, donation	0.834***, F=173.51	0.834, F=173.51	✅
Binary DV IV: nat. unemployment	β=0.002, p=.316	β=0.001993, p=.3162	✅
Binary DV IV: household finances	β=0.004, p=.049	β=0.003951, p=.0497	✅

Table 4 (placebo/selectiveness tests) — 15/15 coefficients ✅, 1 R² labeling issue ⚠️

Cell	Manuscript	Reimplementation
Round1 col1 (all 4 rows + constant)	−0.009/0.018/0.744*/2.187*	−0.0088/0.0175/0.7445/2.1868	✅
Round1 col2 (3 rows + constant)	−0.774/14.35*/48.24*	−0.774/14.347/48.244	✅
Round2 col3 (5 rows + constant)	−0.042/0.009/0.826*/0.320/2.157**	−0.0419/0.0089/0.8260/0.3197/2.1567	✅
Round1 Adj. R² (col1/col2)	0.241 / 0.075	0.2409 / 0.0754	✅
Round2 Adj. R² (col3)	0.229	0.2261 (Multiple R² = 0.2287)	⚠️
N (col1/col2/col3)	1003/1012/1226	1003/1012/1226	✅

The ⚠️ row is a mislabeling of unadjusted vs. adjusted R² in the manuscript — see finding ADV-007 below.

Figure 3 / p.24 decay statistics (Round 3, N=897 merged) — both reported cells ✅ exact

Statistic	Manuscript	Reimplementation
Lagged recession effect	4.71 pp, p=.010, F=6.59	4.715 pp, p=.0105, F=6.585	✅
Lagged donation effect	0.321, p<.001, F=31.41	0.3209, p=3.2e-08, F=31.41	✅

Supplementary Information §V polynomial fits (Round 4, N=1030) — no author code exists; ReproAI reconstruction ✅ exact to 5 decimals

	Manuscript	Reimplementation
Recession: Y(likelihood)	48.932 −0.47027x +0.01530x² −0.00009x³	48.93205 −0.47027x +0.01530x² −0.00009x³
Recession turning points	min 18.5, max 91.4	min 18.47, max 91.35
Donation: Y(log anchor)	2.2179 −0.72892x +0.44294x² −0.05196x³	2.21791 −0.72892x +0.44294x² −0.05196x³
Donation turning points	min 1, max 4.7	min 0.998, max 4.685

Both equations match to every reported digit — strong evidence the numbers are genuine, even though the code that produced them is missing from the package.

Findings (for the author)

P1 — high

P1ADV-001 — Stata unavailable on this host.

Reviewer/tooling constraint, not an author issue.

P1ADV-002 — No code shipped for Supplementary Information §V.

The two polynomial fits that justify the paper's central anchor-value design recommendation have no corresponding script anywhere in the package (Multi_Anchor_Analysis.do only produces the Figure 4 scatter+lpolyci smooth). Our reconstruction (a plain OLS cubic) reproduces both equations exactly, but a reader has no way to verify the specification without independently guessing it. Fix: publish the fitting code.

P1ADV-003 — Round_1_Analysis.do cannot run in Stata as posted.

It creates log_donation_est_donation_est (doubled-suffix typo) but every later reg/ivregress call references the never-defined log_donation_est. Fix: rename the variable.

P1ADV-004 — Multi_Anchor_Analysis.do cannot run in Stata as posted.

It references donation_est, which doesn't exist in Multi_Anchors_data.csv (the actual column is donation_estimate). Fix: correct the variable name.

P2 — medium

P2ADV-006 — No README, no codebook, hardcoded [path] placeholders in every import delimited line.

Fix: relative paths + README + variable codebook.

P2ADV-007 — Table 4's Round-2 column reports the unadjusted R² (0.2287→0.229) under the "Adj. R-squared" label.

Inconsistent with the Round-1 columns, which correctly report the adjusted value (confirmed: our adjusted R² for that column is 0.226, not 0.229). Fix: recompute/relabel.

P3 — low

P3ADV-008 — Preregistration link inconsistencies.

Footnote 16's Round-3 link (aspredicted.org/see_one.php) doesn't resolve as a real registration; footnote 3's link (NSP_4M7) doesn't match the shipped preregistration PDF's own URL (5hp3-kx39.pdf, #181251). aspredicted.org blocks automated fetching (403), so this is flagged unresolved for manual follow-up.

P3ADV-009 — Minor N discrepancy.

Manuscript states 895 Round-3 completions; shipped data has 897 rows, all of which merge successfully and were used (without affecting any verified statistic).

P3ADV-010 — Vestigial code.

Round_1_Analysis.do's vaccine-belief OLS/IV block isn't tied to any reported exhibit (Table 2 has no vaccine column).

What this package already does right

Every regression that could be checked reproduced almost to the last decimal, including two-instrument 2SLS diagnostics (ivreg's "Weak instruments" F exactly matched Stata's reported first-stage F in all cases) and a from-scratch cubic-polynomial reconstruction — strong evidence the underlying numbers are genuine and untampered.
The shipped id_time_treatment[std_2].dta helper file independently cross-validates our own Round-2 variable construction (its precomputed high_recession/high_donation/estimate fields matched what we derived from the raw CSV).
A confusing variable name (no_anchor_donation built from donate_low_endog!=.) turned out, on cross-tabulation, to be correct — a Qualtrics survey-flow artifact, not a bug. Good reminder that naming ≠ logic; we verified rather than assumed.

What this audit does not guarantee

Exact Stata output was never generated or compared — no .do file was ever executed. Numerical agreement with R is strong evidence of correctness but not proof of byte-identical reproduction (e.g., Stata's exact SSC-package behavior for fre/outreg2/coefplot formatting was never checked).
Design fidelity to Roth & Wohlfart (2020) or other cited studies was not assessed.
The Supplementary Information §V reconstruction is ReproAI's own specification (a global cubic OLS), inferred from the manuscript's prose ("fitted a (3rd-order) polynomial function") — it is not confirmed against author code, because none exists.

Artifacts on disk

Path	What
`extracted/Replication dataset/`	unmodified originals
`exec_check/R/round{1,2,3,4}*.R`	ReproAI's independent R reimplementations, one per round
`exec_check/R/_reproai_console.log`	full execution log (all four scripts exit 0)
`exec_check/_abcd_full_text.txt`	extracted manuscript text (used for numerical verification)
`reproai_reports/*.json`	architecture report, risk register, advisory plan