Data and methods
How the corpus was assembled, how LLM agents were used, and where to find the Online Resources.
Scope and search
This review covers research published or posted from November 1, 2022, to September 27, 2026, in which an LLM judges, chooses, forecasts, advises, trades, or participates in a financial market. Eligibility followed a shared protocol and scoping review conventions (Arksey & O’Malley, 2005; Tricco et al., 2018). The protocol, reproduced in Online Resources 1 and 3, was not registered; boundary rules were refined at consolidation and applied to the 155 records on which the search arms disagreed (Online Resource 2). Five categories qualify: synthetic economic subjects, market simulations, task-performing agents, audited financial judgments, and validity and methodology. Studies without a financial task qualify if they test constructs central to behavioral finance, such as risk preferences, loss aversion, anchoring, and rationality. Studies using LLMs only as text-feature extractors, sentiment scorers, or question-answering systems were excluded, as were non-English records and studies in which only humans made decisions. An asset forecast qualifies when the model’s judgment is evaluated. Decision systems qualify when an LLM’s judgment, forecast, proposed strategy, or advice enters an evaluated decision, even if a rule, optimizer, or learned policy executes it. Online Resource 2 provides the full eligibility and boundary rules and the completed reporting checklist of Tricco et al. (2018).
Three search arms, each run independently by a separate LLM agent, followed a shared protocol: a source-first search of arXiv and journal records via Crossref; a concept-first search using web queries, OpenAlex, SSRN, and citation chasing; and a seed-first search of citation networks and working paper series. Citation-neighborhood searches, re-screening of excluded records, and Consensus queries supplemented these arms. The search covered records public through September 27, 2026; the final pass on that date could not query arXiv or OpenAlex directly and reached SSRN only through Crossref, so coverage of the final days is less complete (Online Resource 2). After deduplication and eligibility checks under the rules in Section 2.1, the corpus contained 599 papers. Fig. 1 shows the screening flow, and Online Resource 2 details the sources, queries, and counts at each stage.
Screening flow
Fig. 1 of the review (PRISMA-ScR).
- Re-screen of the arms' excluded lists: +7
- Citation neighborhoods: 7,585 neighbors, 3,265 unscreened, 570 candidates: +14
- Consensus database pass: 64 queries; 1,245 records; 619 new candidates: +40
- Project reference collection and one-step citation chase: +7
- Direct addition: +1
- Targeted Consensus check: +2
- Update search, 24–27 Sep: 540 arXiv entries, 717 Crossref DOIs, 180 Consensus hits: +9
222 read in full
220 read in full
4 also in the synthesis set
The search ledger has gaps: 1,250 seed-first abstract exclusions were counted but not itemized, and 11 source-first exclusions were unaccounted for. Another 24 records were excluded because their abstract or primary record could not be obtained. Title-stage exclusions were counted rather than being listed. Consequently, screening cannot be reconstructed record by record. The arms supply independent discovery routes rather than independent human screening judgments. The author manually validated the stage counts in Fig. 1 and the final corpus against the search ledger but did not re-screen excluded records. Therefore, this review is a transparently bounded scoping review rather than an exhaustive systematic review. The query log completeness is documented in Online Resource 2.
Section 2.2.
How LLM agents were used
From Section 2.3 of the review.
The LLM agents conducted the review under written protocols. Claude models (Anthropic, September 2026) performed the searches, screening, extraction, bibliographic verification, coding, count computation, figure programming, and initial drafting; the retained session logs recorded the model identifiers (claude-fable-5-1 for the search arms and consolidation; claude-opus-5-5 for coding, the system audit, verification, and revision). Sampling parameters were left at the platform defaults and not recorded. The written protocols are reproduced in Online Resources 1 and 3, and the task prompts of the retained agent runs are available from the author. Separate agents ran the search arms, consolidated the records, and produced the evidence files; agents that had not produced the evidence then checked each section’s claims against its sources.
A model from another developer (GPT-6 Astra, OpenAI, September 2026; recorded by model name only) performed a targeted omission assessment using Consensus and assisted with source verification at the recorded reading depths. Its audit of the 360 component codes of the 30 price-setting designs revised 16 of them. Codex (OpenAI, October 2026) recoded a stratified sample of studies for the cross-model check described below, and adjudicated the count-relevant disagreements from that check against the sources. Online Resource 1 records the evidence locators, and Online Resource 2 documents the workflow.
The author set the research question and scope, ruled on the corpus and on cases escalated before the synthesis, developed and revised the argument and manuscript, and takes responsibility for its content. The author also manually validated the counts, figures, evidence locators, and section claims reported before the October 9, 2026 publication update against the underlying sources and evidence files.
Online Resource 1 itemizes the documented checks. The author’s 56 item-level verdicts comprise 17 checks of the codes behind the headline counts and 24 stratified spot checks of synthesis judgments, all of which confirmed the coded values, and 15 rulings on boundary and eligibility cases, of which one recoded a component and three excluded records. One ruling, which kept a human-benchmark code negative on the agents’ reading of Shah et al. (2025b), was later superseded by the source adjudication described below. Later checks recoded five facet values and corrected one transcribed number (sheet “Recodes after check”).
As a cross-model check, Codex agents recoded a stratified random sample of primary studies from their sources (seed 20261009; Online Resource 1, sheet “Cross-model check”); their working records show no access to the Claude agents’ codes. The sample comprised all 30 price-setting designs and 25 studies from three other strata; one design is excluded because a later Codex update also revised its codes in Online Resource 1. Four Codex agents coded separate batches, and other Codex agents rechecked the codes against the sources, changing 13 of them.
Across the eleven facets of the remaining 54 studies, agreement with the Claude agents’ codes ranged from 65% for independent elicitation to 100% for dated model snapshots. Cohen’s kappa was 0.82 for price formation, 0.79 for the human-surrogate role, and 0.74 for the human comparator (Online Resource 2, Table S4). On price formation, Codex coded 28 of the 29 sampled price-setting designs as endogenous (it could not access the remaining one) and none of the other 25 studies. For the six audited components, 153 of 162 codes agreed across the 27 price-setting designs that both sets of agents read in full. Most disagreements arose in multi-agent designs, where Codex also coded independent elicitation (18 studies) or network transmission (8) that the Claude agents had not; among the 27 designs, it also coded more as having a same-design human benchmark (9 rather than 5), as comparing dispersion with humans (5 rather than 3), and as varying the trading mechanism (4 rather than 3). These figures measure agreement between two AI coding processes rather than inter-rater reliability, and Table S4 reports them as scored, before adjudication.
On October 11, Codex agents checked the seven component disagreements that could raise a reported count against the cited source versions. Two codes changed: Shah et al. (2025b) compare numerical bid–value gaps with published human results under the same auction design, and Hansen and Lee (2025) compare an updating market-maker price with a fixed one. The other five remain negative under the written definitions (Section 8; Online Resource 1, sheet “Source adjudication”). The remaining disagreements, nearly all in facets, were not adjudicated.
Online Resources and data
- Online Resource 2: methods and supporting evidence (PDF) (444 KB)
- Online Resource 1: extraction workbook, public copy (XLSX) (2.0 MB)
- Online Resource 3, the scripts that reproduce the counts, tables and figures, runs on Online Resource 1 and is not posted here.
- Corpus records: bibliographic, taxonomy and screening fields for the 599 papers (CSV) (408 KB)
- Synthesis coding: the facet coding of 275 of the 276 synthesis entries; one is withheld at the authors’ request (CSV) (553 KB)
- System audit: the evaluation-practice coding of the 295 decision-system papers (CSV) (451 KB)
- Coded results: the 1,185 coded results of the synthesis set, with their locators, not counting the withheld entry (CSV) (475 KB)
- References: the works cited in the review (BibTeX) (90 KB)
- Source checks: the 360 checks behind the price-setting audit's codes: source, version read, locator, evidence and status, from Online Resource 1 (CSV) (176 KB)
- Release manifest: the version of each part of this release, with the size and SHA-256 checksum of every download (JSON) (2.0 KB)
Statements and declarations
- Funding
- No funding was received for conducting this study.
- Competing interests
- The author has no relevant financial or non-financial interests to disclose.
- No funding was received for conducting this study.
- The author has no relevant financial or non-financial interests to disclose.
- Not applicable.
- The data analyzed during this study are included in this published article and its supplementary information files, except the agents’ per-paper working files and the task prompts of individual agent runs, which are available from the author on reasonable request. Online Resource 1 provides the 599-paper corpus with bibliographic verification records, search and exclusion ledgers, coding protocols, consolidated synthesis and system-audit tables, and the source audit of 12 components for each of the 30 price-setting designs, with their counts under alternative treatments of uncertain codes, the cross-model comparison, and the dated record of its source adjudication. It records evidence locators, assessable denominators, and access limitations for the study. Online Resource 2 provides extended methods, system descriptions, comparative evidence, and coding definitions condensed from the main text.
- Online Resource 3 contains the scripts that reproduce the corpus counts, evidence tables, figures, coverage-sensitivity scenarios, and cross-model agreement statistics from Online Resource 1, along with an execution guide. The scripts recompute these results from the recorded codes; they do not rerun the searches or reproduce the agents’ coding judgments. The task prompts of the retained agent runs are available from the author upon request.
- During the preparation of this work, the author used Claude (Anthropic, Opus 5.5) to help edit the author's manuscript text for clarity and concision, check each section's claims against its sources, and write the scripts that produce the figures. The author also used GPT-6 Sol (OpenAI) to review the manuscript, identify omissions, and verify sources. Codex also recoded a stratified sample of studies for a cross-model check and adjudicated the resulting count-relevant disagreements against the sources. These models' use as agents in the search, screening, extraction, bibliographic verification, and coding of the reviewed literature is part of the review's methodology and is documented in Section 2.3 and Online Resource 2. The author reviewed and edited all AI-assisted output as needed and takes full responsibility for the published article. The author manually validated the results reported in the manuscript against the underlying sources and evidence files, except where Online Resource 2 documents otherwise.
How to cite
Garcia, John (2026). From Individual Decisions to Collective Outcomes: A Scoping Review of LLM Agents in Financial Markets. Working paper, revised October 15, 2026. Available at SSRN: https://ssrn.com/abstract=7549841
@techreport{garcia2026,
author = {Garcia, John},
title = {{From Individual Decisions to Collective Outcomes: A Scoping Review of LLM Agents in Financial Markets}},
institution = {School of Management, California Lutheran University},
type = {Working paper},
year = {2026},
number = {SSRN 7549841},
url = {https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7549841},
}
About the author
John Garcia
School of Management, California Lutheran University, Thousand Oaks, CA, USA
About this site
This companion is generated from the manuscript (version of October 15, 2026) and from Online Resource 1 (the current version). Its text is quoted from the review, and its counts are computed with the same rules as Online Resource 3. It uses the California Lutheran University palette with the Raleway, Source Serif 4 and Oswald typefaces (SIL Open Font License).
- Paper
- Working paper, October 15, 2026; SSRN 7549841
- Corpus
- frozen on September 27, 2026
- Online Resource 1
- the current version
- Website release
- October 15, 2026