LLM agents in economic games Research
Putting LLM agents into real economics experiments, alone or next to human players, to see what kind of society they form when they share a pot, punish and vote on their own rules.
- Role
- Research and code
- Period
- Mar–Jun 2026
- Team
- Solo, with AI coding agents
- Stack
- Python · oTree · EDSL · Docker
- Impact
- Replicated a classic public-goods experiment with LLM agents
- Field map
- awesome-llm-social-simulation
Why
An economic game is a small society you can measure: players decide how much to put into a shared pot, whom to punish and which rules to live by. Put LLM agents into one and you can watch what society they build, how personality changes it, and where it settles. A run with agents finishes in under an hour, and published human studies of the same games give a fixed point to compare against. replicant lets agents play the same oTree pages a human participant sees, so their results can be set next to a published experiment.
What I did
- replicant is a Python package that runs LLM agents with Big Five personalities as participants in real oTree experiments, alone or next to human players, and keeps multi-player rounds in sync. With it I replicated Ertan, Page & Putterman (2009), a public-goods game where players vote on whom they may punish, and the results are written up in a paper (co-authored with Claude).
- Personality check: I measured whether a personality description in the prompt actually shifts the model's Big Five on a second, independent test. It mostly did, with a visible lean towards being nice.
- edsl-test compares eight persona methods and the bare model on the same game, against its human data. Each method first had to reproduce its own source paper. I later extended it into a wider set of one-shot economic games with checked human reference values.
- metron treats deployed AI agents as a survey population: it gives personality tests to published agent persona templates to compare them with human norms. Alongside it there is an agent-compiled map of the field as linked notes.
Continued in simengine.
Results
- Default agents reproduced the paper's main finding: they voted to allow punishing low contributors and never to allow punishing high contributors. Prosocial personalities raised contributions and cut costly punishment.
- Personas built on real human data or calibrated to theory formed the top group; the bare model sat in the lower group with the other methods. Most of the gap was about rules: the best methods kept the rule that lets players punish low contributors, while the bare model always dropped it.
- Scores moved a lot between random seeds, so a ranking from a single run can mislead.
My favourite result: very agreeable agents overthink. On the hardest ethical questions most of them used up their whole reasoning budget deliberating and never answered.
Gallery
Next
The replication is single-shot, while the original study runs over many rounds where players learn and react to each other. The paper names a multi-round version with agent memory as the biggest open gap. Smaller follow-ups it lists are re-running the very agreeable agents with a larger reasoning budget, and repeating the experiment on a non-reasoning model.
