HeadlinesBriefing favicon HeadlinesBriefing.com

LLM random number test shows human‑like bias in gpt‑4.1

Hacker News •
×

Researchers at exmergo uploaded a GitHub repo testing gpt-4.1 with the prompt “pick a random integer between 1 and 100”. They queried the model 10,000 times via the OpenAI Responses API at temperature 1.0, collecting raw integer outputs for statistical analysis. The experiment builds on Reddit and Veritasium analyses of human number‑picking bias, providing an AI comparison.

A chi‑square test rejects uniformity with χ² = 15,604 (p≈0), confirming deviation. The distribution spikes at 37, 42, 73 and clusters around numbers ending in 7, echoing human tendencies, while multiples of ten—except 10—receive zero selections. These patterns mirror heuristics where people favor odd, non‑round values, a bias the model reproduces. The meme‑laden 69 appears far less often than expected, suggesting guardrails shape the output.

The repo provides a reproducible pipeline—collect, clean, transform, then stats—so anyone can rerun the analysis without an API key using the committed CSVs. Fresh runs cost roughly $2 for ten thousand calls to OpenAI’s model. Such quirks matter for developers who might consider LLMs for Monte Carlo simulations or cryptographic seeding. The findings underline that LLMs inherit human‑shaped randomness, limiting their use as unbiased random generators.