HeadlinesBriefing favicon HeadlinesBriefing.com

Why Small Samples Generate False Correlations

Towards Data Science •
×

Correlations drawn from tiny studies, like a 0.62 link between cholesterol and blood pressure in ten patients, can mislead researchers into believing a true relationship exists. The shape of a sample’s correlation distribution depends almost entirely on the number of subjects, not on how many variables are measured.

The author explains that Pearson’s coefficient is the cosine of the angle between two centered vectors. Centering places each vector on an “n”‑1 dimensional hyperplane, while normalizing moves them onto the unit sphere. Gaussian data are rotationally invariant, so after normalization the directions are uniformly distributed on that sphere. This geometry lets the exact sampling distribution of a correlation be derived: it follows a Beta distribution with parameters tied to “n”. The variance scales as 1/(n–1), so typical magnitudes fall near 1/√(n–1). For n=3 the two‑sided 5% threshold is |r|>0.997; for n=4 it is |r|>0.95; for n≥6 the distribution approaches normal.

Practnisse: use the Beta quantiles to set significance thresholds that reflect sample size. High‑dimensional studies do not create spurious correlations; they simply expose them more readily. Code for reproducing the figures lives in the accompanying notebook, offering a hands‑on way to explore these thresholds and avoid over‑interpreting chance patterns.