A real-time fall detector project initially reported a 94.3% accuracy rate, but the results were an illusion caused by data leakage. By using a standard random split on sequential video frames, the model was simply memorizing nearly identical frames rather than generalizing to new environments.

Upon rebuilding the system with a proper Group KFold approach, the true accuracy plummeted to 69.1%. Further investigation revealed a second error: label leakage. The original training labels were derived from folder names rather than precise frame-level annotations, meaning the model was being trained on massive amounts of non-fall data incorrectly labeled as falls.

After rebuilding the dataset using ground truth annotations and adding velocity features, the model's performance improved significantly. The final evaluation shifted from frame-level metrics to event-level results to better simulate real-world production. The revised model achieved a 96.0% fall detection rate, proving that honest evaluation is essential for systems people depend on.