A dermatologist tested GPT-6 Astra and Claude Fable 5.1 against a 10-year-old CNN for skin cancer diagnosis. The older CNN outperformed both frontier LLMs, correctly identifying 62% of 200 biopsy-proven lesions compared to 42% for the AI giants. Using the MILK10k dataset, the study highlights that a fine-tuned Inception v3 model, trained on public dermatology images, surpassed modern LLMs in accuracy.
Both LLMs misdiagnosed a melanoma as a benign seborrhoeic keratosis, while the CNN correctly identified it. The CNN, running on a phone browser via Web GPU, demonstrates that older, specialized models can outperform cutting-edge general-purpose AI in medical diagnostics.
Source: Hacker News · Summarized by HeadlinesBriefing