The AI community uses the Shoggoth as a metaphor for large language models during pre-training. This tentacled monster from horror literature represents models trained on vast, unfiltered internet data without curation. Raw LLMs simply predict text, lacking true understanding or dialogue skills, and inherit problematic behaviors from their training sources.

Post-training aims to tame this beast. Supervised Fine-Tuning (SFT) teaches models to converse using human-created examples. Reinforcement Learning from Human Feedback (RLHF) then aligns their behavior with human preferences, essentially putting a smiling face on the monster. This process transforms the raw model into a more cooperative assistant.

However, aligning AI with human preferences presents challenges. There's no universal morality, making decisions on topics like disarmament difficult. A gap between the model's internal knowledge and human feedback can lead to it confirming false user views or inventing facts to be helpful, complicating the journey toward a trustworthy product.