Researchers Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg, and Tuhin Chakrabarty released an open‑source repository tied to their new arXiv paper, *Alignment Whack‑a‑Mole*. The code supplies data pipelines, finetuning scripts, and evaluation tools that expose how fine‑tuning pushes large language models to recite copyrighted books verbatim. The demo uses excerpts from Cormac McCarthy’s *The Road*.

The repository relies on Python 3.11, uv for dependency management, and NLTK for text tokenization. Users convert EPUBs into JSON chunks, merge short segments, and generate GPT‑4o summaries that become finetuning prompts. The toolkit supports OpenAI, Vertex AI, and DeepSeek APIs, allowing researchers to train models with 100 completions per excerpt at temperature 1.0 overall.

Evaluation scripts compute four memorization metrics—BMC@k, longest contiguous memorized block, longest raw regurgitated span, and count of spans over a threshold. Cross‑excerpt and cross‑model analyses reveal that different providers, including GPT‑4o, Gemini‑2.5‑Pro, and DeepSeek‑V3.1, tend to memorize overlapping passages, underscoring the reproducibility of the phenomenon across architectures in large language model research today and policy.

The open‑source nature of the code invites scrutiny of fine‑tuning’s ethical limits, especially as vendors offer cheaper, higher‑throughput LLMs. By exposing the mechanics behind verbatim recall, the project equips developers and regulators with concrete evidence, enabling tighter controls on copyrighted content in commercial language models for developers and lawmakers as the industry evolves and regulation.