HeadlinesBriefing favicon HeadlinesBriefing.com

Graphique de dépendance de prompt réduit les efforts de réessai

Towards Data Science •
×

I built a pure Python prompt dependency graph to answer what needs re-evaluation after changing a shared component. It calculates two metrics: Reachable (everything downstream) and Candidate (direct dependents plus their downstream). Tested on a 55-node synthetic system, section-aware tracking narrowed evaluation sets by 0% to 85% depending on sharing selectivity.

The key insight: changing one prompt can affect up to 50 others, but impact is unpredictable without running the graph. I changed a base-policy sentence extending refunds from 30 to 14 days and needed to know which agents required re-testing. Running the full suite was expensive, and guessing was risky.

The graph provides structural answers for change impact analysis in prompt engineering, a practice borrowed from software engineering. The methodology identifies what should be evaluated, not what will fail—behavioral impact still requires actual testing. Results show narrowing varies widely: base-policy/refunds change reached 45 nodes but only needed 24 evaluated (47% narrowing), while tone/professional showed zero narrowing (55/55).

The real takeaway is that composability increases blast radius unpredictably, making dependency tracking essential for efficient validation.