Measuring the Creativity Potential of LLM Agents
🇬🇧 English
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
🇸🇦 العربية
قياس إبداع وكلاء نماذج اللغة الكبيرة في مهام هندسة التعلم الآلي
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
كيف يُعرّف الإبداع في سياق وكلاء نماذج اللغة الكبيرة لمهام هندسة التعلم الآلي؟
يُعرّف الإبداع على أنه إنتاج أفكار أو منتجات تكون في الوقت نفسه أصلية ومفيدة، ويتم تفكيكه إلى إبداع P وإبداع H والتأثير والقابلية للتنفيذ لقياس الجدة والفائدة عبر نطاقات مختلفة.
🇧🇩 বাংলা
এমএল ইঞ্জিনিয়ারিং কাজে এলএলএম এজেন্টের সৃষ্টিশীলতা মাপা
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
এমএল ইঞ্জিনিয়ারিং কাজের প্রেক্ষিতে এলএলএম এজেন্টের জন্য সৃষ্টিশীলতা কীভাবে সংজ্ঞায়িত করা হয়?
সৃষ্টিশীলতাকে একই সময়ে মৌলিক এবং উপযোগী ধারণা বা পণ্যের উৎপাদন olarak সংজ্ঞায়িত করা হয়েছে, এবং এটি P-সৃষ্টিশীলতা, H-সৃষ্টিশীলতা, প্রভাব এবং সম্ভাব্যতা-এ ভাগ করা হয়েছে যাতে বিভিন্ন পরিসরে নবীনতা এবং উপযোগিতা মাপা যায়।
🇩🇪 Deutsch
Messung der Kreativität von LLM-Agenten bei ML-Engineering-Aufgaben
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
Wie wird Kreativität im Kontext von LLM-Agenten für ML-Engineering-Aufgaben definiert?
Kreativität wird definiert als die Produktion von Ideen oder Produkten, die gleichzeitig original und nützlich sind, aufgeteilt in P-Kreativität, H-Kreativität, Wirkung und Durchführbarkeit, um Neuheit und Nützlichkeit in verschiedenen Bereichen zu messen.
🇪🇸 Español
Medición de la Creatividad de los Agentes LLM en Tareas de Ingeniería de ML
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
¿Cómo se define la creatividad en el contexto de los agentes LLM para tareas de ingeniería de ML?
La creatividad se define como la producción de ideas o productos que son simultáneamente originales y útiles, desglosada en P-Creatividad, H-Creatividad, Impacto y Factibilidad para medir la novedad y la utilidad en diferentes ámbitos.
🇫🇷 Français
Mesure de la créativité des agents LLM sur les tâches d'ingénierie en apprentissage automatique
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
Comment la créativité est-elle définie dans le contexte des agents LLM pour les tâches d'ingénierie en apprentissage automatique ?
La créativité est définie comme la production d'idées ou de produits qui sont simultanément originaux et utiles, décomposée en P-créativité, H-créativité, impact et faisabilité pour mesurer la nouveauté et l'utilité à différentes échelles.
🇮🇳 हिन्दी
एमएल इंजीनियरिंग कार्यों पर एलएलएम एजेंट की रचनात्मकता का मापन
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
एमएल इंजीनियरिंग कार्यों के संदर्भ में एलएलएम एजेंट के लिए रचनात्मकता कैसे परिभाषित की जाती है?
रचनात्मकता को उन विचारों या उत्पादों के उत्पादन के रूप में परिभाषित किया जाता है जो एक ही समय में मूल और उपयोगी होते हैं, और इसे पी-रचनात्मकता, एच-रचनात्मकता, प्रभाव और व्यवहार्यता में विभाजित किया जाता है ताकि विभिन्न दायरों में नवीनता और उपयोगिता को मापा जा सके।
🇮🇩 Bahasa Indonesia
Mengukur Kreativitas Agen LLM pada Tugas Rekayasa ML
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
Bagaimana kreativitas didefinisikan dalam konteks agen LLM untuk tugas rekayasa ML?
Kreativitas didefinisikan sebagai produksi ide atau produk yang sekaligus original dan berguna, yang kemudian dipecah menjadi P-Kreativitas, H-Kreativitas, Dampak, dan Feasibilitas untuk mengukur keaslian dan kegunaan dalam skala yang berbeda.
🇯🇵 日本語
MLエンジニアリングタスクにおけるLLMエージェントの創造性の測定
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
MLエンジニアリングタスクにおけるLLMエージェントの文脈で、創造性はどのように定義されますか？
創造性は、同時にオリジナルかつ有用であるアイデアまたは製品の生産として定義され、P-創造性、H-創造性、影響、および実行可能性に分解されて、さまざまな範囲での新規性と有用性を測定します。
🇧🇷 Português
Medindo a Criatividade de Agentes LLM em Tarefas de Engenharia de ML
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
Como a criatividade é definida no contexto de agentes LLM para tarefas de engenharia de ML?
A criatividade é definida como a produção de ideias ou produtos que são simultaneamente originais e úteis, dividida em P-Criatividade, H-Criatividade, Impacto e Viabilidade para medir a novidade e a utilidade em diferentes escalas.
🇷🇺 Русский
Измерение креативности агентов LLM в задачах инженерии машинного обучения
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
Как определяется креативность в контексте агентов LLM для задач инженерии машинного обучения?
Креативность определяется как производство идей или продуктов, которые одновременно являются оригинальными и полезными, разбитая на P-Креативность, H-Креативность, Влияние и Выполнимость для измерения новизны и полезности в различных масштабах.
🇨🇳 简体中文
在ML工程任务中衡量LLM代理的创造力
This blog post is based on recent work "Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks," published at COLM 2026 and written with Yunxiang Zhang and Professor Lu Wang at the University of Michigan. The post addresses the disconnect between AI investments in scientific discovery and agent performance on real ML research challenges, noting breakthroughs like Alpha Evolve and claims of solving Navier-Stokes. The authors argue creativity offers a useful lens, defined by Mark A.
Runco and Garrett J. Jaeger as the production of ideas that are simultaneously original and useful. Drawing from creative psychology, the definition breaks creativity into four subcomponents: P-Creativity (novelty relative to program history), H-Creativity (novelty compared to human knowledge), Impact, and Feasibility.
The main question analyzes whether performance differences between agent frameworks can be attributed to how they structure and guide the creative search process. The post further breaks originality down into P-Creativity and H-Creativity following Boden, and usefulness into impact and feasibility following Chan and Schunn. The framework aims to quantify how creativity emerges and evolves within different agent scaffolding systems to explain performance gaps in ML engineering tasks.
在LLM代理用于ML工程的背景下，创造力是如何定义的？
创造力被定义为同时具备原创性和实用性的想法或产品的生成，并细分为P-Creativity、H-Creativity、影响力和可行性，以在不同范围内衡量新颖性和实用性。