Experimenting with Large Language Models (LLMs) to predict biotech stock movements, a new benchmark called BioTradingArena has been developed. Because biotech stocks are highly sensitive to events like FDA decisions and clinical trial readouts, the benchmark assesses how well LLMs interpret these catalysts. The platform includes 317 historical catalysts, focusing on Oncology, with plans to expand the dataset.

Interpreting biotech catalysts requires specialized knowledge. This benchmark provides a valuable tool for evaluating the ability of LLMs to understand complex information and predict market reactions. The platform uses historical data, trial information, and market expectations. The results showed that a linear regression model using LLM-quantified features yielded the most reliable approach.

The benchmark includes both the catalyst and data available before the press release. The developers are exploring prompting strategies and model families. The goal is to build a playground to test different strategies. This is a novel approach, and offers an interesting look at applying AI to financial analysis by quantifying qualitative features.

What's next? The team plans to add more catalysts, and refine prompting strategies. The use of LLMs in finance is an active area. As LLMs improve, their ability to analyze complex data and predict market trends will increase. This project provides a valuable resource for AI developers and finance professionals.