A Hacker News discussion explores the best methods for providing continuous context to AI models, specifically asking about Cursor's approach. Users debate whether true continuity exists or if it's simply re-prompting. The core challenge involves managing conversation history efficiently without hitting token limits or incurring high costs, a central problem for developers building complex, stateful applications on top of large language models.

Commenters point out that LLM APIs don't have memory; every interaction is stateless. To maintain a thread, all previous context must be re-sent with each new query. This makes context collation critical for performance and cost. Developers are essentially building their own memory systems on top of the model's API, managing what to include in each prompt.

A key strategy involves leveraging prompt caching, where re-sent context is stored and reused, dramatically reducing costs and latency. One user noted that placing stable context at the beginning of a prompt is optimal for cache hits. Without this, developers face slow responses and expensive token usage, creating a significant barrier for applications requiring long, complex conversations.

Ultimately, the discussion reveals that managing context windows is a manual engineering task. There's no magic solution for persistent memory. Developers must carefully curate what context to carry over between turns, balancing cost, speed, and the model's inherent bias toward tokens at the start and end of its window. The future may bring better tools, but for now, it's about clever prompt engineering.