The recent surge in tabular foundation models like SAP's SAP-RPT-1 has shifted the paradigm from traditional supervised learning to in-context learning (ICL). This approach eliminates costly retraining but introduces critical accuracy-latency trade-offs at inference time. Context payload optimization has become central to balancing response quality with system responsiveness in ICL-based workflows.

The "iron triangle" framework reveals inherent tensions between response quality, inference cost, and latency. Larger payloads improve prediction accuracy by providing more examples for schema inference and pattern recognition, yet they increase response time. Secondary trade-offs emerge between throughput, stability, and monetary costs, particularly in real-time applications like fraud detection.

Optimization strategies span method and moment dimensions. Task-agnostic approaches like random sampling offer simplicity but limited control. Task-aware methods such as KNN-based sampling identify relevant rows similar to query data, yielding stronger performance though requiring computational overhead. The optimal approach depends on specific use case requirements and acceptable latency thresholds.