In AI, the 'efficient frontier' manages tradeoffs between cost and capabilities for models. A model is 'frontier' if it offers highest intelligence at given cost or size. Two inference engineering techniques exist: those making tradeoffs between factors, and those pushing the entire frontier outward for more efficiency.

Batching and parallelism strategies target specific points on the frontier. Small batch sizes yield excellent per-user latency but high cost per token, while larger batches improve throughput at the expense of latency. Tensor Parallelism (TP) reduces latency via fast NVLink communication, while Expert Parallelism (EP) degree affects both latency and throughput outcomes.

Attention Data Parallelism (ADP) boosts throughput through attention layer replication. The frontier is often jagged, requiring empirical sweeps to find optimal configurations. Techniques like quantization, distillation, and pruning trade quality for throughput, while reasoning levels exchange intelligence for speed. For agentic coding with KV cache reuse and optimal KV-aware routing, deployment strategies must balance these tradeoffs based on traffic nature and user willingness to pay.