Processing 8TB of mobile pings monthly with traditional clustering like DBSCAN becomes prohibitively expensive. An engineer found the compute costs at this scale unsustainable, despite good results. This prompted a shift from running costly algorithms to leveraging spatial indexing for a more efficient approach.

The proposed solution uses H3 hexagons as pre-computed spatial clusters. Instead of calculating clusters in real-time, the spatial grid handles discretization upfront. This method reduces computational overhead significantly by treating each hexagon as a ready-made unit for analysis, bypassing intensive algorithmic processing.

This approach highlights a growing trend in geospatial big data: moving computation to the data structure itself. For developers working with terabyte-scale datasets, it offers a practical path to manage costs. The next step involves testing its limits and scalability in production environments on AWS.