Carl Kolon detailed his work on semantic search for the Joint Fleet Maintenance Manual (JFMM). Frustrated by the limitations of keyword-based search in the lengthy PDF, he built JFMM.net. Initially, he used vector similarity search, leveraging FastAPI, Sentence Transformers, and PGVector to find results based on meaning, not just exact matches.

However, the initial setup proved expensive due to hosted Postgres instances and memory-intensive Sentence Transformers. Kolon then rewrote JFMM.net to address these issues. He swapped the database for SQLite with the `sqlite-vec` extension and implemented quantization using llama.cpp to reduce RAM usage and costs. This change allowed him to embed the manual locally.

By switching to SQLite and using quantization, Kolon optimized the application for both performance and cost. These changes improved the speed and efficiency of the semantic search. The project showcases a practical approach to building a more effective search tool for a specific, data-heavy use case, demonstrating the benefits of optimization.