The article details building a personal search engine for ~500k domains in a weekend for $10. The system prioritized non-corporate sites like art, code, and hardware projects using a crawler, local language model (Gemma, 4B parameters), and a UI. Challenges included 90% corporate content dominance, addressed by queue weighting and domain capping. Technical hurdles with GPU rentals were resolved by optimizing concurrency. The final index stemmed from curated seeds, reflecting maker communities. Initial bugs involved misclassified summaries and parked domains, fixed via prompt adjustments. The project emphasized lightweight metadata storage and dynamic prioritization over indexing the entire web.

The architecture split tasks: a fetcher gathered domains, a worker processed them with the model, a steward maintained a blocklist, and a UI allowed filtering. Page text was discarded after model processing to save space. The crawler’s logic focused on homepages only, avoiding full HTML or JavaScript execution. Key innovations included real-time category-based priority adjustments and subdomain limits to curb spam.

The author highlighted trade-offs: balancing specificity with scalability, and the importance of human-driven seeding. The GPU rental’s initial issues—CPU contention and cold-start crashes—were resolved by switching to a barebones inference server. The project’s success relied on iterative tuning of category priorities and blocklists. The final index, though modest in scale, served as a personalized tool for discovering niche creators.

The article underscores the feasibility of small-scale, DIY search engines with careful design. It avoids traditional indexing pitfalls by focusing on metadata and human curation. The $10 cost included a GPU rental and minimal compute resources, demonstrating that specialized needs can be met affordably with targeted engineering.