HeadlinesBriefing favicon HeadlinesBriefing.com

Rust-based SearXNG-like Search Engine

Hacker News •
×

A new metadata search engine, similar to SearXNG, has been developed in Rust. This engine efficiently fans out search queries to multiple sources like Duck Duck Go, Brave, Startpage, and Yahoo concurrently. It then scrapes the HTML results, normalizes URLs by stripping tracking parameters and sorting query parameters, and deduplicates them.

Ranking is achieved using Reciprocal Rank Fusion (RRF), where pages returned by multiple engines receive higher scores. The aggregated and ranked results are then returned as JSON. The project is available as a library or a server and requires Rust 1.75+.

Installation can be done via Cargo, and the project includes examples for querying single or multiple engines. API endpoints for health checks and search are provided. The project also includes comprehensive tests and instructions for adding new search engines. A terminal UI using ratatui is also available.