HeadlinesBriefing favicon HeadlinesBriefing.com

Building Model-Agnostic Vulnerability Scanning at Fleet Scale

Hacker News •
×

Project Glasswing's research reveals that relying on a single frontier model for security scanning creates blind spots — each model examines code through the same lens, missing distinct vulnerability classes. The solution is a model-agnostic harness where models function as interchangeable components: one handles initial discovery, another validates findings. This architecture treats LLMs as stateless compute engines, externalizing all state to a database to survive context exhaustion, crashes, and rate limits.

The team evolved a ~450-line security-audit skill running a 7-phase audit (recon, hunting, adversarial validation, mechanical schema checks, independent re-verification) into a fleet-wide pipeline spanning 128 repositories across Rust, Go, C, Lua, TypeScript, and Python. The harness traces cross-repo dependencies without per-language tuning, focusing on security orchestration logic rather than syntax parsing. Three walls forced this evolution: context windows cannibalizing memory mid-run, crashes wiping hours of work, and single-repo blindness to interface vulnerabilities between components.

The production workflow splits into two stages: the Vulnerability Discovery Harness (VDH) for hunting and the Vulnerability Validation System (VVS) for cross-checking. A minimal viable harness needs only Recon, Hunt, and Validate stages persisted in a database, plus a separate Validator that cannot file findings. Deduplication and cross-repo tracing wait until noise or multi-repo complexity demands them.

Building model-agnostic from day one prevents vendor lock-in and lets teams swap models as capabilities shift. The harness — not the model — is the durable asset.