HeadlinesBriefing favicon HeadlinesBriefing.com

Ctrlb-decompose: Log Compression for LLMs

Hacker News •
×

Ctrlb-decompose is a new tool designed to significantly reduce the volume of log data by compressing raw log lines into structural patterns. It achieves this by processing millions of noisy log entries into a manageable set of actionable patterns, complete with typed variables, statistical insights, anomaly flags, and severity scoring.

The tool operates through a two-stage pipeline: CLP Encoding, which normalizes variable tokens into typed placeholders, and Drain3 Clustering, which groups similar logtypes into templates using a prefix tree. This process ensures that structurally identical lines produce the same "logtype," regardless of their specific values.

Ctrlb-decompose can be used as a CLI tool, within a browser via WASM, or as a Rust library. It offers various output formats, including human-readable, LLM-optimized markdown, and structured JSON. The tool also integrates with Claude Code as a plugin, allowing users to analyze logs through natural language prompts.