HeadlinesBriefing favicon HeadlinesBriefing.com

OpenTelemetry GenAI Observability: The Messy Reality Behind Standardization

Hacker News •
×

Everyone touts OpenTelemetry (OTel) support for GenAI observability, but the reality is fragmented. Engineers at groundcover discovered this firsthand while building their AI Observability solution. Despite OTel’s semantic conventions, attributes vary wildly across SDKs like Traceloop, LangSmith, and manual instrumentation. Even the same provider—OpenAI vs. Anthropic—emits different telemetry structures. This inconsistency creates a maze of naming conflicts, structural mismatches, and provider-specific quirks. The team spent weeks building a normalizer to standardize GenAI spans into a single canonical view, regardless of origin. Their goal: simplify observability so SREs focus on incident resolution, not data wrangling.

The normalization process tackles three complexity axes: SDKs, frameworks, and providers. SDKs like Traceloop and LangSmith emit different attribute names and message formats. Frameworks such as LangGraph and CrewAI dictate span tree shapes and metadata. Providers like OpenAI and Anthropic vary in API structures and token semantics. The normalizer uses a priority chain to resolve conflicts—e.g., preferring `gen_ai.request.model` over `traceloop.association.properties.ls_model_name`. It also adjusts for token counting discrepancies, adding cache tokens for Anthropic and Bedrock but not OpenAI. This three-tier logic ensures consistent metrics across diverse systems.

The result? A unified Field struct that collapses six model identifiers and five token-naming conventions into standardized fields. While the OTel standard remains aspirational, groundcover’s work shows the practical reality: observability requires more than specs. Their normalizer closes gaps as official instrumentations mature, proving that AI observability demands continuous adaptation. This effort underscores a critical truth—developer tools must evolve alongside the technology they monitor. OpenTelemetry GenAI Observability remains a work in progress, but groundcover’s findings offer a roadmap for building tools that just work.