A developer rebuilds a web‑data ingestion system that pulls from 100+ sources daily. Silent failures leave downstream analytics corrupted, trigger retry storms, and risk IP bans. The new design uses Effect‑TS, a TypeScript effect system that models side effects, errors, and lifecycles in the type system.

Effect’s lazy, typed Effect values let developers declare what a fetch should return, what errors it can throw, and how to retry before execution. By tagging failures—NetworkError, TimeoutError, RateLimitError—the pipeline enforces retry policies and backpressure at compile time.

The implementation wraps Puppeteer browser launches and page navigation in Effect constructs, automatically acquiring and releasing resources. Bright Data proxies handle IP rotation and CAPTCHA solving, while Effect’s `retryIfRetryable` and `withSimpleRateLimit` compose declaratively, keeping the code free of nested callbacks.

Running the pipeline with `Effect.runPromise` triggers the entire flow: browser start, request, retry, parse, and cleanup. The typed error contracts make debugging trivial, and the explicit resource lifecycle prevents leaks. Future changes—new sources, altered retry windows, or added observability—can be added without rewriting core logic.