For months, I built a data pipeline on my Windows machine using RSS ingestion in WSL2, Postgres in Docker, Kestra orchestration, and dbt transformations. Everything worked locally because it all ran on one machine. Moving it to AWS via CLI revealed hidden assumptions.

Setting up a t3.small EC2 instance with Ubuntu 22.04 was easy. Surprises included an 8GB disk that was too small, requiring growpart and resize2fs, but newer Nitro instances name drives `nvme0n1` instead of `xvda`. The instance went “impaired” during Kestra's image pull, fixed by adding a 1GB swap file. An Elastic IP and security group restrictions added minor chores.

Transferring the pipeline with rsync, Docker, and docker compose worked, but Kestra's flows live in its internal database, not files, so my fetch_rss flow needed manual YAML pasting. This highlighted a pattern of assumptions breaking one by one.

The real challenge was not AWS itself but the hidden dependencies built on “same machine” assumptions, teaching more than the original build did.