HeadlinesBriefing favicon HeadlinesBriefing.com

Build & Run Cloud IDP System on AWS

Towards Data Science •
×

This article details the construction and deployment of a cloud-based Intelligent Document Processing (IDP) system on Amazon Web Services (AWS). The system automates the classification and extraction of Personally Identifiable Information (PII) from emails, specifically for handling Freedom of Information-type requests.

Solicitors submit requests via email with attached official documents. An automated workflow processes these emails daily, extracts and classifies attachments, and identifies PII. This extracted data is then used to search a claims database. If matches are found, a password-protected PDF of the claim details is sent to the solicitor, with the password delivered in a separate email. If no matches are found, a notification email is sent.

The IDP system leverages several AWS services, including IAM for permissions, EventBridge for scheduling, Step Functions for orchestration, Lambda for compute, S3 for storage, Secrets Manager for secure credential storage, Textract for Optical Character Recognition (OCR), and Bedrock for AI inference. The process involves ingesting email attachments, extracting text using Textract, and then using Bedrock Claude Sonnet 4.6 to classify documents and extract PII. A Human-In-The-Loop (HIL) process is included for final cross-checks, though the system is largely automated and proved approximately 90% more efficient than its manual predecessor. The simplified version described uses Gmail as the email server and focuses on processing image files (JPG or PNG) of passports, driver's licenses, or bank statements, outputting results to an S3 file. The full code is available on GitHub.