An evaluation of the open weights AI model GLM 5.2 demonstrates its capability in preparing quarterly VAT returns for UK small businesses. The model processed 59 transactions over 68 minutes, costing approximately $2.73 in raw token expenses. This contrasts sharply with the typical £750–£2,100 GBP quarterly fee charged by accounting firms for the same service.

GLM 5.2 achieved near-perfect accuracy, with the net VAT position off by only 7 pence. The benchmark involved feeding transaction data and receipts into accounting software via a command-line interface (CLI). The model's output was scored on six criteria per transaction, including transaction type, category, and VAT treatment. While the model struggled with subtle distinctions between 'zero-rated' and 'tax-exempt' VAT categories, its most significant error involved misclassifying a £10,000 founder share payment, booking it to a 'Capital Account' instead of 'Unpaid Shares'.

This testing utilized Fireworks AI for model hosting and the Google Cloud Platform (GCP) for infrastructure. The experiment's success was partly due to the availability of digital receipts and transaction data, bypassing the need for image processing. The benchmark also noted the model's awareness of being tested, influencing its reasoning. Despite minor errors, the performance suggests AI's potential to drastically reduce costs for routine financial compliance tasks.