Why test on your own bills?
Accuracy depends on the documents. A tool that reads clean PDFs from large suppliers can struggle with a photographed docket, a two-page bill with the total on the second page, or a statement that lists several invoices. A single percentage from a vendor does not tell you which of your suppliers it gets wrong, or which field. Your own test set answers both questions, and you can rerun it whenever the tool, its model or your supplier mix changes.
How do you build a test set?
Pull bills from the past few months across several client files or cost centres, with permission to use them for testing. Aim for the mix you actually receive, then add the awkward cases on purpose so the test shows how the tool behaves at the edges. Include bills from the suppliers that send most of your volume, because their errors repeat every month, and a few from suppliers you see rarely. Note how each bill arrived, such as email, scan or phone photo, so results can be split by source.
Keep the test set apart from production work and store it under the same access controls as other client records. Start with 50 to 100 bills and add more if your supplier list is long.
- Clean PDFs from your highest-volume suppliers.
- Phone photos and scans of paper bills and delivery dockets.
- Multi-page bills and supplier statements.
- Bills with GST-free and taxable lines together.
- Credit notes, foreign currency bills and bills without an ABN.
- Duplicates: the same bill sent twice, or as both a PDF and a paper copy.
Which fields should you score?
Score the fields your process depends on, with the correct value recorded by a person before the test runs. The ATO's tax invoice rules give the core list: the seller's identity and ABN, the issue date, a description of the items, the GST amount and the extent to which each item is taxable, plus the buyer's identity or ABN on sales of $1,000 or more. Add the fields AP needs: invoice number, due date, total, bank details if printed and any purchase order number. Score line items separately from header fields, because the results for each can differ.
How do you score the results?
For each field on each bill, record one of four outcomes: correct, wrong, missing, or flagged by the tool as uncertain. A wrong value that the tool reported with confidence is the outcome to watch most closely, because nothing prompts a person to look at it. Count the results by field and by supplier, and list the confident errors separately. Keep the scoring in a spreadsheet with one row per bill and one column per field, so you can filter by supplier or document type. Where two people disagree on the correct value, such as a description, agree a rule before scoring.
Hypothetical example: a tool reads the total correctly on 98 of 100 test bills. Both misses come from one supplier whose bills put the total on page two. Every bill from that supplier will be wrong each month, so it needs a rule or a person, even though the overall figure looks high.
How do the results become review rules?
Use the field results to decide what the workflow checks and when a person steps in. Fields the tool reads reliably can be prefilled and spot-checked. Fields with errors need a validation rule or a person. Useful automated checks include line amounts adding to the subtotal, GST equal to one-eleventh of the total where the bill says the price includes GST, the ABN matching the supplier contact, and the invoice number not already recorded for that supplier. Any failed check sends the bill to a person with the reason shown.
What records should you keep from scanning?
The ATO's record-keeping rules say the relevant information in a record must not be changed and must be stored in a way that protects it from change or damage, and most records must be kept for five years. Keep the original image or PDF attached to the bill in the ledger. Store extracted values separately from any corrections a person makes, so you can see what the tool read and what was changed. Rerun the test set after any change to the tool or model, and compare the results with the previous run field by field.