A professional services firm with no written data quality rules for its client, matter and billing records should have AI draft rules from records staff found wrong, and have a named person approve every rule before it runs.
Six researchers at Software Competence Center Hagenberg, Johannes Kepler University Linz, the Hasso Plattner Institute and Austrian Post, their industry partner, posted a preprint, which has not been peer reviewed, on September 5, 2026 that lists Austrian ministries, the State of Upper Austria, a project named QuanTD and the Austrian Science Fund as its support. It says keeping a full rule set by hand is hard, as domain experts often overlook essential rules. Their method hands a language model one record that got past the rules, keeps a drafted rule only if it runs and catches that record's error, then tests survivors on a separate set and drops overlaps.
They ran three models on their own hardware against four publicly available research datasets, named beers, flights, hospital and tax, and a synthetic set of contact records built from real Austrian addresses. None came from a professional services firm. Scored on the record each was drafted from, across the study's models, prompts and datasets, rules averaged an F1 score, from 0 to 1 combining missed errors and false alarms, of 0.66, and 0.4 on contradictions in the synthetic set, two values in one record that cannot both be right, such as a postal code and a city that do not match. Naming the field and kind of error raised F1 by an average of 0.33 over giving the wrong record alone. Allowing no false alarms, rules from two of the three models caught at least half the wrong records in four of the five test sets.
At a firm, the billing manager logs each wrong record with that field and that kind of error (a matter opened with no billing partner or an invoice returned for a bad client address) and the model drafts rules from the log. IT runs each draft against a test set the billing manager has marked right or wrong, and the owner of the data, the intake lead for client records, the practice operations lead for matters and the billing manager for invoices, approves or rejects it by name. The paper notes that approaches passing data through a model typically conflict with strict data protection requirements when the data contains personal information. An approved rule runs with no model, and the firm's client confidentiality terms decide whether wrong records may reach a hosted one.
In month one IT installs a rule library and the billing manager starts the log. During month two owners review drafts weekly. By month three approved rules run on each week's new records and the COO sees how many each one flags. None of these sources prices the staff time. An operating partner can ask every professional services firm in a portfolio for the same log and its list of approved rules.
Rules comparing two fields, such as a matter's open and close dates, belong to the type the study scored lowest, so owners should write those by hand.
A separate preprint, not peer reviewed and naming no funder, first posted October 2, 2023 by authors at the University of Toronto, the Investigative Journalism Foundation and MIT, had GPT-3.5 and GPT-4 write validation tests for a Canadian political donations database, and three human evaluators compared them with a suite an experienced data scientist wrote. The evaluators found that no test the models wrote checked for anything the human suite had not tested.
