A manufacturer partway through an ERP replacement should have AI write its data conversion scripts only from a field map written by the owner of each legacy table, and keep a script only after its mock conversion load reconciles to the legacy system.
One published test of AI writing conversion code is a paper presented at the IEEE International Conference on Big Data in December 2023 by researchers at Arizona State University, Lawrence Berkeley National Laboratory, Microsoft and other institutions. It acknowledges support from the Department of Energy's Exascale Computing Project, the National Science Foundation, the Department of Homeland Security and research awards from IBM and Amazon, and IBM, Amazon and Microsoft all sell AI services. The authors had ChatGPT-3.5-turbo-16K write SQL, the language databases run, for 105 real-world building energy data transformation problems from 21 US energy companies, data unlike a manufacturer's item master.
Given descriptions of the target fields, the model's code produced the right table in 28 percent of the 105 cases; adding descriptions of the source fields raised that to 36 percent. Adding hints on how each source maps to its target took it to 96 percent. The authors' examples of such hints are column mapping relationships or instructions as simple as "use aggregation", and most of theirs came from a database one co-author maintains. The test had an answer key, too. A run counted as correct only when its output matched a table from a query the researchers wrote by hand, and each mismatch went back to the model with its errors, up to five tries in all. The authors write that validation in the production environment could be challenging due to the lack of ground truth.
On an ERP project, reconciliation comes nearest to that answer key, but totals can tie over a wrong field. The plant controller's line for inventory might read: legacy table, stock by bin; new table, stock by storage location; hint, sum each item's bin quantities into the location holding those bins; check, quantity and value by item and location equal the legacy stock report, and sampled records match field by field. The conversion lead gives the model that line, a few sample rows and the new table's layout, then runs its script in the next mock conversion. The materials manager writes the item master line and the manufacturing engineering lead the bills of materials line. Across 125 cases in three benchmarks, the paper found the model without hints did relatively better on renamed fields, flattening and joins than on other changes, such as merging, pivoting and aggregation, so owners should write lines for tables that sum or reshape records first.
In the first month, owners write lines for every table still to convert. During the second, the model writes scripts for the next mock conversion. By the third, each owner signs off the tables whose checks tie and the CIO puts those scripts into the cutover plan. A table whose script still misses after two mock conversions goes back to the integrator's team. Neither source gives a cost for having AI write conversion scripts. An operating partner can ask every portfolio company changing ERP for its field map and its list of reconciled tables.
In May 2026 GAO, reviewing the Department of Homeland Security's financial systems modernization, a federal program, found that the department's data migration guidance did not include topics such as reconciling data in the legacy system to the new system and documenting where adjustments are made to converted data, a leading practice in a 2002 federal white paper, while rating component planning documents mostly consistent.
