udintel

7th Floor SQB, 77 Marsh Wall London, E14 9SH
[email protected]
+44 (0) 20 3950 2256

Can AI redefine data parsing in EDI readers?

udintel Can AI redefine data parsing in EDI readers?
Technolody Steven Xi Innovation R&D

In today’s fast paced utility management industry, efficiency and accuracy are paramount. The volume and complexity of data processed daily requires tools that streamline workflows and minimise the risk of human error. At udintel, we have always been committed to driving innovation that supports our clients.

“By embedding AI into our data parsing solutions, we are taking a bold step towards fully automated data management with a focus on reducing human intervention and errors in handling large data files..” Steven Xi, CTO

The challenge: complex data parsing across multiple formats

Historically, our platform has offered advanced data parsing capabilities, allowing our clients to process data in various formats, including EDI (Electronic Data Interchange), MM (Market Message), XML (Extensible Markup Language), and PDF (Portable Document Files). Parsing these data formats manually or even semi automatically required significant human input and brought a degree of risk.

While our solutions already provided a robust level of automation, the need for human verification could sometimes lead to errors and delays in processing, potentially causing delays in SLAs. Our new AI-driven system was developed to tackle these challenges by providing a more sophisticated, intelligent parsing solution. With machine learning algorithms trained on vast datasets, this technology can automatically recognize patterns, detect anomalies, and accurately parse data across all supported formats.

Key benefits of AI, driven data parsing

The introduction of AI brings two significant advantages: a reduction in manual data handling and a notable decrease in error rates. In detail:

  • Minimised Human Error - with advanced pattern recognition capabilities, AI detects errors unnoticed by human operators.
  • Accelerated Data Processing - AI's speed in handling and parsing large volumes of data far outpaces traditional methods.
  • Enhanced Automation of Repetitive Tasks - by using AI, we automate repetitive, time-consuming tasks, freeing up valuable time.
  • Improved Compliance and Audit - AI-driven parsing not only ensures quality but also supports compliance with industry standards.
  • Scalability for large date sets - as data volumes continue to grow, our AI-based solution scales seamlessly to meet these demands.
 

How AI Enhances Our Data Parsing Capabilities

The integration of AI into our platform enables dynamic parsing, where the technology learns and improves with each dataset processed. Our AI algorithms adapt to new data patterns over time, refining their accuracy and reducing the need for human intervention. This adaptability makes our platform a truly future-proof solution, prepared to handle the ever-evolving landscape of data in the utility sector. Additionally, our AI technology supports real-time data parsing, meaning that information from EDI, MM, XML, and PDF files can be read, parsed, and integrated into client dashboards in near real-time. This feature is invaluable for clients looking to make quick, data-driven decisions and maintain a competitive edge.

The adoption of AI in data parsing is just the beginning. As machine learning algorithms evolve, we anticipate even greater opportunities to enhance accuracy and automation in utility data management. At udintel we are continuously investing in research and development to ensure our clients benefit from cutting-edge technology that supports their goals in an increasingly data-driven market. By embedding AI into our data parsing solutions, we are not just enhancing our platform, we are transforming the way utility data is managed, empowering our clients to focus on strategy and innovation with the confidence that their data is handled accurately and efficiently.

 

Case Study: AI in Action

As part of our services, we are required to parse over 110 different invoice sources coming in various formats, EDI, MM, XML, and PDF. The complexity in processing them is related to the different technologies utilised to produce these files and frequent manual intervention from suppliers to customise output or deploy patches, as consuequence of contractual or sector changes. From SSE new XML format to Total Energies EDI or XLSX monthly changes, to EDF MM files, we applied our algorithms to current and historical data to around 1,250,000 invoices across 150,000 feeds with a team 4 people supporting dataset analysis and AI feedback across 3 months period. The data quality received and possible bias related to manual intervention from suppliers has been re-factured a number of times as well as we took in consideration the possible model drift, which seem to be a fundamental issue for this project.

when a supplier changes it's invoicing approach, the entire analysis becomes biased as it was based on a dataset not anymore accurate, and the accuracy of the results starts decreasing sensibly
The results obtained have been compared to our historical machine learning (ML) approach and they showed some interesting results:
• AI driven activities are 15% more accurate than machine learning
• AI and ML are saving up to 95% of user time
This said, the maintenance cost for updating algorithms and identify model drifting issues, is justified only if there it a large database and a dedicated structure aligned to the data processing team.
OUR CONCLUSION
from our experience we beliteve the dataset required for a more robust model should be be a least 10x what we analysed so far. A bigger business scale would justify economically also higher investments on AI modelling.