Case Studies

Our client, a startup in the field of eHealth, has approached us to create a system for processing medical records with the goal of extracting relevant data, like patient data, treatment and cost.

Medical records are complex documents with variable layouts and terminology, unstructured data like doctors notes and handwritten text, so this project requires deep expertise in modern computer vision algorithms and large language models to develop an effective tool for smart health record processing.

We have created an AI-powered medical records processing app capable of extracting data from multi page documents irrespective of their layout and format. Our app extracts only prespecified data, ignoring non-essential information like patient instructions and nurses notes, and navigates the difference in terminology used between hospitals to ensure data uniformity.

Multiple Layout Processing

Since each hospital uses their own medical record format, it was important for our client that the system could process various document layouts. We have implemented an algorithm that detects key elements on each page, like patient contact data part or treatment cost part, and extracts data given the nuances of the layout at hand.

The layout detection module is flexible: when a new document layout is introduced, we can easily train the algorithm to process the new layout type.

Variable Terminology

Medical records from different hospitals often use different words to describe the same thing. This variability of terminology is less than ideal for data extraction and analysis purposes.

Our machine learning team has implemented multiple LLMs to combat the issue of variable terminology and treat different words describing the same subject as one and the same. Using the power of modern language models, our app "understands" the meaning behind terms used in the documents and groups items by meaning, rather than specific words used to describe them.

Relevant Data Retrieval

Medical records, especially ones that are years worth of data, are often PDF files with 100+ pages long. Crawling through such massive documents using regular data crawlers can be very time consuming, and since relevant data is often spread out across multiple pages, traditional document processing methods are entirely ineffective.

We have developed a document crawling module that uses powerful computer vision and OCR algorithms to quickly detect and isolate blocks of text with relevant information even if data is spread across multiple pages. The app is capable of dividing one large file into multiple health records and extracting data from each one, ignoring non-essential information, like ECG, images, and patient instructions.

We have also implemented a handwritten text recognition module to enhance the recognition capabilities of the app.

The app has already been implemented into our client's systems and has shown how effective modern AI systems can be for processing complex documents. Our system successfully extracts relevant data, including handwritten text, from large PDF health records at high speeds.

The data is extracted in a JSON format and is uploaded into our client's database for further analysis.