Feature Extraction from Complex Engineering Drawings With AI

Engineering documents are more often than not processed manually due to their complex structure and lack of standardization. Due to how complex engineering drawings are, using ready-made AI solutions for extracting data may not always yield satisfactory results, especially when the drawings include rotated elements.

In this article we share our experience with developing an AI system for processing complex engineering drawings and how we solved an issue of detecting geometric dimensioning and tolerancing annotations placed at an angle.

AI For Unstructured Data Processing

Unstructured data extraction represents a critical frontier in the realm of information management, offering organizations the opportunity to unlock valuable insights from documents.

Unlike structured data, which adheres to predefined formats or schemas, unstructured data lacks a consistent organization, making it inherently challenging to extract insights manually.

Engineering drawings are a classic example of an unstructured document: despite adhering to a set of strict design standards, the drawings themselves differ from each other significantly and lack rigid structure, and include a mix of text, special symbols, and geometric shapes, making it challenging to extract relevant information manually.

Using AI-driven extraction techniques, engineering drawings can be easily analyzed to extract relevant details, like geometric dimensioning and tolerancing annotations, title blocks, measures, etc.

Ready-made Solutions vs Custom AI Software For Engineering Document Processing

Despite the abundance of document processing solutions, even those geared specifically for processing engineering drawings, there is still a strong need for custom solutions as ready-made systems often can’t handle even slight deviations from a standard engineering drawing layout.

One of the main challenges that present a challenge to ready-made systems are diagonally positioned elements like geometric dimensioning and tolerancing annotations. Document processing solutions often have trouble processing diagonal text, let alone detecting special symbols. Engineering drawings often have interfering lines and elements, adding to the challenge.

Feature Extraction From Engineering Drawings

Extracting engineering drawing features most often includes detecting geometric dimensioning and tolerancing annotations, which include both text and special symbols. Given their variable positioning, including rotation, and multiple data types, the process of detecting and extracting GD&T annotation includes multiple steps.

Step 1: GD&T Position Detection

The first step is detecting the geometric dimensioning and tolerancing annotations on the drawing using OpenCV. Computer vision algorithms can be trained to detect objects irrespective of their position and angle.

Having had experience solving similar challenges, like in our architectural drawing recognition software project, we have a strong basis for building similar systems, which is what we did for our client:

Step 2: Rotation Angle Detection

For each GD&T annotation we calculate the rotation angle which is used to rotate the annotation to make it horizontal and cut it out from the drawing for further processing:

Each annotation is saved as individual raster images ready for text detection.

Step 3: Data Extraction

Next, each annotation is processed by a symbol detection engine. Our AI engine of choice is Tesseract as it provides highly accurate detection results and can handle text segmentation, i.e. text with different heights forming multiple lines.

First, we use OpenCV to detect the text area to increase text detection accuracy further down the line. Next, the image is passed on to the OCR engine to detect all text and symbols present in the annotation.

Step 4: Data Analysis

Extracted text and symbols need to be analyzed and interpreted to provide “digestible” data to use in the client’s systems. Symbols are separated into groups forming tolerances, fits, and radii.

Step 5: Data Export

In our smart document processing projects we usually go about data extraction in one of three ways:

1. The data is extracted in a JSON format, perfect for embedding the data into existing software for further analysis and storage,
2. The data is extracted into an Excel spreadsheet, giving an easy-to-read data overview,
3. The data is post-processed and is sent straight into a digital document processing software, either one custom-made within the scope of the project or a system a client already uses.

The data extraction format fully depends on the needs of a client and can be adjusted and changed to better cover the client’s needs.

How To Choose An AI Development Company

There are multiple steps you can take to make a well-informed decision when it comes to hiring a AI development company:

1. Take a look at platforms like Clutch, Manifest, Upwork and Goodfirms to read client reviews and testimonials to get a better idea of past company projects and overall customer satisfaction,
2. Analyze the size of the vendor as large companies are more likely to have enough resources to provide a dedicated team for your project when compared to smaller vendors,
3. It is important to have a clear understanding whether your outsourcing provider complies with local and international standards like HIPAA, ISO, PCI, and has data security protocols and solid security policy in place,
4. Ask your potential vendor the right questions to be sure that the provider is reliable. Compile a list of questions that you can ask a software development outsourcing company that will help you get all the important information about the vendor. Questions like “Has your company received any industry awards?” and “Do you have experience working with companies of our size and in our industry?” will help you understand the company’s market presence and its average client. Questions like “How do you ensure information security in the workplace?” and “Do you have a comprehensive disaster recovery plan?” will get you an understanding of the company’s facilities and the security practices,
5. Take a look at a company’s portfolio on their website, platforms like Clutch or Goodfirms, or ask the provider directly for a portfolio of successfully delivered computer vision projects,
6. Evaluate the tech stack the company works with and assess the skills of its computer vision developers:
   • 2-5 years of experience in the field of artificial intelligence development,
   • Experience with data science, cloud development, AI system development, big data,
   • Technical stack: Tesseract, Python, OpenCV, TensorFlow, Cafe, etc.,
   • Experience in the development of machine learning models as well as creating and balancing datasets,
   • History of working with object recognition, natural language processing, OCR.

Why choose Businessware Technologies as your software development company?

• Businessware Technologies is a reliable AI development vendor: it has been recognised as one of the top software development companies by Clutch and Manifest, it is a Top Rated Plus agency Upwork, and has received local awards for its excellent work,
• A team of over 70 highly skilled software engineers with extensive experience in developing complex software for both startups and Fortune 500 companies,
• Deep expertise in modern AI technologies and approaches to system development, like data science, machine learning, OpenCV, Python, Tesseract, and many more,
• Businessware Technologies is a Microsoft Gold Certified partner,
• Businessware Technologies is compliant with GDPR, ISO 9001, ISO 27001 standards,
• Businessware Technologies works with Fortune 500 companies and has had decades-long relationships with most of its clients,
• Businessware Technologies has proven to be a reliable AI outsourcing partner by having an excellent track record in AI and ML development backed by an extensive portfolio of successful projects.

If you have a document processing project in mind and need help with implementation, contact our manager and they will be happy to help you.