Case Studies

The app was created for an art museum looking to provide digital guide services to their visitors through their smartphones: the visitor would have to point their smartphone camera at an art piece, and its description would show up on the screen.

The project had significant limitations:

• Ability to add more classes without retraining the model

As our client didn't have any experience with AI nor did they want to hire in-staff ML developers, we faced a significant limitation: we couldn’t retrain the image recognition model when new art pieces are added to a collection. We had to create an app that would allow the addition of new classes (new art pieces) without the involvement of ML developers.

• Recognition speed

Another limitation was how quick the detection and recognition process would have to be. Our client requested the recognition process to take less than 1 second.

We decided to choose a new approach altogether and utilize classic algorithms to detect images as they require far less processing power and allow for the easy addition of new classes.

The app uses keypoint detection to recognize art pieces in real-time and provide their description. The algorithm works just as well as a machine learning model would for their application, but meets the requirements of the project, which regular ML models couldn’t do.

The app automatically detects which museums the user is visiting to provide the most accurate recognition results. The user interface is simple and intuitive:

• the app prompts the user to point the smartphone camera at an art piece
• our algorithm detects its key points, compares them to the key points of art pieces in the dataset, and chooses the one with the most similarities
• the user sees the piece description

Our client didn’t have to invest in servers to run the machine learning model from: our app does all of the heavy work on the mobile device. The recognition process on average takes ~600 ms, depending on the type of mobile device used.

The app we’ve developed meets all of the requirements our client had:

• the detection takes less than 1 second
• there’s no need to retrain the model every time a new art piece is added to the dataset
• no ML developers needed to support and manage the app

This app is a simple and elegant solution for those who want to reap the benefits of AI, but don’t - or can’t - get deep into it, investing money not only into the app development itself but the retraining and general maintenance that comes with AI models. The app’s recognition module works without any supervision, our client can add new art pieces as often as they please.