AI bringing museums to life
How do you build an app that creates a seamless, immersive experience for the user? You immerse yourself in the research, the automation, the testing, and the AI technology.
That’s exactly what Dr. Esteve Hassan (Computer Science) and two fourth-year computer science students, Nitish Sahni and Joyce Adeyi, have done this year, working with a Nova Scotia tech company to develop an app for museums that they hope to see in use across the country.
The company is Real Meta, and they are positioning the app as a “solution to bring any exhibition to life.” Real Meta had the idea and already had a platform, and company founder Raj Nayak came to Acadia for help with developing a fully functioning application. He wanted to make an AI powered app that offers an immersive experience for visitors, whether at a large institution or a small community museum.
“It’s a project where you feel you can help the industry, you can see how this can impact their productivity,” Dr. Hassan says. With funding support from Invest Nova Scotia’s voucher program confirmed in July 2025, the project hired the two students in late September.
The idea: scan a piece of art or a museum artefact with your phone, and the app identifies it, names it, generates a text summary telling all about the piece and offers an audio description in up to 16 languages. That was the vision. Was it doable?
The two students took to the work passionately, Dr. Hassan says. They began the project in October and had aprototype by the end of 2025. In more technical language, the system combines computer vision models for artifact recognition with generative AI to produce multilingual descriptions and audio narration. They tested four different models, and since some worked better than others, they created their own version that feeds from all of the other models and generates greater accuracy.
“The students go beyond expectations. They like to be part of something useful. We think AI is great, but it has its weaknesses. They were able to identify those weaknesses and tweak it for this project,” Dr. Hassan says.
While many big museums have their own version of this kind of technology, Real Meta wants one that will work anywhere—a win for smaller museums and for Real Meta.
Training the AI model
“It’s very hard to build the model,” Nitish explains, saying that they have to train the AI model to gather accurate and reliable information. “With time it will make the model better.
“ChatGPT came in my second summer at university ,” Nitish adds, talking about the speed at which this technology evolves. “Everything changed after that. It’s a very exciting field to be in."
Joyce took Acadia’s AI courses but didn’t originally see herself working on an AI project. “It’s scary. I came into computer science thinking to do more with web page design. With AI, the more I can understand it, the more comfortable I’ll be at using it,” she says.
Together, they developed both the user-facing application and the backend system, including AI model integration, data pipelines, and an administrative dashboard. Joyce worked on the background AI, researching what was available, and then they both worked on the machine learning model. In the process, they are creating their own proprietary identification model that they will launch on a small scale before handing it over to Real Meta.
“Small museums don’t have the resources to showcase everything. We’re the first people building a model for small museums. It’s a solution. So that’s what’s exciting. It’s helping people solve a problem,” Nitish says.
There are two parts to the project, and Joyce and Nitish have tackled different components that combine to make the app function. The first part is for the visitors: scan a QR code at the museum to access the app. It has several AI models running in the background to gather all the available information on the art piece or artefact. The Acadia team has tested the model with great success and is tweaking it to continually improve its performance.
The second part is for museum administrators who can work in the back end of the app. It generates the same information users will see and allows them to manually adjust or update the content as well as maintain a database of their entire collection. The app will help museums reduce costs and equipment needs by replacing the headsets and digital handsets that museums often sign out to visitors.
Getting with the program
In this stage of the project, they are testing with local museums in Wolfville and Halifax to build the interfaces. Joyce notes that for small museums where the contents aren’t well known, they add the info and then it becomes part of the information available to anyone online.
While they are still testing and tweaking, Joyce is using this work for her research honours degree. That includes testing with well-known pieces of art to work out the hallucinations and biases that current AI is susceptible to. And that means fact checking and finding other databases to see if what AI is saying is correct.
“It’s an opportunity to put into practice things we learn in classes. Just getting to test the AI is a nice change of pace,” she says.
Dr. Hassan says that for many students who work as research assistants, these kinds of projects help them with their honours and Master’s degrees.
“A business came to us to solve their problems. It became a great learning experience for industry work,” he says. “The pair has learned how to do lab journals. It will give them a good boost on their resumes. Joyce is now applying for her Master’s degree, and I plan to keep Nitish busy over the summer.”
“That’s the best thing about Acadia, you’re so connected to your professors,” Nitish says.
“It’s not just an assignment. There’s a real-world application for it,” Joyce adds. “It’s encouraging to have all my experiences lead up to something that is doing something for someone.”
The team is wrapping up their work on the project, just in time for Nitish and Joyce to graduate this spring. Real Meta is working with museums across the country to develop customized models as well as automations that will save time with the set-up process. Eventually, Nitish predicts, you will be able to see any museum virtually, anywhere in the world.