A new artificial intelligence system developed at the Norwegian University of Science and Technology (NTNU) aims to make industrial workplaces more inclusive for Deaf and Hard-of-Hearing (DHH) employees by combining computer vision, haptic feedback, and real-time transcription in a single wearable tool.
The prototype, designed by NTNU master’s student Karin Sofie Syversveen Lie as part of her thesis Communicating Beyond Sound, assists DHH individuals in industrial assembly and workers in high-noise environments. The system integrates live speech transcription using OpenAI’s Whisper model, visual task guidance, and a vibrating belt that alerts the wearer to movement in their surroundings, improving communication and safety on the factory floor.
“To our knowledge, this is the first system that combines haptic alerts, AI-driven live transcription, and visual assembly instructions into a single assistive tool tailored for DHH workers in manufacturing,” said Syversveen Lie. “It’s a step toward an inclusive future of work, where no one is left out due to sensory barriers.”
The system was developed under the Horizon Europe-funded SkillAIbility project, which explores how AI can support inclusive upskilling and workplace integration for people with disabilities. Early testing with participants simulating hearing loss showed positive usability ratings, while a Deaf participant in a recent lab test confirmed the system’s potential and offered feedback for further development.
“This system has potential,” said Davy Van Landuyt, Project Manager at the European Union of the Deaf. “With more iterating and feedback from Deaf and Hard-of-Hearing workers in real environments during development, it could become useful for daily work in industrial settings.”
Globally, over 5% of the population—around 430 million people—experience disabling hearing loss, a figure expected to exceed 700 million by 2050, according to the World Health Organisation. Industrial workplaces, however, remain largely inaccessible to this group.
Sotirios Panagou, Researcher at NTNU and Syversveen Lie’s academic supervisor, described the project as “a prototype for the future of inclusive industry”. “This work is a real example of how SkillAIbility can support innovation that directly improves equity in industry,” he said.
The system’s design uses open-source, modular components, and its source code and documentation are publicly available on GitHub, encouraging replication and adaptation by other researchers or industry partners.
SkillAIbility, a €3 million Horizon Europe project, brings together 14 organisations across nine European countries to advance human-centric AI and automation in manufacturing, aligned with the EU’s Industry 5.0 vision.
The NTNU team behind the project includes Syversveen Lie, Panagou, Martin Francis Berg, Martin Steinert, and Patrick Mikalef, in collaboration with SINTEF Digital. The team is now working on upscaling the system for broader testing and potential industrial deployment.
