Interdisciplinary research project on AI-based real-time translation (speech-to-speech) launched at the University of Koblenz

You speak into a microphone – and within seconds, your words are translated into multiple languages.

What sounds like a modern reimagining of the Tower of Babel has, in fact, evolved into an established and rapidly expanding field within AI research. AI-based real-time translation is already technically feasible and will continue to improve in the coming months and years – with potentially far-reaching consequences for the way we communicate.

This applied field of research is the focus of an interdisciplinary project team at University of Koblenz, led by JProf. Dr. Andreas Weilinghoff (Faculty 2, English Linguistics), Prof. Dr. Andreas Mauthe (Faculty 4, IT and Data Security), Prof. Dr. Ralf Lämmel (Faculty 4, Software Languages) and Christian Schneider (Centre for Information and Media Technologies (ZIMT)). The project team is coordinated by Joshua Dohmen, the Artificial Intelligence Officer at the University of Koblenz. The aim of the project is to systematically investigate the potentials and limitations of AI-based real-time translation for everyday university settings and to implement it in a responsible way.

The project focuses on three central contexts: (1) its use in teaching, both in lectures and seminars, (2) in consultation settings, and (3) in examination contexts. The aim is to implement the technology to enhance transparency, address relevant legal requirements, and develop data protection–compliant solutions for everyday university life. In particular, the project aims to reduce language barriers for international students and thereby sustainably enhance their participation in academic life. AI-supported real-time translation can serve as a low-threshold gateway to new linguistic and social contexts.

The project members emphasize that “AI-based translation cannot and should not replace language learning. Language proficiency remains a core competence - indispensable for academic success, professional careers, and active participation in society.”

Overall, AI-based real-time translation opens up new strategic opportunities for the University of Koblenz. The project will examine whether currently available AI applica-tions meet the necessary requirements for efficient, effective, data protection-compliant and sustainable use in everyday university life – particularly with regard to GDPR requirements and the digital sovereignty of Europe – or whether alternative solutions need to be developed.

The interdisciplinary project team brings together expertise from linguistics and cultural studies as well as from computer science and engineering, equally addressing technical, linguistic, and cultural questions. In the field of English Linguistics, the primary focus lies on the semantic and linguistic quality of the translations. Translation performance is analysed both qualitatively and quantitatively, with particular attention to the precise rendering of subject-specific content. The team also examines the extent to which the models can be adapted/finetuned to different academic disciplines, languages, and language varieties. In the field of Software Engineering, the focus lies on systematic quality assurance and the validation of deployed models, as well as on the development of automated testing procedures. In addition, the team will explore innovative approaches that enable AI systems to verify their own outputs, thereby enhancing reliability and robustness. In the field of IT and data security, the team focuses on functional requirements, data quality, and system security. Among other aspects, the project investigates which factors influence the performance of AI systems and how resilient the applications are to manipulation or attacks. Furthermore, the team will systematically examine the relevant legal frameworks, with particular attention to examination regulations and data protection in live usage.

Further information can also be found on the project website:

https://www.uni-koblenz.de/de/ki/projekte/speech-to-speech/s2s