As tools like ChatGPT become part of our daily lives, the field of translation studies faces a new challenge: evaluating the reliability of AI-generated translations. Institutional services like the EU’s translation units find themselves in a perfect storm, balancing the immense potential of AI against its limitations.
One type of linguistic unit that is often taken for granted, yet is remarkably frequent in institutional texts, is that of proper names. In this type of public communication, we often find names referring to specific individuals, or anthroponyms (Keir Starmer); names of places, also called toponyms (Spain, Berlin); and names of organisations, events, products, and other miscellanea, which fall under the category of chrematonyms (Ministry of Foreign Affairs, the Istanbul Convention). And if the reader has never heard of “chrematonyms” before, it is in fact further proof of the general lack of knowledge and research surrounding these important symbols.
Names are cultural signs within a continuously evolving setting, and they can be adopted and modified based on their social connotations, making them very difficult to automate. We see this clearly with politicians and administrations, who constantly create new name variations. Researchers working in the field of Natural Language Processing have already identified the challenges that named entities pose to machines, including:
One type of linguistic unit that is often taken for granted, yet is remarkably frequent in institutional texts, is that of proper names. In this type of public communication, we often find names referring to specific individuals, or anthroponyms (Keir Starmer); names of places, also called toponyms (Spain, Berlin); and names of organisations, events, products, and other miscellanea, which fall under the category of chrematonyms (Ministry of Foreign Affairs, the Istanbul Convention). And if the reader has never heard of “chrematonyms” before, it is in fact further proof of the general lack of knowledge and research surrounding these important symbols.
Names are cultural signs within a continuously evolving setting, and they can be adopted and modified based on their social connotations, making them very difficult to automate. We see this clearly with politicians and administrations, who constantly create new name variations. Researchers working in the field of Natural Language Processing have already identified the challenges that named entities pose to machines, including:
- homographic pairs (Washington as both the capital city and the US state);
- variant spellings (Kyiv vs. Kiev);
- morphological inflections (Praha and Praze in Czech);
- phraseological patterns (Brussels says/warns/recommends sth.);
- multi-word expressions (Organisation for Economic Co-Operation and Development);
- Acronyms (OLAF).
The NAIMES project, co-funded by the European Union and the Czech Ministry of Education, Youth and Sports (2025-2027), aims to evaluate and compare how ChatGPT translates institutional proper names from English, German, and Czech into Spanish.
The project focuses on non-English-centric language combinations (German/Czech > Spanish), allowing for better detection of translation errors in low-resource language contexts, and it introduces new AI evaluation metrics. NAIMES’ results will provide insights into the quality of AI-generated translations of proper names and the differences in quality between the three language combinations (Czech > Spanish, German > Spanish, and English > Spanish).
The project focuses on non-English-centric language combinations (German/Czech > Spanish), allowing for better detection of translation errors in low-resource language contexts, and it introduces new AI evaluation metrics. NAIMES’ results will provide insights into the quality of AI-generated translations of proper names and the differences in quality between the three language combinations (Czech > Spanish, German > Spanish, and English > Spanish).
This work was supported by OP JAC Project “MSCA Fellowships at Palacký University IV.” CZ.02.01.01/00/22_010/0013054, run at Palacký University in Olomouc, Czech Republic.