Multilingual and Localised AI

The AI revolution has a language problem. While English-speaking users enjoy sophisticated assistants that understand context, nuance, and cultural references, billions of people worldwide are left with systems that treat their languages as afterthoughts—if they're supported at all. As we move deeper into 2025, the industry is finally confronting this disparity, but building truly multilingual AI requires far more than simply translating prompts.

The Scale of the Challenge

The numbers tell a stark story. Of approximately 7,000 languages spoken globally, current AI systems meaningfully support only a tiny fraction. Even among the "supported" languages, performance varies dramatically. Recent evaluations show that state-of-the-art models achieve high variance in performance between different languages, often struggling with questions requiring regional knowledge specific to non-English contexts.

This isn't just about translation quality. It's about cultural understanding, regional nuances, and the fundamental architecture of how these systems process and generate language. Modern multilingual models must navigate idioms, grammatical structures, and cultural references that don't map neatly across linguistic boundaries.

Architecture Decisions: Building Multilingual from the Ground Up

The most successful approach to multilingual AI starts at the foundation. Rather than building English-centric systems and retrofitting multilingual capabilities later, leading developers now integrate language diversity throughout the entire training process. This fundamental shift in thinking has produced models that deliver strong performance across dozens of languages simultaneously.

Pre-training across a range of languages enables models to deliver strong multilingual performance in capabilities such as question answering, translation, and summarization. However, this approach introduces unique challenges. Deciding which languages to include requires balancing business needs, data availability, and computational constraints. Some organizations select languages to support the variety used in global business contexts, while others prioritize linguistic diversity or underserved populations.

Training data remains one of AI's most persistent challenges in the multilingual space. High-quality datasets for many languages are scarce, leading to performance gaps. Innovative approaches now incorporate web crawling, licensed corpora, and synthetic data generation to build more balanced training sets. The key is avoiding the pitfall of simply translating English content, which introduces "translationese" and fails to capture authentic cultural expressions.

Prompt Engineering for Multiple Languages

Effective prompt design differs fundamentally when working across languages. The temptation to create a single "master prompt" and translate it for each target language rarely produces optimal results. Translation of prompts is more akin to transcreation in marketing—it requires cultural adaptation, not just linguistic conversion.

Successful multilingual prompting starts with globalized prompts designed to work across multiple languages and locales. These prompts should be culturally neutral, avoiding idiomatic expressions or language-specific elements that don't translate well. They must be clear and concise, providing necessary information without cultural assumptions.

However, some languages have unique characteristics that require customized instructions. Languages with unique grammatical structures, idiomatic expressions, or translation challenges may require customized instructions to improve AI performance. The challenge lies in balancing scalability with accuracy—master prompts offer consistency but customized prompts deliver better results for specific language pairs.

Key strategies include providing clear, specific instructions with step-by-step reasoning. Rather than asking an AI to "translate this sentence," effective prompts specify the desired language, tone, context, and any cultural considerations. They also define roles and include relevant contextual information that guides the model's attention toward appropriate cultural frameworks.

Evaluation: Moving Beyond English Benchmarks

Traditional AI evaluation metrics often fail to capture multilingual performance accurately. Benchmarks built by translating English resources miss regional and cultural knowledge essential for real-world deployment. This has led to the development of evaluation frameworks that use locally-sourced materials created by native speakers in their own languages.

Evaluation must assess multiple dimensions: linguistic accuracy, cultural appropriateness, regional knowledge, and consistency across languages. Models may perform well on general language tasks while failing on culturally-specific queries. Testing output with native speakers and continuously soliciting feedback becomes essential for quality assurance.

The evaluation process should measure not just what models produce, but how well they serve diverse language users in actual deployment environments. This includes assessing performance on domain-specific content, analyzing how models handle language confusion (switching between languages mid-response), and testing their ability to maintain appropriate tone and formality across cultural contexts.

When Region-Specific Models Make Sense

Despite advances in general-purpose multilingual models, certain scenarios demand region-specific approaches. European markets may benefit from models optimized for EU languages and compliance requirements. Asian markets often require models with strong performance in Chinese, Japanese, and Korean—languages with fundamentally different structures than European ones.

The decision to deploy region-specific models depends on several factors. If your primary market uses languages where general models underperform, specialized alternatives may deliver better results. Regulatory requirements around data sovereignty and compliance may necessitate regional deployment. Performance needs for domain-specific applications—legal, medical, or technical content—often justify the investment in specialized models.

Region-specific content becomes critical when dealing with local regulations, cultural sensitivities, and market-specific terminology. AI systems serving financial services in Japan need different knowledge than those serving European healthcare providers, even if the underlying language capabilities overlap.

The Path Forward

Building effective multilingual AI requires a holistic approach that integrates architecture decisions, prompt engineering, evaluation frameworks, and strategic deployment choices. Success depends on treating multilingual capabilities as core features, not afterthoughts.

Organizations should start by clearly defining their target markets and languages, then evaluate models based on actual performance in those languages—not just claimed support. They must invest in proper prompt engineering tailored to each language and culture, while building evaluation frameworks that capture regional knowledge and cultural appropriateness.

The multilingual AI landscape continues to evolve rapidly. Models now support broader language coverage while delivering more consistent performance. However, significant gaps remain, particularly for lower-resource languages and specialized domains. The future of AI must be multilingual by design, built on diverse data and evaluated against authentic cultural contexts.

For businesses expanding globally, the question is no longer whether to invest in multilingual AI, but how to do so effectively. The organizations that get this right—building systems that truly understand and serve diverse language communities—will unlock markets and capabilities that remain inaccessible to those clinging to English-only approaches.