Lionbridge Machine Translation Tracker

The longest-standing measurement of the leading Machine Translation engines — and now several generative AI models — will help you choose which automated translation option will best meet your needs.

TALK TO US

The Role of Machine Translation in Global Communication

Still essential in an era of technological change

As companies produce ever-growing amounts of content in different languages, they seek to extend their reach to the globalized world faster and more cost-efficiently.

At their disposal? Machine Translation (MT) engines and generative AI (GenAI) technology, with each technology playing an important but different role in localization.

Whereas GenAI / Large Language Models (LLMs) are highly suited for assisted post-editing and optimizing localization workflows through automation, MT engines are unsurpassed for initial translation due to their superior speed and cost-effectiveness. Moreover, the quality of MT is generally better than that of LLMs for initial translation, though we are tracking developments closely.

Comparing Four Machine Translation Deployment Strategies

Machine Translation solutions are comprised of the following four basic strategies.

Public MT

This strategy includes services like Google Translate or Bing Translator. Such services are readily available at no cost. However, the strategy’s shortcomings are its lack of security and quality issues in some situations, as engines have not been trained for specific domains or particular use cases.

On-Premises MT

This strategy requires a company to deploy an MT server in its IT environment. While this is the most secure option, it comes at a significant cost, is complex to deploy and manage, and requires ongoing maintenance. Importantly, this strategy often produces suboptimal MT output across multiple language pairs or content types.

Cloud MT

This strategy works like the public MT option because it is also hosted in the cloud. However, unlike public MT, it creates a company-dedicated instance. Any data provided to the service is tightly secured and not shared with third parties. It delivers additional capabilities around terminology customization and has other benefits. However, it can result in vendor lock-in and less-than-optimal MT quality across multiple language pairs.

Best-of-Breed MT

This strategy involves a single platform that allows companies to leverage multiple engines. It provides a single layer of terminology customization, an easy-to-manage interface, and the ability to choose the best option for different language pairs, industries or domains, and types of content. This approach—offered by Lionbridge—is designed to deliver the best translation results and is a key Lionbridge differentiator.

Want to learn more about Machine Translation terminology and the different types of automated translation technologies? Check out our blog, Machine Translation in Translation.

Which Automated Translation Engine Is Best?

There’s no simple answer.

When choosing among the many available automated translation systems, it’s important to note that some MT engines are trained to address a specific function or domain. If your needs don’t align with that purpose, the engine may perform sub-optimally, no matter how advanced it is.

Identify why you are using automated translations to determine the most suitable option. If you want an MT engine for general use, it may be appropriate to use Google Translate or Bing Translator. However, you may achieve better results using a different tool when seeking MT services for a specific language or domain.

How does Lionbridge select among the different engines? Our best-of-breed approach utilizes one platform with five engines: Amazon, Bing NMT (Microsoft), DeepL, Google NMT, and Yandex. The platform is accessible via our Smart MT™, Application Programming Interface (API), and our customers’ localization workflows.

We provide the flexible application and customization of MT system selection based on language or use case, using language-specific rules, glossaries, and Do-Not-Translate terminology added directly at the platform level instead of at the engine level.

Our MT solutions cover 115 language pairs for leading global brands.

Analyzing Machine Translation Performance

Insights from the Lionbridge Machine Translation Tracker

With over two decades of MT experience, we have gathered a large volume of linguistic and quality data about MT technology.

Lionbridge’s Machine Translation Tracker analyzes the performance of the top five MT engines and several GenAI models. We publish the results periodically to help you understand how these tools perform, depending on the content’s domain and the used language pairs.

Check out our findings below. The results—which are everchanging—can give you an idea of which options are right for you.

Evaluating Overall MT Performance

Time

Chart 1: Comparison of the average quality of top MT engines and several GenAI models for German, Russian, Spanish, and Chinese. Many factors influence the performance of MT engines and GenAI models. These factors include the quality of the source content, the combination of source and target languages, the subject matter, and whether the MT engine was trained. In the overall quality comparison, neural MT engines primarily outperform GenAI models, though the GenAI models perform respectably.

Per Language Pair Quality

Choose German, Spanish, Russian, or Chinese in the drop-down menu.

Time

Chart 2: Comparison of the average quality of top MT engines and several GenAI models for specific language pairs.

Per Domain Performance

Select the Domain/Subject Matter in the drop-down menu.

Time

Chart 3: Comparison of the average quality of top MT engines and several GenAI models for a specific domain.

Lionbridge Machine Translation Tracker Methodology

Lionbridge uses the inverse edit distance as a scoring method. The edit distance measures the number of characters (for Asian languages) or words (for Western languages) that need to be changed by a human post-editor before it achieves the quality level produced by a human translator. The higher the metric, the better the quality.

Additional Details About Lionbridge’s Methodology:

We routinely assess the performance of Machine Translation engines and GenAI models, making our findings public periodically.
We provide the data for illustrative purposes; each case should be treated and assessed individually.
We generate this report based on source data preselected by Lionbridge’s Machine Translation teams. The same source data is submitted to every MT engine / GenAI model and language pair each time, making comparisons between the tools possible.
We do not use customer data to generate the report.

Lionbridge Expert Commentary

Our team of seasoned experts closely observes the evolution of MT.

They noted when the paradigm was ripe for disruption and projected the emergence of a new paradigm (generative AI). Their foresight proved to be correct. They have periodically shared insightful perspectives and thought-provoking analyses across a wide range of topics of interest to our readers in expert commentaries.

Read the commentaries to understand automated technologies’ historical context and gain foundational knowledge.

Access the Commentaries

Get Smarter About Automated Translation

Review some of the most engaging content.

Smart MT™: Enterprise-Grade Machine Translation + AI

Discover Lionbridge’s enterprise-grade Machine Translation and AI solutions that utilize the best MT engines and AI reviews for enhanced global communications.

LEARN MORE

Connecting the World: How Diverse Technologies and Human Excellence Enhance Global Communication

Lionbridge executes four key steps as part of its cutting-edge translation services to produce the translation outcomes required by today’s enterprises. Our video shows that the technological elements involve a combination of Machine Translation and generative AI and includes our proprietary, state-of-the-art AI localization platform, Lionbridge Aurora AI™. A human in the loop is equally important to the process.

LEARN MORE

How Lionbridge’s Automated Solution Enabled the Le Monde Newspaper to Boost Its Subscriber Count

When iconic French newsgroup Le Monde released its English service, Le Monde, in English, it needed a translation solution that could handle the complexities of the 24-hour news cycle. Using Lionbridge’s Machine Translation solution, Smart MT, Le Monde was able to slash translation turnaround time, cover the French presidential election, and attract thousands of new subscribers to their service.

LEARN MORE

Lionbridge Now Shares GPT-4 Machine Translation Results via Its Tracker

Explore GPT-4's translation performance via Lionbridge's MT Tracker. Compare GPT-4’s translation results to the translation results of the major Neural MT engines and monitor this generative AI engine’s progress over time.

Note: Lionbridge periodically updates the Tracker, adding or deleting models, to ensure evaluations reflect the latest technological advancements. Since this piece was published, the Tracker has been adjusted accordingly.

LEARN MORE

Know the difference between MT and NMT? Can you define MTPE or hybrid MT engine? Click the image below to view key terms and their meanings to understand Machine Translation better.

If you require additional information, please fill out our contact form.

Send us a message to chat about your needs or learn more about our services and pricing. We are always here for you—drop us a line to let us know how we can help. If you are looking to work for Lionbridge, please visit our "Join Our Team" page.

Let's Talk

Business Email Only

To find out how we process your personal information, consult our Privacy Policy.

Lionbridge Machine Translation Tracker

The Role of Machine Translation in Global Communication

Still essential in an era of technological change

Comparing Four Machine Translation Deployment Strategies

Public MT

On-Premises MT

Cloud MT

Best-of-Breed MT

Which Automated Translation Engine Is Best?

Analyzing Machine Translation Performance

Insights from the Lionbridge Machine Translation Tracker

Lionbridge Machine Translation Tracker Methodology

Lionbridge Expert Commentary

Never Miss a Thing

Get Smarter About Automated Translation

Smart MT™: Enterprise-Grade Machine Translation + AI

Connecting the World: How Diverse Technologies and Human Excellence Enhance Global Communication

How Lionbridge’s Automated Solution Enabled the Le Monde Newspaper to Boost Its Subscriber Count

Lionbridge Now Shares GPT-4 Machine Translation Results via Its Tracker

Language Ranking Based on Machine Translatability for More Effective MT

Machine Translation and Catastrophic Errors

Machine Translation and Formal vs. Informal Language

Using Terminology to Improve Machine Translation

Machine Translation Customization vs. Machine Translation Training

Smart MT™: Machine Translation for the Digital Age

If you require additional information, please fill out our contact form.

Send us a message to chat about your needs or learn more about our services and pricing. We are always here for you—drop us a line to let us know how we can help. If you are looking to work for Lionbridge, please visit our "Join Our Team" page.

Let's Talk

WHAT WE DO

INDUSTRIES

RESOURCES

WHO WE ARE

Lionbridge Machine Translation Tracker

The Role of Machine Translation in Global Communication

Still essential in an era of technological change

Comparing Four Machine Translation Deployment Strategies

Public MT

On-Premises MT

Cloud MT

Best-of-Breed MT

Which Automated Translation Engine Is Best?

Analyzing Machine Translation Performance

Insights from the Lionbridge Machine Translation Tracker

Lionbridge Machine Translation Tracker Methodology

Never Miss a Thing

Get Smarter About Automated Translation

If you require additional information, please fill out our contact form.

Send us a message to chat about your needs or learn more about our services and pricing. We are always here for you—drop us a line to let us know how we can help. If you are looking to work for Lionbridge, please visit our "Join Our Team" page.

Let's Talk

INDUSTRIES