Creating and Evaluating Multilingual AI-Generated Content

This is the first piece in the two-part Lionbridge GenAI Content Use Cases Series, which explores Generative AI’s multilingual content creation and optimization abilities. Part two will be linked here.

Theoretically, generative AI can create and optimize content so that it’s engaging and has strong SEO value. Sometimes, it may even deliver copy better than humans. For example, our team has noticed that GenAI tools are sometimes better at writing and translating with strict character limits. Does AI-generated content actually deliver on these content creation and optimization promises, though? Can it create and optimize content in languages besides English? Lionbridge’s generative AI content creation experts will explore these questions using four AI content creator tools to optimize content in German.

Read about our methodology and approach here.

AI-Generated Content for New Languages

This experiment will cover three specific use cases that are helpful for digital marketing.

1. Optimizing existing pages in German: As mentioned above, we’ll focus on German web pages in this case. For more impactful digital marketing, it’s ideal to translate and optimize current content in new languages. (Professional translation companies can assist with this via website localization services.) Localizing content allows an organization to reach new markets.

After localization, it’s crucial to ensure the page performs. Now, AI-generated content may be part of this process. In this experiment, we’re exploring whether AI-optimized German versions of web pages perform by:

• Ranking well in the SERPs

• Drawing in users via organic search

• Converting well if paid ads point to it

• Being engaging for visitors when a series of social posts promote the page

2. Extending the life of older posts: Digital marketing is always better if web pages can perform and continue to be impactful. We’ll consider if we can use an AI content generator to:

• Add new H2 text

• Add a paragraph or two with new information, trends, developments, or questions on the topic

• Remove old information and dates

3. Create new content based on existing content: It’s especially impactful to create new content in other languages based on effective existing content. For this experiment, we’ll use AI to generate a product or service landing page based on existing product or service copy.

Guiding Questions and Tools to Evaluate AI-Generated Content

In this experiment, we’ll use four tools: Amazon Bedrock, ChatGPT, Bard, and Llama (Meta’s generative AI tool). These are our guiding criteria for the AI content we’ll generate:

• How well can these tools do keyword research?

• How well do the tools generate questions for the content?

• How well do the tools generate angles for the articles?

• How well do these tools create answers to the questions they generated?

Methodology

Our methodology will include these nine steps:

1. Create and utilize prompts for the four generative AI tools mentioned above in German. Add in language to ensure the tools do not “hallucinate,” or offer erroneous information in their results.

2. Have prompt engineers review the prompts for efficacy.

3. Ask the four different generative AI tools to do keyword research and generate questions to drive content creation.

4. Review the responses and select the best results.

5. Create persona prompts (in English). Use them to require the GenAI tools to answer the selected questions as experts. Include a stipulation in the prompts that the tool should never guess the answers (this will reduce hallucinations).

6. Use our own localization services to get the prompts localized into German. Then, have the prompts reviewed by a native German speaker.

7. Run both the English and German sets of prompts using each of the four tools.

8. Have a panel of native German speakers (including business and language specialists) review the AI-generated content. They should check for how engaging the content is and how effective any SEO text is. Make the review “blind” by not marking or sharing which tool created which content.

9. Compare the evaluations of the content, tool by tool. The results will show which tool is better at which tasks, including content creation, producing keywords and questions, and SEO value of the content, keywords, etc.

Want to find out the results?

Check out our upcoming in-depth case study with the results of this experiment.

Get in touch

Discover how Lionbridge’s content creation services can help your organization. Need assistance with your website content optimization? We utilize human reviewers and innovative generative AI technology to create customized translation solutions. You can trust our decades of experience building language services. Get in touch today.