Azerbaijani machine translation with mt5-enaz-10m: a practical guide

mt5-enaz-10m is an open model for Azerbaijani machine translation from English. We published it on Hugging Face in April 2023. This guide covers what the model is, how it was trained, how to run it in Python and what to keep in mind before you put it into a product.

What the model is

The model is based on mT5-base, the multilingual version of Google's T5 text-to-text architecture. We fine-tuned it on our own open corpus translate_enaz_10m: about 10 million English-Azerbaijani sentence pairs, where the Azerbaijani side is original cleaned text from the sources of the National Library of Azerbaijan. That means the model learned to produce Azerbaijani written by people, not by another translation system.

Training at a glance

How to run it

You need the transformers library, sentencepiece for the mT5 tokenizer and PyTorch:

pip install transformers sentencepiece torch

The model expects a task prefix in front of the text: translate English to Azerbaijani: {text}. Here is the basic example from the model card, with deterministic (greedy) decoding:

from transformers import MT5Tokenizer, MT5ForConditionalGeneration

model_name = 'learningmachineaz/mt5-enaz-10m'
tokenizer = MT5Tokenizer.from_pretrained(model_name)
model = MT5ForConditionalGeneration.from_pretrained(model_name)

text = "Artificial intelligence is already superior to human learning in numerous domains."
input_ids = tokenizer(f'translate English to Azerbaijani: {text}', return_tensors="pt").input_ids
outputs = model.generate(input_ids, max_length=128, do_sample=False, num_return_sequences=1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# Süni intellekt artıq çoxsaylı domenlərdə insanın öyrənilməsindən üstünlük təşkil edir.

Several variants with sampling

Sometimes it is useful to see more than one translation, for example to let an editor pick the best phrasing. Switch on sampling and ask for several sequences:

outputs = model.generate(input_ids, max_length=128, do_sample=True,
                         top_k=10, num_return_sequences=3)
for out in outputs:
    print(tokenizer.decode(out, skip_special_tokens=True))

With sampling, each run can return different variants. For reproducible results in a pipeline, keep do_sample=False.

Translating many sentences at once

For a batch of texts, pad the inputs to the same length and decode every output:

texts = ["The meeting starts at ten.", "Please send me the report."]
batch = tokenizer([f'translate English to Azerbaijani: {t}' for t in texts],
                  return_tensors="pt", padding=True)
outputs = model.generate(**batch, max_length=128, do_sample=False)
for out in outputs:
    print(tokenizer.decode(out, skip_special_tokens=True))

A GPU makes this much faster, but the model also runs on a CPU for small volumes.

Practical tips

• Keep the prefix exact. The model was trained with translate English to Azerbaijani:; a different wording can degrade the output.
• Translate sentence by sentence. Generation is capped with max_length=128, so split long documents into sentences or short paragraphs and join the results.
• One direction only. The model translates English into Azerbaijani; it is not trained for the reverse direction.
• Check domain terminology. The training texts are books, news, poems and articles. For legal, medical or technical content, review the output or fine-tune the model on in-domain pairs.
• Keep a person in the loop for published text. Like any machine translation, the output reads best after a human edit when quality matters.

Licence: non-commercial

mt5-enaz-10m is released under CC BY-NC-SA 4.0. You may use, share and adapt it with attribution, for non-commercial purposes only, and derivative works must be shared under the same licence. Research, teaching and personal projects fit these terms. For commercial use, contact us.

Where to go next

The model and its card are on Hugging Face, and the project page collects the details. If you need Azerbaijani machine translation, text classification, search or another language task built into your product, see our natural language processing service or describe your task.