Problem

AI and MT pre-translations often contain segments that are not good enough. Today a human must find and correct these segments. Sometimes it makes sense to let the system post-edit a pre-translation automatically: send the segment back to an AI resource and ask it to improve the translation. If translation quality estimation (TQE) is available for the segments, the TQE score should decide which segments get post-edited, and the post-editing can be repeated in a loop until the score is good enough.

Solution

In the task

Introduce a new tab analog to the tabs for language resources, pivot language resources and TQE language resources. In this tab a resource for automatic post-editing can be assigned. Only AI (LLM) resources are listed. Like for TQE, only 1 language resource can be assigned per task.

The new tab has the following configuration options. They are only selectable if a TQE language resource is also assigned:

• Do auto-post-editing, if TQE below X: "TQE score threshold per segment to start auto-post-editing"
• Repeat auto-post-editing up to Y times, if TQE stays below X: "Number of loops"
• Only in case the same language resource is assigned for TQE and for auto-post-editing: checkbox: Query for auto-post-editing in same prompt as for TQE Can be a later on improvement.

If TQE threshold after Y loops is still not met, keep best-result output in the segment.

Defaults for these options are definable on system and client level. The same panel is also available in the import wizard, analog to the TQE wizard panel.

If no TQE resource is available: Do the post-editing exactly once. 

In the language resource settings

Definition of pre-defined language resources for automated post-editing is done analog to TQE, including InstantTranslate:

• add "Use for automatic post-editing per default"
• add "Use for automatic post-editing in InstantTranslate per default"

Both options are only visible for AI (LLM) resources.

If a user assigns a duplicate for the same client and language combination, the user is prompted to pick which one stays, analog to the existing TQE logic.

Runtime behaviour

Automatic post-editing runs as a worker during the import, after the pre-translation. If a TQE resource is assigned, it runs after the TQE step.

Which segments are post-edited:

• Only segments that were pre-translated and not edited by a user.
• Locked and blocked segments are skipped.
• If 100% and higher should be skipped, "edit 100% match" should not be set for task import
• Repetitions: First occurance will be post-edited, repeated segments should be auto-propagated
• With TQE assigned: only segments with a TQE score below the configured threshold X.
• Without TQE assigned: open question, see comments. Proposal: every pre-translated segment is post-edited once, without loops, because there is no score to decide with.

The loop with TQE works like this: the segment is post-edited, then TQE runs again on the new translation. If the new score is still below X and the maximum number Y of loops is not reached, the segment is post-edited again. The TQE reasoning of the last run is passed into the next post-editing prompt.

Which translation is kept when the loop ends: open question, see comments. Proposal: keep the attempt with the best TQE score, also when the loop ends without reaching the threshold. A post-edit that lowers the score is then never kept.

After post-editing a segment:

• The stored TQE score of the segment is updated to the score of the kept translation.
• The segment keeps a match rate and origin that show it was auto-post-edited, so users can filter for these segments.
• The internal tags of the post-edited translation are checked and repaired with the existing tag repair mechanisms, like in pre-translation.
• AutoQA runs on the changed segments.

Post-editing multiplies AI requests: up to X+1 translation requests plus X TQE requests per segment. The requests are sent in batches and use the parallel request mechanism per resource (TRANSLATE-5226), so large tasks stay fast.

InstantTranslate

In InstantTranslate there is no task tab where a user assigns resources. The post-editing resource comes from the resource defaults: for each client, the AI resource with the checkbox "Use for automatic post-editing in InstantTranslate per default" is used. This is analog to how the TQE resource is selected for InstantTranslate today.

Automatic post-editing in InstantTranslate applies to file translations, analog to TQE. The post-editing worker runs in this pipeline like for a normal task: after the pre-translation, and after the TQE step when a TQE InstantTranslate default resource exists. The full loop with threshold and repeat count works here too. The threshold and loop settings come from the system and client level defaults.

The user only sees the final result: the translated file contains the post-edited translations, and the overall quality score shown for the file translation is calculated after post-editing, so it reflects the improved translations.

Prompting

In the automatic post-editing we pass the AI source and target and ask it to post-edit the translation where it thinks it needs to be enhanced. Like all other AI resources, prompts from the prompt library can be used to customize the prompting.

If a TQE result and reasoning exists, we pass this along and ask it to enhance things.

For the combined mode (checkbox "Query for auto-post-editing in same prompt as for TQE") one request returns both values: the quality estimation and the improved translation. This needs a new response format and a new result processor, because the current TQE response only carries the score and reasoning.

Shared mechanism with TRANSLATE-5611

The loop "post-edit with TQE reasoning, run TQE again, repeat up to a maximum" is the same mechanism that TRANSLATE-5611 needs for its hallucination safeguard. The loop is built as one reusable worker with the trigger, the threshold and the loop count as parameters. TRANSLATE-5611 uses it always-on with a length-based trigger; this ticket adds the assignable resource, the task tab and the configurable thresholds on top.