Problem

AI pre-translation and TQE use the terminology of the task. They look up the terms segment by segment.

For every segment, translate5 loads all terms of all term collections that are assigned to the task. It does this more than once per segment. With small term collections this is not noticeable. With large term collections (many thousands of terms) this can take seconds per segment.

As a result, AI pre-translation and TQE can take many times longer than needed. The pre-translation workers stay busy longer, so other tasks wait longer in the queue. In such cases the import of a task can take hours instead of minutes.

Solution

• Load the terms of a task once and reuse them for all segments of the same task. Do not load them again for every segment.
• When a different task is processed, load the terms of that task.
• If the term tagger finds a term that is not in the loaded terms (for example because the terminology was changed in the meantime), load the terms again. This uses the existing retry.
• The terminology lookup of AI pre-translation and TQE keeps one term tagger service for the whole run, so the loaded terms are really reused.