InstantTranslate: show the user when a pre-translated file contains untranslated segments

XMLWordPrintable

    • Type: New Feature
    • Resolution: Unresolved
    • None
    • Affects Version/s: None
    • Component/s: InstantTranslate

      Problem

      When InstantTranslate translates a file, the hidden task is pre-translated in batches. If some of those batch requests fail (rate limits, provider errors - logged as E1783/E1785 since TRANSLATE-5226), the affected segments simply stay untranslated. The run is not aborted and the task reaches its normal end state, so the InstantTranslate file list presents the file as a full success: green header/status and a normal download button. The user gets no hint that anything is missing.

      The delivered file actively hides the problem: files imported via Okapi (docx, pptx, pdf, … - the normal InstantTranslate case) are exported with "source to empty target" always on (TRANSLATE-2384), so untranslated segments come back filled with source-language text. The document looks complete unless the reader notices the wrong language. Only native XLF/CSV uploads produce visibly empty targets.

      Two mechanisms that should or could surface the problem are broken:

      • FileTranslationDownloads::taskIsNotPretranslated() calls array_key_exists(NOT_TRANSLATED, ...) on the result of Segment::getAutoStateCount(), which returns a plain list of rows, not a map keyed by autoState - the check has never detected untranslated segments since its introduction. Even if fixed, it is all-or-nothing: it would block the download completely instead of reporting a partial result.
      • The errors field of the file list response is hard-filtered to the log domain editor.languageresource.service.connector. The batch pre-translation warnings E1783/E1785 are logged under editor.languageresource.batchquery (and re-logged chunk exceptions under core / plugin.matchanalysis), so they are structurally excluded - although they already sit in the task log and E1785 even carries the count and segment numbers.

      The synchronous API (filepretranslatenow) is worst: it waits only for the import state to end and returns the file body with no completeness signal at all.

      Solution

      • After pre-translation, determine the number of untranslated segments per file-translation task from the segment autoState statistics (editor_Models_Segment_AutoStates::getStatistics() returns a correctly keyed autoStateId → count map) and replace the broken array_key_exists check in taskIsNotPretranslated() with it.
      • Extend the file list status contract with a partial state instead of the current all-or-nothing logic: when some but not all segments are untranslated, keep the download available and additionally deliver untranslatedSegments / totalSegments (e.g. via the existing additionalData collector mechanism that TQE already uses - both frontends read it already). A fully untranslated file keeps the existing isNotPretranslated error state.
      • Widen the log-domain filter of the errors field so batch pre-translation warnings (domain editor.languageresource.batchquery, E1783/E1785) appear in the collapsible errors block of the file entry.
      • Frontend: render the partial state as a warning, not a success - e.g. yellow/orange header (legacy) resp. warning chip (React rework) with a text like "N of M segments could not be translated", download button still available, errors block visible. The React frontend must also stop hiding row.errors when the status is not error.
      • Synchronous endpoint filepretranslatenow: return the untranslated-segment count as a response header (e.g. X-Translate5-Untranslated-Segments) so API consumers can detect a partial result without parsing the file.
      • This covers failed batches and any similar cause (no TM/MT match above the threshold, connector down mid-run, …) uniformly, since it is based on the actual segment state, not on a specific error source.

            Assignee:
            Aleksandar Mitrev
            Reporter:
            Aleksandar Mitrev
            Aleksandar Mitrev Aleksandar Mitrev
            Marc Mittag [Administrator]
            Axel Becher
            None
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              None
              None