-
Type:
Bug
-
Resolution: Fixed
-
Affects Version/s: None
-
Component/s: TermTagger integration
-
High
-
None
-
Fix processing of segments with characters that may be escaped by TermTagger
-
None
-
Emptyshow more show less
Problem
When a segment text contains characters such as `>` (commonly seen in navigation instructions like `"Connexions > Interfaces"`), the external TermTagger service converts these characters into HTML entities (such as `>`) in its response.
Before applying terminology highlights, Translate5 checks that the text returned by TermTagger matches the requested text. Because `>` does not match `>`, Translate5 assumes the text was modified or corrupted. As a result, Translate5 logs a warning (`E1696: TermTagger changed the plain field text; terminology markup was skipped for this field`) and discards all recognized terms for that segment. The user therefore does not see their terms highlighted in the editor.
Solution
- TermExtractor: Added support for decoding HTML entities ($decodeHtmlEntities = true) when extracting terms from TermTagger responses, which decodes entities back to plain characters and computes term $start and $end character boundaries directly against the decoded text.
- TermFinder: When the raw response string from TermTagger does not match the requested segment text, attempt extraction with entity decoding enabled. If the decoded text matches the requested text, accept the terms and their calculated offsets without logging warning `E1696`.
Test Case for Manual Testing
Preconditions:
1. A project/task with TermTagger enabled and a termbase assigned containing at least one term present in the test segment (e.g., term: `KNX` or `adresse IP`).
Test Steps:
1. In the editor open a segment containing a `>` character followed by a known term, for example:
> `dans l’onglet Bus, sous Connexions > Interfaces, relever l’adresse IP du module KNX`
2. Trigger terminology tagging for this segment (e.g., update and save the segment).
3. Inspect the segment in the editor UI.
4. Check the application log (data/logs/php.log or in task/system log).
Expected Result:
- The terms (e.g. `adresse IP`, `KNX`) are properly recognized and highlighted in the editor.
- The term markup wraps the exact words (offsets are not shifted or displaced).
- The warning `E1696` (`"TermTagger changed the plain field text; terminology markup was skipped for this field"`) does *not* appear in the log.