Problem

Term tagging can fail or place term tags incorrectly when a segment contains literal pipe characters (|), content-protection tags, or HTML entities such as ® and °.

The protected content previously used | internally, making it indistinguishable from user-entered pipes. Additionally, entity conversion by TermTagger changed text lengths and shifted term positions.

Solution

• Use a collision-free private placeholder for protected content instead of `|`.
• Normalize HTML entities before sending text to TermTagger.
• Map returned term positions back to the original segment text.
• Verify that TermTagger did not change the plain text.
• If a response or position is unsafe, leave only the affected field untagged and preserve its original content and tags.
• Log detailed `E1696` diagnostics for rejected responses.

  Manual Testing

1. Process the supplied segment containing multiple pipes, ®, and °.
   - Term tagging completes without an overlapping-position error.
   - All literal pipes remain unchanged.
   - Terms before and after ® and ° are tagged correctly.
   - Original internal tags remain intact.

2. Process a segment containing both literal pipes and content-protection tags.
   - Pipes remain visible and unchanged.
   - Protected content is restored correctly.
   - Terms around both elements are tagged correctly.

3. Process a segment containing the preferred private-use character and a content-protection tag.
   - The original character remains unchanged.
   - Protected content is restored correctly using another internal placeholder.

4. Test terms containing an entity, for example `CORDURA ®` or `60°C`.
   - The complete term is tagged.
   - The entity remains in its original encoded representation.