Details
-
Improvement
-
Resolution: Fixed
-
None
-
High
-
Enhancement: Remove Rule from t5 default file-format settings that tried to segment malformed sentences "A sentence.The next sentence." but did more harm than good
-
Empty show more show less
Description
Remove Rule from t5 default file-format settings that tried to segment malformed sentences "A sentence.The next sentence." but did more harm than good.
The rule is
"[\.!?…]['"\p{Pe}\u00BB\u201D]?" => "\p{Lu}[^\p{Lu}]"