-
Improvement
-
Resolution: Unresolved
-
None
-
None
-
High
-
Emptyshow more show less
Problem:
When exchanging the visual in a batch or importing a task with several target-languages, all workloads of all visual workers before the segmentation-worker are processing identical workloads. That spoils resources, time and costs.
Solution:
Only for one task (master-task) the import/exchange of the visual is done and before segmentation the transformed resources are copied over to the other tasks/workers.
Challenges:
- The imports waiting for the cloned resources may wait forever, when the master-task fails. "Communication" between workers only possible with flag-files.
- The process cannot be achieved for VIDEO visuals and XML/XSLT visuals - what is no loss since there no resource-hungry/expensive services are involved
- The detection, a multi target-language import is a batch does not exist yet and this must be implemented in the core first to enable batching for imports
Implementation Idea:
- One task will import the visual (master-task), a special "CloneToOthers" worker will be added before segmentation with the other batchTaskGuids and the operation-type as param
- This worker copies the unsegmented visual and the linked resources (contents of /visual) to all batch-clones.
- It also adds a special JSON-file "t5-visual-cloned.json" containing all added DataModels during import in JSON Format (entries from LEK_visualreview_files & LEK_visualreview_font_taskassoc)
- The other tasks are queued with a special "ImportClonedResources" Worker (with the master-task Guid and the operation-type as param) that will add the file-based resources from "t5-visual-cloned.json" to the data-base. This worker will delay itself with an interval of 60 sec until he finds the "t5-visual-clone.json". After 5 min waiting it always checks, if the master-task is not erroneus & then also fails. As usual, after max-delay-time the import/exchange fails as with other delays