Issue purpose

In this issue we collect all code places which should be refactored / improved, containing general improvements.

Import of multi-target tasks

• Currently we basically process every task as singular (complete) import
• This is major waste of performance as apart from the pretranslation/targets, all data is identical. Some industry clients import 15-30 languages regulary per task
• Long-term/Ideal Programming: The Import is changed so it uses one (or several) JSON file(s) as data-container for the task throughout the import (which would radically reduce DB-strain -> Deadlocks). As a later step, this file is flushed to the DB-Model. This would enable a "natural" capability for cloning tasks based on these file(s) and triggering a pretranslation afterwards for the sibling tasks after the first is imported. Throughout the file-based process, the segmentNrInTask acts as segment-id. A potential Pivot-workfile also has to be processed to a file-based separate model to be re-usable for the sibling-tasks
• Short term Improvement: Workloads to several Docker-Containers could be adjusted to cache the workload for ~1d with a hash of the workfile as key and deliver the cached result in case of multitarget tasks. The containers are: Visual PDF-converter (hash: PDF-file), Visual browser (hash: html-file, only for reflow), OKAPI (needs to be implemented in t5, hash: workfile), Visual Image Conversion via Google Cloud Vision (not very relevant, has to be done in t5, hash: image)

System structure

• Introduce a class loading map for productive systems to improve performance (should be implicitly solved for namespaced classes)

Search and Replace

• JS: Window and tab class structure, tab creation structure unclear / to complicated
• invocation which tab (search or replace) partly to complicated

Extending Services Layer

• request recovery on timeout  / DOWN - if needed / wanted

Import

The import preparation steps (Okapi, Visual Review preprocessing) should be callable in a DB independent manner.
When all preparation is done, the prepared data should go into the database.

pros: The preparation steps could be run on a different server. The preparation is task independent and can be run once per project (and not like at the moment once per target language in a project)

Worker

The default worker state on queueing should be scheduled instead waiting. If waiting is needed that should be explicitly defined. That should be less error prone as currently waiting: see also TRANSLATE-3556

Workflows

The workflow system is to inflexible, since it is basing on fixed PHP classes, which indeed can be overwritten, but this does not solve the following problem:
Plugins should be able to introduce new workflow steps.

This results in the following refactoring:
Currently a workflow defines it self and the possible steps. What we need is a separation: The possible workflow steps (and the dependencies to each other) must be definable solely. So a new workflow step can be defined also in Plug-Ins.

The definition of a workflow will then change to: A workflow is a composition and order of workflow steps, and a composition of actions/notifications triggered on specific transitions between the steps.

The latter one is already achieved with the LEK_workflow_action table.

Current Task

The current opened task is stored just as taskGuid in the session. There is no encapsulating code surround that.
What we need is a currentTask singleton (or so) which is one dedicated interface to get the current task. This enables easier change away from taskGuid in session to taskGuid in URL for example. (See TRANSLATE-2534)

Current authenticated User

Same problem as for "Current Task" currently we just take the value from the session without any wrapper class (like a Authenticated User singleton or so). ZfExtended_Authentication introduced with TRANSLATE-3051

Task

• Eliminate the "terminologie" field in the task entity. Where it was currently evaluated the associated language resources (in that case term collections) should be checked instead. We had multiple problems with that field when it was not up to date. Latest issue about that TRANSLATE-3282
• Rename emptyTargets to a more understandable name: translationOnImport or similar

ZfExtended merge into main repository

• pro: reduced the submodules
• Still it remains a lib, so no editor_ classes should be used there. Only ZfExtended in editor.
• Cleaning up all editor_ references which are already in ZfExtended. Thats mostly editor_Models_Config
  which should be a ZfExtended_Models_Config anyway

Session

The sessionInternalMappingUniqId should be moved into the session table itself.

Also the current authenticated userGuid / login should be stored in the session table, would make clean ups easier.

-https://jira.translate5.net/browse/TRANSLATE-3507-

API

• Accept raw POST / PUT data in addition to the currently needed data variable
• simplified filters in addition to the current complex filters in extjs syntax
• file uploads as normal raw POST / PUT in addition to the current multipart/form/data call (if possible)
• See TRANSLATE-1068 and linked issues

ACL

• The current roles should be used as templates of roles
• The concrete roles, additional / custom roles should be addable in the UI by using above templates and add / remove rights
• Advanced right / role control in UI
• each role and right should get a meaningful description (implement rights as class constants → there could also go the documentation in docblock and read out automatically).
• This was already concepted somewhere but know suitable issue was found, so its added here. See also TRANSLATE-3223 and

Implement a Model/Entity Instance "Cache"

• In the lifetime of a request for example the current task is loaded very often, this could be improved by providing a convenient way of loading such an entity and a internal caching of such an entity. It is no real cache since it is not persistent over different requests, but at least only one DB request is needed instead multiple ones.
• Also we have multiple static field based caches in the application exactly doing that on the place where it is needed, this code can be removed by using the instance cache.
• → Created with the class editor_ModelInstances
• Introduce the Repository layer → moving bulk loading methods (not single load methods) from Model to Repository

 Error Logging

The database based logging leads to big and slow tables. Consider to switch to another backend. Filebased one JSON per line?

Internal Tags

The internal tags div span construct should be eliminated (see TRANSLATE-828) → axelbecher is working on it (https://jira.translate5.net/browse/TRANSLATE-3047)

Internal Translations

See TRANSLATE-3389 and linked issues!

Database

https://jira.translate5.net/browse/MITTAGQI-245

 
 
 Remote filters and sorters

The current definition and implementation of the remote sorters and filters on the backend is difficult to work with. Better structure is required.