problem

Currently the workers are triggered via HTTP requests, containing the whole overhead of creating HTTP requests and apache dealing with them. Also the workers are basically using and blocking free apache slots for ordinary requests.

The overhead of a small import was enormous: 2 seconds for the import instead of 14 seconds. For bigger tasks that may not be relevant, but still it saves performance in not triggering apache and the network.

The workers via HTTP requests are hard to debug, since its never really clear which apache process belongs to a running worker.

solution

Start the workers as independent processes.

Open:

• more logging and debugging for process based approach → worker and taskGuid can be listed with top now
• use the pid as process id in the worker table - for dead worker entry recognition
• implement dead running worker entry cleanup → on longer as a day base TRANSLATE-514
• implement switch to change between processes and http → DONE