Improvement of parallel working workers of different type

XMLWordPrintable

      Currently the workers are started sequentially per worker type and taskGuid.
      That means, all queued workers of one task are processed (with parallel running workers iff permitted per worker type), after finishing one task, the workers of the next task are started.
      This should be improved, so that workers of different tasks are allowed to run parallel.
      For example after running the termtaggers of task one, the termtaggers of task two can be started, at the same time the statistics can be created for task one.

      This is indeed very complex, since we have workers like the termTagger which are blocking the whole resource, so that multiple termTagger workers of different tasks are blocking each other.

      The main hook ins for this task will be:

      • ZfExtended_Models_Worker::wakeupScheduled Method which is responsible for settings workers from scheduled to waiting.
      • ZfExtended_Models_Worker::getListQueued which returns the waiting workers which can be started next

      As test case for implementing parallel workers can be used BEO-12.
      The original SQL in the wakeupScheduled method was containing also a taskGuid = ? in the inner EXISTS select.
      This led to following error:

      1. Task 1 queue termTagger → was started immediately after queue it
      2. Task 1 queue stat → is not started because of above running task
      3. Task 2 queue termTagger → is also started if there is a free termtagger slot
      4. Task 2 queue stat → is started immediately after queue it, which should not be, because the task is not tagged already. Why this is the case, is currently unclear. By removing the above mentioned "taskGuid = ?" all workers have to wait to the before finished tasks.

            Assignee:
            Marc Mittag [Administrator]
            Reporter:
            Thomas Lauria
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: