problem

For scalable pooled services such as Spellcheck / LanguageTool, TermTagger and client specific tagger in private plugin, translate5 currently derives the number of parallel workers from the number of reported service nodes/IPs.

This limits throughput in cases where a service can handle multiple concurrent requests per reported endpoint. Examples from hosting:

• Spellcheck currently reports one endpoint, so only one worker is queued, although the service can handle more parallel work.
• TermTagger now has a proxy in front of the actual nodes, so translate5 may only see one endpoint even if more capacity exists behind the proxy.

As a result, available service capacity is not fully used and large workloads can be processed too slowly.

solution

Adapt the handling of max_parallel for scalable / pooled workers so it can be used as a configurable multiplier or effective parallelism factor per service.

The intended behaviour:

• keep automatic detection of available pooled service endpoints
• allow each scalable service to define how many workers may run per reported endpoint
• apply this to Spellcheck / LanguageTool, TermTagger and client specific tagger in private plugin where applicable
• keep the existing hard upper limit of 20 workers per service as protection against misconfiguration
• ensure all hosting instances, including new instances, can use the optimized defaults/configuration
• document where the value is configured and how hosting can tune it per service

Test instruction

• Use a big task
• At least so many termtagger workers should be running as termtagger max_parallel is set