Problem

The larger the text, that is send to an LLM, the more unspecific gets the info to the LLM, what it should do/what this is about. That means, the more segments you send, the worse is the quality.

On the contrary, the less segments are send the higher are the costs because the overhead (system messages, examples) stay the same.

Solution

Users need to be able to define themselves, how many segments are send in one request (be it translation or quality estimation) to the LLM.

The max should still be defined by the max number of tokens, that can be send in one request. So this should still be checked before the request is send. And if needed, the payload should be split across multiple requests (will only be valid for very high number of configured segments/request usually).

The config should be an AI language resource property and its default is a system config. The default of the default should be 6.