Uploaded image for project: 'translate5'
  1. translate5
  2. TRANSLATE-4781

translate5 AI: do not merge user defined system prompts on fine tuning

XMLWordPrintable

    • Critical
    • When generating jsonl file, training pairs are now sent with assigned system message from the prompt

      Problem

      When jsonl file is constructed for fine-tuning, all user defined system messages are merged into 1 large system message and added for each example set. 

      Each user define system message should only be added to each example set where it belongs.

      The final line should look like this:

      {custom t5 sys msg + user sys message + custom t5 post sys msg}

      {example set}

       

      What is:

      custom t5 sys msg -> 
      "You are a machine translation engine and translate single texts or multiple segments {fromTo}"
      custom t5 post sys msg -> "Translate the following text _

      {fromTo}

      "_

            leonkiz Leon Kiz
            aleksandar Aleksandar Mitrev
            Aleksandar Mitrev, Axel Becher
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: