CSV import does not work with OKAPI nor with the internal CSV parser

XMLWordPrintable

    • Critical
    • None
    • Hide
      FIX: csv-files could not be imported anymore because OKAPIs okf_multiparser Filter does not work properly, use okf_table_csv instead.
      FIX: internal CSV parser to properly process UTF-8 files with BOM
      Show
      FIX: csv-files could not be imported anymore because OKAPIs okf_multiparser Filter does not work properly, use okf_table_csv instead. FIX: internal CSV parser to properly process UTF-8 files with BOM
    • None

      PROBLEM
      The import of a basic CSV file (UTF-8 encoded, comma-seperated), content:

      id,source,target
test,test,test

      does not work with our default file-format-settings (where OKAPI with okf_multiparser Filter is used), which results in a "Okapi Plug-In: Error in converting file "test.csv" on import: HTTP Status was not 200/201/401 body: org/apache/commons/io/input/UnsynchronizedBufferedReader". It also fails when using the internal CSV fileparser (config "runtimeOptions.import.fileparser.csv.active") and produces an "In application.ini configured column-header(s) 'id' not found in CSV: 'test.csv'" error despite there is such column.

      SOLUTION

      • for OKAPI: Make okf_table the default for CSV parsing
      • for internal-parser: Fix parser to properly handle BOMs

      FRONTEND-TESTS

      • Import CSV files with and without having runtimeOptions.import.fileparser.csv.active active
      • Import the attached CSV with BOM both ways

        1. test-with-BOM.csv
          0.0 kB

            Assignee:
            Axel Becher
            Reporter:
            Axel Becher
            None
            None
            Aleksandar Mitrev
            Sasha
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved:
              None
              None