Remaining TermPortal todos prior first release

Most important: In my TermPortal suddenly the processStatus attribute is not shown any more at all. I did pull today. Perhaps something there is incomplete right now?

1. Link in main translate5 of the TermPortal tab there needs to be adjusted. At the moment it still points to https://marc.translate5.net/editor/apps?name=termportal&apiUrl=/editor/termportal, but should point to https://marc.translate5.net/editor/termportal
2. Please remove the black border around the iframe-content on the InstantTranslate page, when it is integrated into ExtJs
3. There is a 404 error on trying to load https://marc.translate5.net/client-specific/termportal.css. Yet if this file exist is optional. So if it does not exist, no error should happen
4. When editing an existing term, currently an empty text  field appears. Yet the existing term should be content of this text field and the cursor behind the term
5. Please implement the things listed in attached screenshots
   1. Begriff-portal.png, except:
      1. Geographische Verwendung
      2. Deutsch (and for English: Englisch)
   2. Begriff-portal(1).png
   3. Begriff-portal(2).png
   4. Term-Portal(10).png
   5. Term-Portal(2).png
      1. top and center things
      2. right thing
      3. left thing
   6. Term-Portal(3).png
   7. Term-Portal(5).png
   8. Term-Portal(6).png
   9. Term-Portal(7).png
   10. Term-Portal(9).png
   11. translate5(30).png - there is no need for that roles to be set
   12. translate5(31).png
   13. InstantTranslate(1).png
   14. Annotate-a-local-image(12).png
6. initial images folder for a termCollection inside of data/tbx-import/term-images-public is not created, when a new termCollection is created through the GUI without selecting a TBX. Therefore to those TermCollections no images can be added.
7. When uploading a tbx with images, they appear in the Termportal. Yet adding new ones through the UI still does not work.
8. For the first TermPortal release we need to hide the "Batch edit" button, since it does not work so far
9. Trying to add a new term with a comment that contains line-breaks, leads to an 400 error on the server side (at least on linux)
10. When clicking the "Add comment" button on language or entry-level nothing happens. At least if a comment on term level already exists.
11. The "Definition" attribute should have a text area instead of a text field.
12. If all available attributes are already used and the plus icon is pressed, a message should appear:
    1. DE: Alle verfügbaren Attribute sind bereits in Verwendung.
    2. EN: All available attributes are already in use.
13. "Merge term entries" should be "off" by default in the "Add language resources" and in the "Import TBX file" dialogues in the language resources area of translate5 (I know, not TermPortal, but since Aleks is in vacations it would be good if you change this).
14. This link produces a 400 error on the server and therefore the middle and right columns do not load: [https://marc.translate5.net/editor/termportal#termportal/refEntry/LCwsLGlkNjUzMWI0MjYtOGIyZC00NGU1LWE0YjAtYmI2Yzk2ZmY0MzYz]
15. The left search result grid has rendering problems. To reproduce
    1. Search for something with a lot of results (at least a few hundred)
    2. Scroll down
    3. Search for something else with at least a few hundred results
    4. Now the search result grid stays empty until you start to scroll up and down a few times. Then the results are shown. This seems to be related to an ExtJs bug that caused a similar problem in the segment grid of translate5s editor. Please talk to Thomas and Aleks about it.
16. In the filter the attributes should have the same GUI text as they currently have in the drop-downs where you can add them. Figure must also be renamed to "Image" in the GUI (of course not on database/tbx-level)
17. Text field-based attributes should be saved on leaving the attributes text field. Still the save-button should stay.
18. When adding a new attribute, after selecting one from the drop down list it should automatically be added as new attribute, without having to press the "Save"-button next to the add-attribute-dropdown.
19. Searching for text-field or text-area-based attributes should always search for the search string as part of the value (fuzzy search). Currently the value must match exactly.
20. It should be possible to select the text of terms and attributes (at least in the middle and right column) in order to copy it.
21. Search result panel seems to query without scrolling already 11 pages with each 25 records. This is much to much. It should only query one page with as many records as it is needed to fill the left column and scroll a bit. I think 50 terms/request is a good value here. The next page should only be queried, when needed. I have the impression it would be good to talk with Thomas about this, because he has a lot of experience with these kind of ExtJs problems from the translate5 editors segment list.
    1. Changed from 25 to 250, so now search result panel query without scrolling only 1st page, but it's doing it twice somewhy. This should be investigated and fixed in the termportal release #2. fixed.
22. Display of attribute values:
    1. There are values that use camelCase, like "shortForm". They originate from TBX basic standard and are correct like this on DB level and in TBX. Yet in the GUI they should show like "Short form".
    2. Also please make sure, that in the GUI the attriute values always start with an uppercase letter.
23. In my terms_term table I do now have 860029 terms. Each search query when usiing * does now take around 3 seconds. This is much to long, as it is very likely that real life DBs are much bigger, if really all terms of alll termCollections are stored in the same table. We must very much improve the performance here. If this is not possible with one table, we must use separate tables: One for each termCollection. Yet I think you said something about partitioning of tables or something like that. Anyway: Speed must we very much improved. Even with 50 Million terms in a translate5 instance a query should not be longer thant 0,5 seconds. Please discuss this in a team with Thomas and Axel, as this is a complex design question and they are as you are very experienced developers.
    1. Performance of '/terminfo'-requests should also be improved, as currently it tooks more than 5 seconds:
       1. mysql-indexes should be added on columns, that are involved in '/terminfo'-queries
       2. as an additional step, '/terminfo' inner logic may be amended, for level-distinction for queried attribute/transacgrp-recrods to be made on php-level rather than on mysql-level, as for on mysql-level it requre complicated WHERE clause, and moving it to php-level will allow to replace current WHERE clause with just `termEntryId` = ?
    2. Datatype of term and proposal columns in terms_term table should be changed from TINYTEXT to VARCHAR(255) due to PDO bug related to excessive RAM usage for BLOB-data, that TEXT and TINYTEXT is a subtypes of.
    3. Make a preliminary preparation for a user-provided query before using it in MATCH AGAINST  IN BOOLEAN MODE statements due to how it works, e.g. query should be splitted on words containing no non-alphanumeric characters
    4. Re-implement point 30 (termCollection languages update on term POST/DELETE), as current implementation takes 5-7 seconds
24. Please use the following German translations of GUI texts (see screenshots for orientation and below for copy & paste):
    1. Deutsch
       (and for English: Englisch)
    2. Filter setzen
    3. Filter löschen
    4. Kunde:
    5. Gefunden:
    6. Benennungen, die zum Eintrag des ausgewählten Suchtreffers gehören
    7. TermCollection:
    8. Neue Benennung vorschlagen
    9. Benennung
    10. Attribute auf Eintragsebene
    11. TBX-Eintrags-ID:
    12. Stapelbearbeitung
    13. Anzuzeigende Sprachen
    14. Attribute auf Benennungsebene
    15. TBX-Benennungs-ID:
    16. Geographische Verwendung
    17. Kunde
    18. TermCollections
    19. Freigabestatus
    20. Unbearbeitet
    21. Vorläufig bearbeitet
    22. Finalisiert
    23. Abgelehnt
    24. Filter
    25. Keine Benennung für diese Sprache vorhanden ("en" findet NICHT "en-US")
    26. Attribut auswählen
25. Sort languages in drop-downs according to language names, not short-cuts - see Term-Portal(11).png
26. Search for
    äm
    in the Bosch Terminology does not find a result, yet a search for
    ä
    shows there should be result (please note the German Umlaut ä in the search)
    Similar for a search for
    *ß
    it should show a result, as the first search result of the search
    **ä
    shows. Yet there is none.
27. Add support for attributes labels to be stored in the database in json format {en: 'English attr title', de: 'German attr title'} in two columns, one for default attributes labels, second for client-defined attribute labels
28. Please introduce a loading screen for the TermPortal as such (initial loading). The loading takes to long to not do that.  - now loading takes much less.
29. Adding an attribute should be disabled, if no terms are loaded (initially after opening the TermPortal)
30. The listing of existing languages in a TermCollection in the language resource overview is not updated, when a term for a new language, that so far did not exist in the TermCollection is added to it. - that was previously mentioned in point 5.12 and fixed
31.  tlauria  Please check, if TermTagging and the TermPortlet in the editor works with the new termCollections. Marc did check it - it works.
32. For users with the roles TermSearch a search for * should be automatically triggered, when opening the TermPortal. For other users not.
33. ensure indexes are updated when importing TBX or editing terms in the GUI. Please ensure together with tlauria , that OPTIMIZE TABLE `terms_transacgrp` is only called, if the indexes are not updated any way in the import (choose the most performant way).
34. Set default processStatus from Zf_configuraiton for terms without a processstatus on tbx import.
    1. Create migration sql to setup a `value` = 'finalized' WHERE `name`="runtimeOptions.tbx.defaultTermAttributeStatus" AND `value` = ''
35. A TermPortal URL with a Hash should still work, even if the user is not logged in any more. He should be redirected to the login page, log in and afterwards be redirected to Termportal again. PLease coordinate that with Aleks (or he does it, because he has a connected issue for InstantTranslate)
36. On term/attribute update - terms_transacgrp-records should be created if not exist
37. If I add a new, empty TermCollection and do not import any TBX into it, it is in status "error" after creation. But without any info about the error in the log of the resource.
38. If I: Search for a word, that do not exist and then click on "Propose a new entry using search word" and then create a new proposal. Then the proposal is created, but it is not shown in the search results and in the center and right panels, because the search is not newly fired.
39. Please integrate theRootCause in the TermPortal in the same way it is in the rest of translate5
40. Make that search to be auto-submitted on search-term-language change.
41. Make that filter window to be closed on Cancel button (in that window) clicked
42. aleksandar  check if fulltext index is updated, when updateing or inserting rows
43. When I log in marc.translate5.net as manager and then open the TermPortal, I always first get InstantTranslate. The behavior here should be the following:
    1. If I open TermPortal from the main translate5 by clicking on the termPortal button, I should get TermPortal
    2. If I open InstantTranslate form the main translate5 by clicking on the InstantTranslate button, I should get InstantTranslate
    3. If I log in and do only have TermPortal rights, I get only TermPortal (without InstantTranslate available)
    4. If I log in and do only have InstantTranslate rights, I get only InstantTranslate (without TermPortal available)
    5. If I log in and do only have InstantTranslate AND TermPortal rights, I get the application I did access last, when I had opened translate5. If nothing like this is saved somewhere, the default is InstantTranslate
44. Getting the following error in InstantTranslate window. Probably reason is, that the click on the InstantTranslate-button opens the wrong (old) InstantTranslate-URL). To reproduce:
    1. Open InstantTranslate from the main translate5 by clicking on the InstantTranslate-button there
    2. Switch from there to TermPortal
    3. Search for a word
    4. Select a term in termPortal for InstantTranslation into a language, that does not exist so far
    5. wait for translation and then click on the plus-button of the translation to add it to the TermPortal
    6. now you get this error in the konsole:
       Uncaught TypeError: top.window.Ext is undefined
           <anonymous> https://marc.translate5.net/modules/editor/instanttranslate/js/Instanttranslate.js?1631807086:1000
           jQuery 8
           <anonymous> [https://marc.translate5.net/modules/editor/instanttranslate/js/Instanttranslate.js?1631807086:854]
45. could you order them by language names (same order, as in the drop down for the languages next to the search field? Same for the languages in the drop-down in TermPortal, where you select the language for InstantTranslate into?
46. This leads to error 400 on the POST request to create a new term:
    1. open TermPortal from main translate5
    2. Search for a word
    3. Select a term in termPortal for InstantTranslation into a language, that does not exist so far
    4. wait for translation and then click on the plus-button of the translation to add it to the TermPortal
    5. click on the save button to save the term
47. The filter in the filter field-set is still titled " Begriff Sammlungen" in German interface. It should be "TermCollections". See screenshot TermPortal(4).png.
48. some search queries can produce '/'-chars after search params base64-encoded, so that hashed-params structure is broken. Example: Base64.encode('["Γειά σου Κόσμε","259","","418","","",""]')  >> 'WyLOk861zrnOrCDPg86/z4UgzprPjM+DzrzOtSIsIjI1OSIsIiIsIjQxOCIsIiIsIiIsIiJd' . query here is 'Hello world' on Greek
49. aleksandar The specific data column should go into the frontend and rendered for the Term collection. There we want to show how many entries the term collection has. This info is show to the user.
       - update this value on each import
50. aleksandar Add menu with submenu in the export Tc.
      - export proposals (current default on button click)
       - export term collection (Pavel stuff){
51. flush tbx contents directly into browser as a downloading file
52. Show dialog on 'Export tbx' menu item click (menu item created by Aleks), on press OK - done except locales
53. Refactoring: move sql's to data-models methods
54. aleksandar add support for descripGrp and back-data in the import
55. add support for descripGrp and back-data in the export
56. aleksandar It would be very useful if you could upload TBX files zipped as well. It should be possible to import all the tbx files then present in the zip file. Both on the top level, and in subdirectories of the top level (arbitrary deep nesting of subdirectories does NOT have to be supported).
    And please add an info-icon with a tooltip behind the label of the TBX-upload form field in the language resources area. The tooltip should read:
    EN: Allowed file formats: TBX or a zip file containing one or multiple TBX files.
    DE: Erlaubte Dateiformate: TBX oder eine Zip-Datei, die eine oder mehrere TBX-Dateien enthält.
57. add an additional parameter to Zf_configuration, that defines the minimum number of characters that need to be added for a live-search to be started? I think for big termCollections it is good, if this can be set to a higher value than the default 3. Should be overwriteable on client-level. If the user has access to the TermCollections of multiple clients, the largest configured value of all clients is used.
    guiGroup: TermPortal
    guiName: When to start live search
    description: Number of typed characters to start live search in the search field
58. Add the full language name to the "Languages to show" drop-down and the flag to the main languages drop down so that both look identical in the look&feel.
59. show in the second column the terms of language that is current search language as top most entries. And afterwards everything else in the current order
60. Change processStatus attribute to isTbxBasic=1, because it is not TBX Basic standard, yet from our point of view no TBX export makes sense without processStatus
61. On ajax requests that return 401 or 403 status code: Redirect to the login page
62. Update total count of the terms,entries and attributes should be updated after new one comes from the term portal
63. tlauria Please decide, who should care for this here: I have strange search behavior on marc.translate5.net regarding without any collection-filter:  Those to queries are identical except the last parameter.
    The first one takes 8,3 sek, the second one 0,37 seconds.
    So far so good, might be mysql caching you think. Yet: The second one was fired nearly at the same time as the first one (but a moment later) - and returned ~7 seconds EARLIER.
    How can that be? What can be optimized with the first one? What is wrong here? And: Who of you should look after this best?
    Here are the queries:
    https://marc.translate5.net/editor/plugins_termportal_data/search?_dc=1632240004905&clientIds=&collectionIds=&processStatus=&noTermDefinedFor=&query=Testbild*&language=361&page=1&start=0&limit=10&autocomplete=Testbild*
    [https://marc.translate5.net/editor/plugins_termportal_data/search?_dc=1632240004985&clientIds=&collectionIds=&processStatus=&noTermDefinedFor=&query=Testbild*&language=361&page=1&start=0&limit=250]
64. Merge the current develop into the TermPortal branch
65. aleksandar The term "Revisionsdienstleistung" does not lead to a translation from the TermCollection in InstantTranslate (from de to fr or en), despite TermPortal shows the translations and the term itself as present
66. aleksandar a termCollection is assigned, yet in the concordance search you never get a match from the termCollection, regardless what you search there (search for "i" for example).
67. aleksandar Please implement the contents of image translate5(34).png (info icon instead of warn icon in certain circumstandes in the language resource management)
68. Please name the downloaded TBX "termCollection name".tbx - so choose the name of the termCollection for it.
69. Please add the martif-header as described here: https://jira.translate5.net/browse/TRANSLATE-1274#Martifheader
70. Rename label of attributes in EN:
    1. processStatus: From processStatus to "Process status"
    2. normativeAuthorization: From normativeAuthorization to "Normative Authorization"
71. With mouse over the active filter, the current active filter value should be displayed as tooltip
72. If i create new term in new entry, and the term language does not exist in the term collection, after save js error happens
73. Please automatically save a comment or a text field attribute on leaving the field. Please change the color of the save icon of the comment field to green, when it has been saved. Same for all other text attbibute fields and for the term and comment fields on term creation
74. The updated date of the processStatus attribute shows 00000, if the attribute has not updated yet. It should show the same date as the creation date.
75. If i propose new term, and i type in the language selection combo to search for proposal language  then save the term and after this propose another term, the language combo contains the languages of the previous search (the combo filter is not resset after term save)
76. For this bug, please check the video in #termportal channel in slack. For this we have separate Jira issue TRANSLATE-2639
77. the term status updating in GUI if a term status relevant attribute was updated
78. also the term status update on creation of new term attributes (But as far as I understand there should be no need to do there something, since the creation of the attribute does not set a value, this is done in a second step in the GUI, so the update is triggered which then updates the state too)
79. on videos.translate5.net search in EN for "test". You will see, that the icon for "preferred" is set for this term. Yet there is no normativeAuthorization or administrativeStatus attribute set. Why? I newly created the term. So the default should be "admitted".
80. In InstantTranslate definitions are not shown for results of termCollections. They should be shown in the following way:
    1. If a language-level definition exists for the term in the selected source language, it should be shown. Same for a language-level definition in the target language. If both exist, both should be shown.
    2. If no language-level definition for the source-language exists, but an entry-level definition exists, this should be shown instead of the source language-level definition. Still a target-language definition should be shown, if it exists.
81. The termPM_allClients role should only be applied when loading the clients for the term-portal. Remove the check for this role from getUserCustomersFromSession() function, this is dangerous since now everywhere in translate5 (where this function is used to get the user customers) users with termPM_allClients role will have all customers. 
82. delay the init of theRootCause after the loading of the ajax call of the hash
83. prevent deletion of the attribute of administrativeStatus (same as implemented for processstatus)
84. for the same administrativeStatus attribute the picklist values should not be decamelcased, they should be display plain
85. disable termportal menu point if term portal plugin is disabled
86. implement migration script, that adds administrativeStatus default value to all terms, that have the attribute not set so far.
87. tlauria Please make sure, that in the import administrativeStatus is set for all terms, that do not have it in the TBX. Please make also sure, that administrativeStatus is set for all terms always, if a new term is created. Please coordinate that as discussed with the default values for administrativeStatus as set in the system config. Multiple state relevant attributes should be coupled together.
88. crossreference grid is "hanging" in certain situations. To reproduce: Login as manager1 here: https://demo.translate5.net/editor/termportal#termportal/search/WyIqIiwiNSIsIiIsIjciLCIiLCIiLCIiXQ==/MjUwMTc= and switch then between the first 3 terms in the GUI. This is caused by non-existent values of createdBy and updatedBy:
    1. columns affected: 
       1. terms_attributes: createdBy and updatedBy; 
       2. terms_attributes_history: updatedBy;  
       3. terms_term: updatedBy
       4. terms_term_history: updatedBy
    2. create db migration
       1. to set NULLs for above columns having non-existent values
       2. to set foreign keys referencing to Zf_users.id with ON DELETE SET NULL
    3. adopt client app to display 'Unknown user' for NULL-values of (created|updated)By
89. crossreferences show error "not a valid tbx id", despite the id is valid. To reproduce login as manager (not manager1) here https://demo.translate5.net/editor/termportal#termportal/search/WyIqIiwiNSIsIiIsIiIsIiIsIiIsIiJd/MzUzNDc= , copy the tbx-id and enter it at another search result of the left panel als crossreference.**
90. Please change the German GUI name of "Gebrauch" (administrativeStatus) to "Verwendungsstatus".
91. Please order the attributes in the list that adds attributes in the GUI alphabethically according to the current GUI language - and also in the GUI. If a new attribute is added, it is ok, if it is the topmost one, until the term-attributes-panel is reloaded.
92. In the last release on demo.translate5.net it is not possible to add the attribute "Part of speech". This is creating an error 400 on the serverside.
93. on videos.translate5.net I have the user testlektor, to him only the roles InstantTranslate and TermSerach are assigned. When I log in I get the JS error https://app.therootcause.io/#marc-mittag/translate5/errors/0162c32b56bd7fa0778d7bede74fdff9faa8b452
94. see rootcause: A JS error is happening, when calling this URL: [https://videos.translate5.net/editor/termportal#termportal/search/WyIqIiwiMzU4IiwiIiwiIiwiIiwiIiwiIl0=]
95. Please rename in the German GUI: "Schöpfer" => "Ersteller"; "Modifizierte" => "Geändert durch"
96. Please use the stop sign as icon for forbidden terms instead of the red striked-through S. Same or similiar icon like we use it in the editor GUI or the old TermPortal
97. theRootCause.io is slowing down some js so much, that we need to disable it until we solved that in a next issue.
98. When switching to InstantTranslate by translating a term and then switching back, some old TermPoral hash is called. Yet it should simply go back to where it has beeen before.
99. please solve (user had been lestlektor pw asdfasdf): https://app.therootcause.io/#marc-mittag/translate5/errors/bf02849b312b197cc2169a7573974f4d247d469a
100. When opening this URL with the user testlektor2 pw: asdfasdf in TermPortal the center and right panels stay with loading mask: https://videos.translate5.net/editor/termportal#termportal/search/WyIqIiwiNCIsIiIsIiIsIiIsIiIsIiJd/MTY1Mjc2
101. In the customers table: openIdServerRoles,openIdDefaultServerRoles are not migrated to the new role names (termSearch)