Uploaded image for project: 'translate5'
  1. translate5
  2. TRANSLATE-3776

Indi Engine: events aggregation

    XMLWordPrintable

Details

    Description

      Problem

      Currently Indi Engine logger shows as separate records all occurences of all events coming from all translate5 instances, and it is currently not possible to see one events behind the others, because it will require to scroll the hundreds occurences of one event to see the other events, each might having hundreds of occurences as well.

      Solution

      Events occurences aggregation should be implemented, with 'day' as aggregation period, so that occurences of a same event happened in the same day on the same translate5 instance should be counted as a single database record, but having special fields for occurences counter, timestamp of first occurence and timestamp for the most recent occurence so far.

      What mean 'same events'? For now, this means events having same EXXXX-code. This is not a final solution, because there are dozen (among more than 600) of e-codes each used for events with a pretty different reasons behind, so the further step might be to introduce a config  named 'Aggregation' within the 'Event code' properties, with values 'Enabled' (default), 'Disabled' and 'Conditional',  which allow to prevent aggregation for a certain error codes when neeeded, or to enable if certain conditions are met - defined, for example, as an SQL WHERE expression. If 'Conditional' is selected, when it should be possible to define a list of conditions, so that each of those will be used to distinct between event occurences having same e-code. This will be useful for cases when events are caused by SQL-errors, which might differ by reason shown in messages produced by MySQL and point of the application where they come from

      Additional way to improve is to introduce 'muting' of events, so that events that met certain conditions are ignored / not counted when events are POSTed or processed normally but excluded from when filtering search results, but this need more details conception to be prepared, which might be the case once the basic aggregation is implemented so it will be obvious which way to go further.

      Also, as long is the database size is growing, at some point it will make sence to apply partitioning-by-month for the events database table, and make Month filter to be preset with current month by default with ability to change the month but not to clear that filter, as the most use cases when checking events are based on that there is no need to work on time range more than certain month, so this may improve the performance

      Attachments

        Activity

          People

            pavelperminov Pavel Perminov
            pavelperminov Pavel Perminov
            Thomas Lauria
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: