@m-anish
Can we get Terry into this thread to discuss the way the logs are compiled now. I think the easiest would be that he shares the scripts that do it, I modify it, and he implements it.
The issue currently is that the logs are parsed in a totally non-standard way, which results in risk of errors when they are parsed to meaningful format. Ideally for any data, but particularly for something as important as QOS data, it's best practice to handle formats in the source.
For example, now we have entries where one row has a timestamp, and then the following rows are entries related with that timestamp. Whereas the standard way is to have the timestamp together with each event. More generally, all the logs should follow a standard "line == event" approach where we always know that if it's a row, it's a self-contained event.
@m-anish
Can we get Terry into this thread to discuss the way the logs are compiled now. I think the easiest would be that he shares the scripts that do it, I modify it, and he implements it.
The issue currently is that the logs are parsed in a totally non-standard way, which results in risk of errors when they are parsed to meaningful format. Ideally for any data, but particularly for something as important as QOS data, it's best practice to handle formats in the source.
For example, now we have entries where one row has a timestamp, and then the following rows are entries related with that timestamp. Whereas the standard way is to have the timestamp together with each event. More generally, all the logs should follow a standard "line == event" approach where we always know that if it's a row, it's a self-contained event.