Skip to content

Output date is wrong when I use more than one date filter #136

@jasonperrone

Description

@jasonperrone
  • Version: 3.1.9
  • Operating System: Ubuntu 18.04LTS
  • Config File (if you have sensitive info, please remove it):
input {
  beats {
    port => "5044"
  }
}
filter {
  csv {
    separator => "|"
    columns => [
      "event_time",
      "category",
      "message",
      "app",
      "level",
      "thread",
      "ip",
      "email",
      "cjisId",
      "requestId",
      "session",
      "beforeThread",
      "afterThread"
    ]
  }
  date {
    match => [ "event_time", "yyyy-MM-dd HH:mm:ss" ]
    target => "@timestamp"
  }
  date {
    match => [ "event_time", "yyyy-MM-dd HH:mm:ss" ]
    target => "timestamp"
  }
}
output {
  elasticsearch {
    hosts => [ "localhost:9200" ]
    index => "logs-%{+YYYY.MM.dd}"
  }
}
  • Sample Data:
2020-04-03 09:35:11| e.DirtyHibernateSessionManager | Jason is in Baltimore | development|DEBUG|qtp477594197-26|127.0.0.1|JASON@JASON.COM|523|XUxarCZX65lr|1dzdxt1cd9b1xhvqhw7f9vgz6|04/03/2020 09:35:09|04/03/2020 09:35:10|26|
  • Steps to Reproduce:
    As you can see I have two date filters in there. The purpose of the one to @timestamp is so that the index name will be formed correctly. The one to timestamp is so that in Elastic, the entry has a timestamp field which an application using this index needs.

If I have only the first filter in place, my @timestamp is produced like so:

"@timestamp" : "2020-04-03T16:35:10.000Z",

That's straight from Elastic using a cUrl command to search my index.

With both filters in place, my @timestamp ends up looking like:

"@timestamp" : "0020-04-03T16:35:10.000Z",

The other timestamp field also:

"timestamp" : "0020-04-03T16:35:10.000Z",

Notice how the year is now wrong (0020).

What in the?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions