https://linen.dev logo
Join Slack
Powered by
Hey! I'm parsing logs that only contain a time (H...
# fluent-bit
s
Hey! I'm parsing logs that only contain a time (HHMMSS) with no date component, using a regex parser with Time_Format %H:%M:%S. My system timezone is CET/CEST (Europe/Stockholm). This works fine except around midnight. At 00:00 local time, the UTC date is still the previous day. Fluent Bit appears to assign the UTC date to the parsed time, so logs between 00:00 and 02:00 local time get timestamps on the wrong date (previous day). I've tried Time_System_Timezone on with TZ=Europe/Stockholm set on the systemd service, and Time_Offset, but neither resolved the issue. Is there a supported way to make the parser assign the correct local date when parsing time-only formats? Im using Fluent Bit v4.2.3 with classic config format.
p
Can you provide your full config?
Just to confirm, are you saying the time is parsed correctly? It sounds like maybe it is not and picking up current time
s
Of course!
Copy code
[SERVICE]
    Daemon Off
    Flush 1
    Log_Level info
    Parsers_File parsers.conf

[INPUT]
    Name              tail
    Path              /home/server/survival/plot/logs/latest.log
    multiline.parser  paper_multiline
    Tag               survival/plot
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/survival/resource/logs/latest.log
    multiline.parser  paper_multiline
    Tag               survival/resource
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/survival/spawn/logs/latest.log
    multiline.parser  paper_multiline
    Tag               survival/spawn
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/uhc/logs/latest.log
    multiline.parser  paper_multiline
    Tag               uhc
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/event/logs/latest.log
    multiline.parser  paper_multiline
    Tag               event
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/parkour/logs/latest.log
    multiline.parser  paper_multiline
    Tag               parkour
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/crew/logs/latest.log
    multiline.parser  paper_multiline
    Tag               crew
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/comp/logs/latest.log
    multiline.parser  paper_multiline
    Tag               comp
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/creative/logs/latest.log
    multiline.parser  paper_multiline
    Tag               creative
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/hub/logs/latest.log
    multiline.parser  paper_multiline
    Tag               hub
    Refresh_Interval  1
    skip_empty_lines  On

[INPUT]
    Name              tail
    Path              /home/server/velocity/logs/latest.log
    multiline.parser  velocity_multiline
    Tag               velocity
    Refresh_Interval  1
    skip_empty_lines  On

[FILTER]
    Name     parser
    Match    velocity
    Key_Name log
    Parser   velocity

[FILTER]
    Name     parser
    Match    *
    Key_Name log
    Parser   paper

[FILTER]
    Name   lua
    Match  *
    script /etc/fluent-bit/add_tag.lua
    call   add_tag

[OUTPUT]
    Name               es
    Match              *
    Host               localhost
    Port               9200
    HTTP_User          USERNAME
    HTTP_Passwd        PASSWORD
    tls                On
    tls.verify         Off
    Index              server-logs
    Suppress_Type_Name On
    buffer_size 10485760
    Retry_Limit        False
And here's my parsers.conf:
Copy code
[MULTILINE_PARSER]
    name          paper_multiline
    type          regex
    flush_timeout 1000
    rule          "start_state"   "/^\[\d{2}:\d{2}:\d{2}\] \[/"   "cont"
    rule          "cont"          "/^(?!\[\d{2}:\d{2}:\d{2}\] \[)/"   "cont"

[MULTILINE_PARSER]
    name          velocity_multiline
    type          regex
    flush_timeout 1000
    rule          "start_state"   "/^\[\d{2}:\d{2}:\d{2}\] \[/"   "cont"
    rule          "cont"          "/^(?!\[\d{2}:\d{2}:\d{2}\] \[)/"   "cont"

[PARSER]
    Name        paper
    Format      regex
    Regex       ^\[(?<time>\d{2}:\d{2}:\d{2})\] \[(?<thread>[^/]+)/(?<level>\w+)\]: (?<message>[\s\S]*\S)
    Time_Key    time
    Time_Format %H:%M:%S
    Time_System_Timezone on

[PARSER]
    Name        velocity
    Format      regex
    Regex       ^\[(?<time>\d{2}:\d{2}:\d{2})\] \[(?<thread>[^/]+)/(?<level>\w+)\] \[(?<logger>[^\]]+)\]: (?<message>[\s\S]*\S)
    Time_Key    time
    Time_Format %H:%M:%S
    Time_System_Timezone on
And then I have a Lua filter to add a tag:
Copy code
function add_tag(tag, timestamp, record)
  record["tag"] = tag
  return 2, timestamp, record
end
The time is parsed correctly, yes, but that is because it's taking it from the log. Here's an example log line also: 
[15:57:33] [pool-7-thread-1/INFO] [proxycore]: [MSG] From: Player1 To: Player2 Message: Hello!
p
Can you simplify it down to a simple tail with stdout output to demonstrate it? Some sample input as well that shows the failure and working conditions too.
It should be easy to reproduce I think with a simple container and mounting in logs - that's what I normally do (you can have everything in the current directory and just mount in then): https://www.telemetryforge.io/blog/debugging-tips You can also have an 
expect
filter as well to check time ranges potentially to give an all in one reproducer.
That way we can be explicit in everything required, including environment variables.
s
I've managed to reproduce the issue with a minimal example just now. When parsing logs that only contain HHMMSS with Time_System_Timezone on and TZ=Europe/Stockholm, times around midnight seem to get assigned to the wrong date. The 000000 and 000100 lines have unix timestamps that are earlier than the 235800 and 235900 lines, indicating they've been assigned to the previous day. Input log:
Copy code
[23:58:00] [Server thread/INFO]: Before midnight
[23:59:00] [Server thread/INFO]: Just before
[00:00:00] [Server thread/INFO]: After midnight - WRONG DATE
[00:01:00] [Server thread/INFO]: Still wrong
[02:00:00] [Server thread/INFO]: Back to normal
Test config:
Copy code
[SERVICE]
    Daemon Off
    Flush 1
    Log_Level info
    Parsers_File parsers.conf

[INPUT]
    Name              tail
    Path              ./test-logs.txt
    Tag               test
    Refresh_Interval  1
    Read_from_Head    On

[FILTER]
    Name     parser
    Match    *
    Key_Name log
    Parser   paper

[OUTPUT]
    Name   stdout
    Match  *
Test parser:
Copy code
[PARSER]
    Name        paper
    Format      regex
    Regex       ^\[(?<time>\d{2}:\d{2}:\d{2})\] \[(?<thread>[^/]+)/(?<level>\w+)\]: (?<message>.*)
    Time_Key    time
    Time_Format %H:%M:%S
    Time_System_Timezone on
Ran in docker on v4.2.3:
Copy code
vikto@Mac fluent-bit-tz-test % docker run -it --rm \
  -e TZ=Europe/Stockholm \
  -v $(pwd):/workspace \
  -w /workspace \
  fluent/fluent-bit:4.2.3 \
  fluent-bit -c fluent-bit.conf
With the result:
Copy code
Fluent Bit v4.2.3
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF graduated project under the Fluent organization
* <https://fluentbit.io>

______ _                  _    ______ _ _             ___   _____ 
|  ___| |                | |   | ___ (_) |           /   | / __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| | `' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| |   / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |_./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)_____/
                                                                  
             Fluent Bit v4.2 – Direct Routes Ahead
         Celebrating 10 Years of Open, Fluent Innovation!

[2026/03/30 17:04:16.981008383] [ info] [fluent bit] version=4.2.3, commit=40c2f27605, pid=1
[2026/03/30 17:04:16.981509425] [ info] [storage] ver=1.5.4, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2026/03/30 17:04:16.981532008] [ info] [simd    ] NEON
[2026/03/30 17:04:16.981535758] [ info] [cmetrics] version=1.0.6
[2026/03/30 17:04:16.981551675] [ info] [ctraces ] version=0.6.6
[2026/03/30 17:04:16.981629717] [ info] [input:tail:tail.0] initializing
[2026/03/30 17:04:16.981638133] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2026/03/30 17:04:16.982802842] [ info] [sp] stream processor started
[2026/03/30 17:04:16.982861050] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2026/03/30 17:04:16.986270550] [ info] [input:tail:tail.0] inotify_fs_add(): inode=7 watch_fd=1 name=./test-logs.txt
[2026/03/30 17:04:16.989471300] [ info] [output:stdout:stdout.0] worker #0 started
[0] test: [[1774907880.000000000, {}], {"thread"=>"Server thread", "level"=>"INFO", "message"=>"Before midnight"}]
[1] test: [[1774907940.000000000, {}], {"thread"=>"Server thread", "level"=>"INFO", "message"=>"Just before"}]
[2] test: [[1774821600.000000000, {}], {"thread"=>"Server thread", "level"=>"INFO", "message"=>"After midnight - WRONG DATE"}]
[3] test: [[1774821660.000000000, {}], {"thread"=>"Server thread", "level"=>"INFO", "message"=>"Still wrong"}]
[4] test: [[1774828800.000000000, {}], {"thread"=>"Server thread", "level"=>"INFO", "message"=>"Back to normal"}]
The timestamps for the midnight entries (1774821600, 1774821660) are significantly earlier than the pre-midnight entries, leading me to believe something isn't working as expected with the date assignment.
Is it best if i create a GitHub issue for this, to keep the discussion and resolution in one place?
p
Yeah sorry, meant to say that
Been a busy week!
s
It's alright, thanks for your help! 🙌
Open in Slack
PreviousNext
Powered by