Issue parsing http log (Time_Format %b %d %H:%M:%S...
# fluent-bit
l
Issue parsing http log (Time_Format %b %d %H:%M:%S.%f %Y) Apache Error log writes the time in messages as "May 30 110956.231593 2025", however I am trying to update our config to use
%b %d %H:%M:%S.%f %Y
and there is an issue with it. I looked at github issues but I dont seem to find any bug filed yet for this.
p
What's the issue? Fluent bit just uses standard time parsing
l
hi @Pat, Fluent Bit is rejecting the parsing fir when I modify the
Time_Format
In apache the access_log and error_log have different formats for the Time: • access_log Time looks like:
[10/Oct/2000:13:55:36 -0700]
and its regex is
%d/%b/%Y:%H:%M:%S %z
• error_log Time looks like:
[Thu May 12 08:28:57.652118 2011]
and its regex is
%b %d %H:%M:%S.%f %Y
• The access_log works, but the error_log won't. I suspect it has to do with the microseconds int the error log
.%f
p
Fluent bit just uses a standard system call
b
Pretty sure you're getting that error because
%f
is not a valid option in
strftime(3)
. This is the list I usually refer back to: https://en.cppreference.com/w/c/chrono/strftime
l
Let me try
šŸ˜ž @Braydon Kains @Pat I followed Braydons advice but even that fails. for the strftime I would lyave to use
%6N
but even that fails :S
%b %d %H:%M:%S.%6N %Y
Using the sandbox test it looks ok to me, but I may be missing something else
b
The playground I linked must be wrong for some reason, including a directive that doesn't exist in the actual strftime from libc
Based on https://www.man7.org/linux/man-pages/man3/strftime.3.html I'm not seeing a way you can actually add nanoseconds to the format string
šŸ˜ž 1
@Leoswaldo Macias Mancilla It looks like we use the following format for Apache Error:
%a %b %d %H:%M:%S.%L %Y
Ah I see, Fluent Bit hacks support for fractional seconds in manually. Not sure if this is documented anywhere. https://github.com/fluent/fluent-bit/blob/4be7e3c03bb5b7d46810c9dfab645ea152de1657/src/flb_parser.c#L309
So the
%L
is what you're looking for
(It was also inaccurate for me to link
strftime(3)
resources for this because this is parsing, and the actual call is
strptime(3)
though it is much the same directives https://man7.org/linux/man-pages/man3/strptime.3.html)
p
I recall Leo added nano seconds a while back, ideally send a docs PR on the time parsing page I think
l
Yeah, this is the format that works for Apache Errors
%b %d %H:%M:%S.%L %Y
Thanks @Braydon Kains and @Pat for your help. As follow up, is this something you want to document or I can help you documenting but pretty sure someone will have this issue later
b
I don't really have any time to contribute to docs any time soon, I do highly recommend contributing to docs though if you are able to
l
ok, I'll take a look at it. Thanks again @Braydon Kains
šŸ‘ 1