I dont see any information about plan to fix syslo...
# fluent-bit
d
I dont see any information about plan to fix syslog TCP input filter, to be RFC-compliant? At a moment, we have only non RFC implementation (basically UDP way translated to TCP fabric)
p
Is there an issue raised for it?
a
I'm not sure what this means, could you provide more information?
p
I saw something about using other delimiters or octets I think but yeah I'm not sure which RFC to look at
@Dubravko Penezić please confirm
d
Sorry for the late answer, but yes, it is connected to RFC 6587, it isn't a question of the delimiter, because TCP and UDP are two different protocols. In UDP, you send a message in one package up to 8 KB, and you may rely on \n or \0 delimiter to be the end of the message and forget. However, in TCP, you don't know if the message is sent in packages; you receive, you may get a partial message, two messages or more, and any combination in between, so relying on the delimiter isn't an option, and yes, RFC 6587 covers exactly how to receive SYSLOG messages over TCP.
Finally solution we have at the moment cover only the situation where in one TCP packet is exactly one SYSLOG message, which needs to be ended with \n or \0
Why is it important to implement proper flow ?, I work in an environment where millions of syslog messages arrive every day, and some recommendations like NIS 2 (EU) and some unstable Internet connectivity ask for SYSLOG over TCP (TLS if possible). Also, I am not a fan of too many proxies in the communication channel (Yes it is possible to put some syslog services software before fluent-bit, but that isn't best way to do)
p
I mean there's no real requirement in UDP to only send a single syslog message in a single packet or a restriction on the packet size either - I've also worked on systems where they've split messages over multiple UDP packets too! 🙂 However I understand what you mean. It seems a good idea to do, can we expand the issue with any more detail to help focus it?
d
Yes, I can do that, I can even suggest some code, but I am not a native C programmer, and especially not involved in fluent-bit logic. What I can also do is test because we have that possibility in small and large datasets
p
I mean, testing is to be honest more important for OSS so that's great - even if you can suggest some ways to test in unit testing/etc. to help
even a PR that just adds the tests (and fails) is a good step for someone to then implement on top
d
O.K. ... I think I can do that. Do you prefer to extend existing issue https://github.com/fluent/fluent-bit/issues/10910 or to create new one ?
p
I would extend/update the existing one - it shows more people care
d
O.K.