https://linen.dev logo
Join Slack
Powered by
# fluent-bit
  • s

    Sven

    07/01/2025, 11:01 AM
    Hi everyone, Fluent Bit regularly crashes in our EKS cluster with exit code 139. I've already increased the memory limit to 500MiB, but that didn't help. I suspect it's not a memory issue, as there are no unusual memory usage patterns visible in the pod monitoring. Could you please give me some tips on how I might be able to debug this? Best regards, Sven P.S.: We're using the latest version (4.0.3)
    p
    • 2
    • 1
  • m

    Max

    07/01/2025, 4:42 PM
    Hey everyone I am wondering if anyone has experience using fluentbit with SAP Audit logs. I am trying to use fluentbit tail input on SAP Audit logs on a linux operating system but i cannot get it to work. Fluentbit does not seem to see any file appending for them or at least seems to be unable to read even with a REGEX parser. Has anyone encountered this use case?
    p
    • 2
    • 16
  • e

    Eric D. Schabell

    07/01/2025, 5:36 PM
    Two PR’s for YAML example updates for input plugins ready for review: • https://github.com/fluent/fluent-bit-docs/pull/1818 • https://github.com/fluent/fluent-bit-docs/pull/1820
    🙌 1
  • h

    henrik Rexed

    07/02/2025, 9:13 AM
    so i have run several tests...and i can confirm it seems that there is a memory leak in the Sampling processor
    p
    • 2
    • 2
  • e

    Eric D. Schabell

    07/02/2025, 5:01 PM
    Three PR’s for YAML examples for input plugins docs ready for review: • https://github.com/fluent/fluent-bit-docs/pull/1832 • https://github.com/fluent/fluent-bit-docs/pull/1833 • https://github.com/fluent/fluent-bit-docs/pull/1834
    🙌 1
  • a

    Anusha Hegde

    07/03/2025, 5:07 AM
    Hello! I was wondering why does latest fluentbit still not support CentOS 10 (released back in December 2024), and RHEL 10.0 (released on 2025-05-20).
    p
    • 2
    • 5
  • n

    Nizar

    07/03/2025, 9:08 AM
    Hello everybody , i'm discovering fluentbit and playing with it a little bit , but i have a issue : So i collect my logs and send them to loki with the officiel loki fluentbit configuration (docs i used : https://docs.fluentbit.io/manual/pipeline/outputs/loki ) So everything works fine my logs are collected and sent to loki but i have a issue is when i add a label , this label is not visible in my logs ( i use grafana ) i did follow the documentation steps and did this configuration :
    outputs: |
    [OUTPUT]
    Name        loki
    Match       kube.*
    host        lokiurl
    port        3100
    uri         /loki/api/v1/push
    labels collector="fluentbit"
    auto_kubernetes_labels  true
    So logically in my logs i should have a new label called collector with the value fluentbit right ?
    p
    • 2
    • 10
  • e

    Eric D. Schabell

    07/03/2025, 9:15 AM
    Three new PR’s for YAML examples for input plugin docs ready for review: • https://github.com/fluent/fluent-bit-docs/pull/1848 • https://github.com/fluent/fluent-bit-docs/pull/1849 • https://github.com/fluent/fluent-bit-docs/pull/1850
    🙌 1
  • e

    Eric D. Schabell

    07/03/2025, 9:32 AM
    New PR for input plugin fix, missed it on the first pass with OTel plugin: https://github.com/fluent/fluent-bit-docs/pull/1851
    🙌 1
  • a

    Anshuman Goyal

    07/03/2025, 10:10 AM
    The password for loki is stored in a file for us, and we don't want to provide this as a string in helm chart. It there any option/way to provide the password from file in FB?
    Copy code
    basic_auth:
                username: loki
                password_file: /etc/secrets/password-file
    p
    • 2
    • 40
  • e

    Eric D. Schabell

    07/03/2025, 10:10 AM
    Four new PR’s for input plugin docs ready for review: • https://github.com/fluent/fluent-bit-docs/pull/1855 • https://github.com/fluent/fluent-bit-docs/pull/1856 • https://github.com/fluent/fluent-bit-docs/pull/1857 • https://github.com/fluent/fluent-bit-docs/pull/1858
    🙌 1
  • e

    Eric D. Schabell

    07/03/2025, 1:16 PM
    The last PR ready for review, when merged, all input plugin docs are YAML ready: https://github.com/fluent/fluent-bit-docs/pull/1860
    🙌 1
  • s

    Supriya Mishra

    07/03/2025, 5:32 PM
    Hi Everyone, we are trying to use fluent bit http input plugin to ingest the logs to our cloud watch but from the documentation I see it only supports http and posting a application log which might contain sensitive data..could someone kindly suggest if it is good idea to use this approach for application logs .. . Thank you so much for your help..
    p
    • 2
    • 3
  • a

    Ashish

    07/04/2025, 7:59 AM
    Hi everyone.. This documentation for S3 output: https://docs.fluentbit.io/manual/pipeline/outputs/s3 has a retry_limit configurable option mentioned to specify the number of allowed retries. But if i see the output code in latest master: https://github.com/fluent/fluent-bit/blob/master/plugins/out_s3/s3.c , this is not mentioned / used anywhere, the max retries are hardcoded to 5 https://github.com/fluent/fluent-bit/blob/1ae8ed9e6b86b0a44c3c60dcf27b89c724825409/plugins/out_s3/s3.h#L59 Is the documentation wrong? This parameter should ideally be configurable, there are many cases where we would need more than 5 retries
    • 1
    • 2
  • a

    Anton

    07/04/2025, 12:26 PM
    @Anton has left the channel
  • s

    Saral

    07/04/2025, 12:26 PM
    Hi team, I'm running Fluent Bit via the Fluent Operator and using a Lua filter to drop logs based on log level. Here's a snippet of my filter config:
    Copy code
    - lua:
        call: normalize_product_logs
        script:
          name: fluent-bit-modify-k8s-fields
          key: custom.lua
    The
    custom.lua
    script uses environment variables like
    drop_error_product
    ,
    drop_debug_product
    , etc., to control which logs should be dropped dynamically. For example:
    Copy code
    if log_level == "ERROR" and drop_error_product == "true" then
      return -1  -- drop log
    end
    These variables are passed via environment variables in the Fluent Bit container or set via config maps. Problem When I update the values of these variables (e.g., setting
    drop_error_product
    from
    false
    to
    true
    ) and confirm that the
    ConfigMap
    is updated and pods are restarted manually, the updated logic does not take effect. However, the moment I: • remove the entire Lua block from the filter config and deploy • then add the same Lua block back and deploy again ...the changes start reflecting and logs get dropped correctly. Is Fluent Bit or the Lua plugin caching the script or its environment somewhere internally? Is there a supported or recommended way to force Fluent Bit to reload updated Lua script logic and/or environment config? Thanks a lot — any insights or best practices would be appreciated!
    p
    • 2
    • 14
  • e

    Eric D. Schabell

    07/04/2025, 2:02 PM
    A holiday YAML example docs update for Parsers section ready for review: https://github.com/fluent/fluent-bit-docs/pull/1866
    🙌 1
  • e

    Eric D. Schabell

    07/04/2025, 3:47 PM
    Another YAML docs update for Processor sections ready for review: https://github.com/fluent/fluent-bit-docs/pull/1869
    🙌 1
  • e

    Eric D. Schabell

    07/07/2025, 1:05 PM
    Small doc YAML display fix PR ready for review: https://github.com/fluent/fluent-bit-docs/pull/1871
    🙌 1
  • e

    Eric D. Schabell

    07/07/2025, 4:07 PM
    Half of the Filter docs updated with YAML examples, PR ready for review: https://github.com/fluent/fluent-bit-docs/pull/1873
  • e

    Eric D. Schabell

    07/07/2025, 4:13 PM
    As a result of the filter docs YAML examples updates, discovered an error in multiline parsing docs YAML, fixed and PR ready for review: https://github.com/fluent/fluent-bit-docs/pull/1874
  • w

    William

    07/07/2025, 5:51 PM
    Hey, open question for those that have used the
    rewrite_tag
    filter as an input processor in their YAML config. I currently use the
    tail
    plugin to read Kubernetes container logs files. On this input, I define 3
    rewrite_tag
    "filters as processors" in order to categorize the records based on some internal grouping logic. The idea is to ensure that logical batch of records are grouped together in the final chunks, and consumed by different outputs. Each filter re-emit a record if a condition matches (based on the presence of a kubernetes pod annotation enriched by the
    kubernetes
    filter). The thing is, when I check the metrics, especially the v2 metric
    fluentbit_filter_records_total
    , I noticed that the nth emitter defined in the config always shows N+X records, where N is the numbers of records processed by the
    kubernetes
    filter itself, before in the processors list, and X is the amount of records that matched the condition of the previous
    rewrite_tag
    filter. To give an example, here is the config of the 3 processors:
    Copy code
    - name: rewrite_tag
                  match: kube.*
                  alias: rewrite_syslog
                  rule: "$kubernetes_annotation_syslog/enabled ^(true)$ syslog true"
                  emitter_name: re_emit_syslog
                  emitter_storage.type: filesystem
                - name: rewrite_tag
                  match: kube.*
                  alias: rewrite_kafka
                  rule: "$kubernetes_annotation_kafka-topic ^([a-zA-Z0-9]+[-._]?)+[a-zA-Z0-9]+$ kafka true"
                  emitter_name: re_emit_kafka
                  emitter_storage.type: filesystem
                  emitter_mem_buf_limit: 100M
                - name: rewrite_tag
                  match: kube.*
                  alias: rewrite_graylog
                  rule: "$kubernetes_annotation_logs/sink-addr ^(host1|host2)$ graylog.$0 true"
                  emitter_name: re_emit_graylog
                  emitter_storage.type: filesystem
    The v2 metrics are as follows:
    Copy code
    2025-07-07T17:30:55.121280325Z fluentbit_filter_records_total{name="kubernetes"} = 242
    2025-07-07T17:30:55.121280325Z fluentbit_filter_records_total{name="rewrite_syslog"} = 242
    2025-07-07T17:30:55.121280325Z fluentbit_filter_records_total{name="rewrite_kafka"} = 414
    2025-07-07T17:30:55.121280325Z fluentbit_filter_records_total{name="rewrite_graylog"} = 426
    If I match this with the metrics of the processed records per output (that match the new tags emitted by the
    rewrite_tag
    processors):
    Copy code
    2025-07-07T17:30:50.282031744Z fluentbit_output_proc_records_total{name="kafka"} = 12
    2025-07-07T17:30:54.274884272Z fluentbit_output_proc_records_total{name="graylog"} = 300
    2025-07-07T17:30:54.275732061Z fluentbit_output_proc_records_total{name="syslog"} = 172
    We can see that: • kafka - syslog = 172 (syslog output processed records count) • graylog - kafka = 12 (kafka output processed records count) Per my understanding, the new records emitted by the
    rewrite_tag
    filter are re-evaluated from the beginning of the pipeline (and this shows that this is the case since the filters are clearly re-ingesting the new records emitted by previous filters), but I don't understand why that applies to the filters as processors of the
    tail
    input plugin. I have tried to add a
    match: kube.*
    field to all
    rewrite_tag
    processors, but this has no effect, other that suppressing a warning at startup about a potential infinite loop (which never happens anyway). I know that I could move the filters in the main thread, but I actually do not want to go that route for performance reasons, and because I use threaded inputs.
    c
    • 2
    • 2
  • e

    Eric D. Schabell

    07/08/2025, 9:46 AM
    New PR for review fixing and standardizing on YAML for filter docs: https://github.com/fluent/fluent-bit-docs/pull/1881
    🙌 1
  • e

    Eric D. Schabell

    07/08/2025, 11:49 AM
    YAML example update for filter docs PR ready for review: https://github.com/fluent/fluent-bit-docs/pull/1883
    🙌 1
  • f

    F

    07/08/2025, 12:49 PM
    Hello all, I would like to use the gcp pubsub plugin but it isn't enabled by default. Any suggestions on how to enable it? Thanks
    p
    b
    • 3
    • 5
  • p

    Pawel Gieniec

    07/08/2025, 8:17 PM
    Hello, I just wanted to bump these issues again. I know last time we spoke it was said that there wasn’t enough information to easily repro. https://github.com/fluent/fluent-bit/issues/10518 https://github.com/fluent/fluent-bit/issues/9927 https://github.com/fluent/fluent-bit/issues/9917 https://github.com/fluent/fluent-bit/issues/9906 https://github.com/fluent/fluent-bit/issues/9903 (gh actions closed this as stale, but it’s not according to the original submitter) 9927 has a lot more info since I last posted. I just want to get an 👁️ on this, and if it’s still sparse on details I can help drive the submitters/commentors for more information just provide me with an explanation as to what’s missing or needed. Thank you!
    h
    • 2
    • 4
  • p

    Priyanka

    07/09/2025, 9:47 AM
    Hi @all I am trying to build fluent-bit_1.9.7 in yocto. While it builds successfully with default flags, if I enable LUA filter the build is not successful. Somewhere in the web I read that bundled lua is not compatible with any GCC version. I find it hard to believe. Can someone please confirm this ?
    p
    m
    • 3
    • 6
  • e

    Eric D. Schabell

    07/09/2025, 6:05 PM
    Simple maintainer update PR ready for review: https://github.com/fluent/fluent-bit/pull/10569
  • d

    DennyF

    07/10/2025, 9:19 AM
    Hello, we use at the moment NXlog, which forwards messages to Graylog and old style (R)Syslogd. I'm trying out fluentd and fluent-bit in parallel. We have in our Nxlog something like this:
    Copy code
    $message =~ s/cardNo>\d\d\d\d\d\d\d\d\d\d\d\d/cardNo>XXXXXXXXXXXX/g;
    	 $message =~ s/cvv>\d\d\d/cvv>XXX/g;
    	 $message =~ s/cvv=\d\d\d/cvv=XXX/g;
    	 $message =~ s/Verification%3E\d\d\d/Verification%3EXXX/g;
    	 $message =~ s/R19%3E\d\d\d/R19%3EXXX/g;
    	 $message =~ s/R16%3E\d\d\d\d\d\d\d\d\d\d\d\d/R16%3EXXXXXXXXXXXX/g;
    	 $message =~ s/ccn=\d\d\d\d\d\d\d\d\d\d\d\d/ccn=XXXXXXXXXXXX/g;
    	 $message =~ s/card_account=\d\d\d\d\d\d\d\d\d\d\d\d/card_account=XXXXXXXXXXXX/g;
    	 $message =~ s/cvc_code=\d\d\d/cvc_code=XXX/g;
    	 $message =~ s/password=/Password=XXX\&/g;
    to replace sensitive creditcard data. How we can do the same in fluent-bit ?
    p
    e
    • 3
    • 16
  • e

    Eric D. Schabell

    07/10/2025, 4:58 PM
    Amazon output plugins PR for yaml examples ready for review: https://github.com/fluent/fluent-bit-docs/pull/1891
    🙌 1