FuentBit Tail Input Hi everyone, I’m using a simpl...
# fluent-bit
n
FuentBit Tail Input Hi everyone, I’m using a simple config on Fluent Bit with an INPUT tag
kube.*
and Parser
cri
, and an OUTPUT to stdout. I’m not using any additional filters or anything else. The result is that my logs contain fields like
message
,
time
, … and also a
kubernetes
field (which includes labels, annotations, container info, …). While that’s what I expect, I only need a few fields from it. I’ve already tried add FILTER kubernetes, modify, and record_modifier, but they don’t seem to have any effect (nothing changes in the OUTPUT). inputs: | [INPUT] Name tail Path /var/log/containers/mybunkerweb-default.log Parser cri Tag kube.* outputs: | [OUTPUT] Name stdout Is there a hidden part of the pipeline that automatically adds these fields and cannot be modified? If my INPUT tag is not
kube.*
(e.g.
myappp.*
), then there is no
kubernetes
field in the log. But if the INPUT tag is
kube.*
, the Kubernetes fields appear, and I don’t know how to remove the unnecessary fields from them.
p
The kubernetes filter adds those fields and the tags have to match the expected format which defaults to kube.*
If you change the tag then you have to modify the kubernetes filter config otherwise it you end up sending requests for meta data to the K8S API with invalid pod names
I did a webinar a while back on it plus added some blog posts
You can customise the kubernetes filter as to what it adds, check the docs
What is it you need and what are you currently seeing? A full config would be helpful too
n
Copy code
inputs: |
    [INPUT]
        Name tail
        Path /var/log/containers/myapp-*_default_*.log
        Parser cri
        Tag kube.*
        ## <https://docs.fluentbit.io/manual/pipeline/outputs>
  filter: |
    [FILTER]
        Name                kubernetes
        Match               kube.*
    [FILTER]
        Name nest
        Match *
        Operation lift
        Nested_Under kubernetes
    [FILTER]
        Name record_modifier
        Match *
        Record hostname $kubernetes['host]

  outputs: |
    [OUTPUT]
        Name stdout
        Match *
p
you know you have un-nested the kubernetes keys but then you've said to add a new nested key for
host
?
what is it you want to achieve and what are you currently seeing?
n
I don’t need too much customization, it’s just that there are too many Kubernetes fields, and I want to drop some of them. For example, in the blog’s output, I don’t need fields like
docker_id
,
labels
,
annotation
, etc.
p
I would use the whitelist functionality in record_modifier to only allow the ones you want: https://docs.fluentbit.io/manual/data-pipeline/filters/record-modifier
what actual fields do you need though?
n
I’m sorry, I sent you a later revision of mine by mistake.
p
you can disable labels and annotations in the k8s filter but to be honest it sounds like you're not really using it at all then so maybe better to just remove it?
Or you can disable the call to the API entirely and just use the tag?
Probably something like:
Copy code
[FILTER]
  name kubernetes
  match kube.*
  labels off
  annotations off
  use_tag_for_meta on
n
Copy code
inputs: |
    [INPUT]
        Name tail
        Path /var/log/containers/myapp-*_default_*.log
        Parser cri
        Tag kube.*
        ## <https://docs.fluentbit.io/manual/pipeline/outputs>
  filter: |
    [FILTER]
      name kubernetes
      match kube.*
      labels off
      annotations off
      use_tag_for_meta on
  outputs: |
    [OUTPUT]
        Name stdout
        Match *
image.png
p
I think your config is not being used mate, it works fine for me
I notice your YAML is not properly formatted but that might just be a typo here
again, what actual fields do you need though?
also use the setup recommended in the docs for parsing, i.e. multiline parsers: https://docs.fluentbit.io/manual/data-pipeline/inputs/tail#multiline-and-containers
n
Can I check the runtime config of Fluent Bit? I did call
curl <http://127.0.0.1:2020/api/v1/metrics/> | jq
, but it seems to be processed. Actually, I need
kubernetes.host
and
pod_name
.
image.png
p
pod name comes from the log file name
host is something you can inject with the downward API
n
I looked into the documentation, but I feel like I’m running into some error somewhere, so the configs I followed from the documentation aren’t being applied as expected.
p
neither of those need the kubernetes filter
for host name I would use downward API to set an environment variable: https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/ This is possible in the helm chart: https://github.com/fluent/helm-charts/blob/770dee2678273e85fd8eb5d7c0d7c038490bb17c/charts/fluent-bit/values.yaml#L320 Then you can just inject that value as a field directly with modify or record_modifier or whatever by referring to the environment variable using the
${VAR_NAME}
syntax for the value. For pod name you can extract it from the filename using
tag_regex
in the
tail
input: https://docs.fluentbit.io/manual/data-pipeline/inputs/tail You can also add the whole filename automatically with
path_key
n
Thanks you, I will look for more documentation.
p
Copy code
envFrom:
  - name: MY_NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName

config:
  inputs: |
    [INPUT]
      name tail
      tag kube.*
      path /var/log/containers/*.log
      multiline.parser docker, cri
      tag_regex (?<pod_name>[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_.*\.log$
  filters: |
    [FILTER]
      name modify
      match *
      add nodename ${MY_NODE_NAME}
something like this - I've not tested