DennyF
07/18/2025, 11:27 AMpipeline:
inputs:
- name: forward
port: 24224
tag: nomad
listen: 127.0.0.1
but I need to check the content, to route to different outputs:
for example: JSON field com.hashicorp.nomad.namespace contains "backoffice", then I want, that the output goes to a different endpoint.
I thought, that I can use TAG on filters, but I guess, it needs to be done in a different way.Pat
07/18/2025, 11:28 AMrewrite_tag filter or use a grep processor on outputDennyF
07/18/2025, 11:31 AMPat
07/18/2025, 11:32 AMPat
07/18/2025, 11:33 AMPat
07/18/2025, 11:35 AMpipeline:
outputs:
- name: forward
match: '*'
processors:
logs:
- name: grep
# Only forward records that do have a critical priority set.
regex: 'fluentdo_priority ^critical.*$'
Adjust as required but notice I match all records then only allow through ones that have a specific field set to a specific value - look at the grep docs for more details.DennyF
07/18/2025, 11:42 AMDennyF
07/18/2025, 11:43 AMPat
07/18/2025, 11:46 AMtag: kube.* it will be expanded to something like kube.var.logs.containers.podname_namespace_containername and so you can use it for matching, e.g. match_regex: *_namespace1_*
But yeah, it's probably a choice of re-tagging after input or filtering on output.DennyF
07/22/2025, 3:59 AMpipeline:
inputs:
- name: forward
port: 24224
tag: container
listen: 127.0.0.1
filters:
- name: rewrite_tag
match: container
rule: $com.hashicorp.nomad.namespace ^(backoffice-test)$ ns.backoffice false
emitter_name: re_emitted
With stdout I see:
Jul 22 05:53:47 fra-test-nomad-01 fluent-bit[766365]: [0] container: [[1753156426.000000000, {}], {"source"=>"stdout", "com.hashicorp.nomad.namespace"=>"backoffice-test", "com.hashicorp.nomad.node_id"=>"b9e216fb-d242-76e1-641e-ff3efb8f6760", "com.hashicorp.nomad.job_id"=>"backoffice_gunicorn", "log"=>"127.0.0.1 - - [22/Jul/2025:03:53:46 +0000] "GET /adminlogin/ HTTP/2.0" 200 4466 "-" "curl/7.88.1" "-"", "com.hashicorp.nomad.job_name"=>"backoffice_gunicorn", "com.hashicorp.nomad.node_name"=>"fra-test-nomad-01", "com.hashicorp.nomad.task_name"=>"backoffice_gunicorn", "container_id"=>"770f67d86cc22aeedc65fb6801ebdf7fff4fa122947903a8772f6a6fa1522683", "container_name"=>"/backoffice_gunicorn-7d213d9b-a23e-eaf9-124a-5c5d41915de8", "com.hashicorp.nomad.task_group_name"=>"backoffice_gunicorn", "com.hashicorp.nomad.alloc_id"=>"7d213d9b-a23e-eaf9-124a-5c5d41915de8"}]
So from my understanding the docs, it says: I should use "com.hashicorp.nomad.namespace" as key and then check the content, for "backoffice-test". If it matches, than retag "container" to ns.backoffice.
Just as background: Nomad uses docker with fluentd plugin (so "forward" is used).
So, something still is wrong 🙂
btw: big thanks for helping Pat :-)DennyF
07/22/2025, 4:10 AM$ from key and it started to work !