Saral
07/04/2025, 12:26 PM- lua:
call: normalize_product_logs
script:
name: fluent-bit-modify-k8s-fields
key: custom.lua
The custom.lua script uses environment variables like drop_error_product, drop_debug_product, etc., to control which logs should be dropped dynamically. For example:
if log_level == "ERROR" and drop_error_product == "true" then
return -1 -- drop log
end
These variables are passed via environment variables in the Fluent Bit container or set via config maps.
Problem
When I update the values of these variables (e.g., setting drop_error_product from false to true) and confirm that the ConfigMap is updated and pods are restarted manually, the updated logic does not take effect.
However, the moment I:
• remove the entire Lua block from the filter config and deploy
• then add the same Lua block back and deploy again
...the changes start reflecting and logs get dropped correctly.
Is Fluent Bit or the Lua plugin caching the script or its environment somewhere internally?
Is there a supported or recommended way to force Fluent Bit to reload updated Lua script logic and/or environment config?
Thanks a lot — any insights or best practices would be appreciated!Pat
07/04/2025, 12:47 PMPat
07/04/2025, 12:47 PMSaral
07/04/2025, 12:52 PMPat
07/04/2025, 12:56 PMPat
07/04/2025, 12:56 PMSaral
07/04/2025, 12:58 PMfunction normalize_product_logs(tag, timestamp, record)
local function canonicalize(val)
if not val then return nil end
local lvl = tostring(val):lower()
if lvl == "error" then return "ERROR"
elseif lvl == "warn" or lvl == "warning" then return "WARN"
elseif lvl == "info" then return "INFO"
elseif lvl == "debug" then return "DEBUG"
elseif lvl == "fatal" then return "FATAL"
else return lvl:upper()
end
end
-- Canonical drop list
local drop_levels = {}
{{- if .Values.log_filters.drop_debug_product }} drop_levels["DEBUG"] = true {{- end }}
{{- if .Values.log_filters.drop_info_product }} drop_levels["INFO"] = true {{- end }}
{{- if .Values.log_filters.drop_warn_product }} drop_levels["WARN"] = true {{- end }}
{{- if .Values.log_filters.drop_error_product }} drop_levels["ERROR"] = true {{- end }}
{{- if .Values.log_filters.drop_fatal_product }} drop_levels["FATAL"] = true {{- end }}
local log_level = nil
local top_keys = {"logLevel", "level"}
for _, key in ipairs(top_keys) do
local val = record[key]
if val then
local canon = canonicalize(val)
if drop_levels[canon] then
record[key] = canon
log_level = canon
end
end
end
if record["log_processed"] then
for _, key in ipairs(top_keys) do
local val = record["log_processed"][key]
if val then
local canon = canonicalize(val)
if drop_levels[canon] then
record["log_processed"][key] = canon
log_level = log_level or canon
end
end
end
end
if not log_level and record["log_processed"] then
log_level = canonicalize(record["log_processed"]["logLevel"] or record["log_processed"]["level"])
end
if not log_level then
log_level = canonicalize(record["logLevel"] or record["level"])
end
if log_level and drop_levels[log_level] then
return -1
end
return 2, timestamp, record
end
{{- end }}Saral
07/04/2025, 1:00 PMfunction normalize_product_logs(tag, timestamp, record)
local function canonicalize(val)
if not val then return nil end
local lvl = tostring(val):lower()
if lvl == "error" then return "ERROR"
elseif lvl == "warn" or lvl == "warning" then return "WARN"
elseif lvl == "info" then return "INFO"
elseif lvl == "debug" then return "DEBUG"
elseif lvl == "fatal" then return "FATAL"
else return lvl:upper()
end
end
-- Canonical drop list
local drop_levels = {} drop_levels["INFO"] = true drop_levels["ERROR"] = true
local log_level = nil
local top_keys = {"logLevel", "level"}
for _, key in ipairs(top_keys) do
local val = record[key]
if val then
local canon = canonicalize(val)
if drop_levels[canon] then
record[key] = canon
log_level = canon
end
end
end
if record["log_processed"] then
for _, key in ipairs(top_keys) do
local val = record["log_processed"][key]
if val then
local canon = canonicalize(val)
if drop_levels[canon] then
record["log_processed"][key] = canon
log_level = log_level or canon
end
end
end
end
if not log_level and record["log_processed"] then
log_level = canonicalize(record["log_processed"]["logLevel"] or record["log_processed"]["level"])
end
if not log_level then
log_level = canonicalize(record["logLevel"] or record["level"])
end
if log_level and drop_levels[log_level] then
return -1
end
return 2, timestamp, record
endSaral
07/04/2025, 1:03 PM-- Canonical drop list
local drop_levels = {} drop_levels["INFO"] = true drop_levels["ERROR"] = true
to
-- Canonical drop list
local drop_levels = {} drop_levels["FATAL"] = true drop_levels["INFO"] = true
Cm gets updated but it doesn't reflect the same and when i remove lua block in filter and add again it worksPat
07/04/2025, 1:15 PMSaral
07/04/2025, 4:35 PMcustom.lua) stored in a ConfigMap, which is mounted into the Fluent Bit container.
• The Lua script contains logic that checks environment variables (set in the container) like drop_error_product, drop_debug_product, etc. Example in custom.lua
• When I want to change behavior (e.g., start dropping ERROR logs), I do not edit the Lua script itself — I only update the environment variable value in the pod spec (e.g., drop_error_product=true).
• I confirm that:
The ConfigMap is correctly mounted
The pod has the updated environment variable.
However, after restarting the Fluent Bit pods, the new environment variable values do not seem to affect the Lua behavior — ERROR logs are still not dropped.
The only thing that works is:
• Remove the Lua filter block from the config
• Then re-add the same Lua filter block
I hope this clear at bit 😅.Pat
07/04/2025, 6:34 PMSaral
07/05/2025, 5:46 AMPat
07/07/2025, 8:51 AM-debug to the tag namePat
07/07/2025, 8:51 AM