This message was deleted.
# opals
Slackbot
12/13/2022, 1:19 PMThis message was deleted.
a
Asaf Cohen
12/13/2022, 1:23 PMHi @Jack Geek, these logs are propagate to stderr (this is the loguru default, you can read about that here) and that could be the reason.
We are using datadog for log aggregation and there is a way to configure log processing on their end, i assume GKE has a similar log processing feature.
o
Or Weis
12/13/2022, 1:56 PMThis seems relevant: https://stackoverflow.com/questions/71158475/gcp-log-explorer-shows-wrong-severity-level-of-log-records
The simplest fix would be to add a severity field to the structured logs
j
Jack Geek
12/13/2022, 1:58 PMHi @Or Weis: "add a severity field to the structured logs" , on the OPAL server side ?
o
Or Weis
12/13/2022, 1:59 PMYes, either that or adding another log enriching solution in between.
Should be a pretty easy PR though (add a flag to add the severity field, and add it when logging based on the actual log level)
@Asaf Cohen WDYT ?
a
Asaf Cohen
12/13/2022, 2:05 PMOPAL already have a severity field in the logs (log level) - info, warning, error, critical, etc
Asaf Cohen
12/13/2022, 2:05 PMwhy add another one?
Asaf Cohen
12/13/2022, 2:05 PMOPAL can output the logs as json, simply take the log level and map it correctly in GKE
Asaf Cohen
12/13/2022, 2:23 PMHi @Jack Geek,
From what i understand, you ingest logs in one of several ways and you can also ingest logs with fluentd.
If you do use fluentd i found how to transform json logs, maybe that's helpful?
https://cloud.google.com/logging/docs/reference/v2/rest/v2/entries/write
https://docs.fluentd.org/filter/record_transformer
We can also add a configuration variable to OPAL that let's you control the json format of the logs (without actually duplication the severity field).
Asaf Cohen
12/13/2022, 2:23 PM@Ori Shavit wdyt about adding a configurable json format for opal logs?
o
Ori Shavit
12/13/2022, 2:33 PMAnother format or something customizable?
a
Asaf Cohen
12/13/2022, 4:12 PMa customizable json format
Asaf Cohen
12/13/2022, 4:12 PMis that possible?
o
Ori Shavit
12/13/2022, 6:15 PMI feel this could be difficult from a UX standpoint. How do users specify the format? some kind of schema? not sure how this would look.
o
Or Weis
12/13/2022, 6:16 PMYeah, we can use confi.model() with a Pydantic schema