Is it possible for Druid to generate multiple rows...
# general
j
Is it possible for Druid to generate multiple rows per events? I have a json structure like:
Copy code
{
  "Timestamp": "2025-02-24T14:13",
  "Results": [
    {
      "Key": "AAA",
      "Value": 35
    },
    {
      "Key": "BBB",
      "Value": 44
    }
  ]
}
I'd like to end up with rows aggregated hourly like:
Copy code
__time,           Key, sum_Value
2025-02-24T14:00, AAA,     33675
2025-02-24T14:00, BBB,     44876
Or at the very least be able to run queries like:
Copy code
SELECT Key, SUM(sum_Value)
FROM datasource
WHERE __time >= CURRENT_TIMESTAMP - INTERVAL 1 DAY
GROUP BY 1
The important point is there is a json list containing dimensions and metrics. I would like to group by the dimensions in that list while aggregating the associated metrics. I'm expecting peak load around 600 K events / second. Storing the metrics without aggregation is not ideal.
j
Hi JRob, You can use unnest() and joins in batch ingestion (e.g. MSQ SQL ingestion) to generate multiple rows per event. I don't know that it can be done via streaming ingestion though.
👀 1
j
@AR The problem with that approach is it explodes the data volume. The cardinality of using
Key
as a dimension field in our specific use case is ~16K but the cardinality of storing the JSON is ~200M.