Hi team, I have a question regarding MSQE ingestio...
# general
a
Hi team, I have a question regarding MSQE ingestion in Druid. While checking the segment metadata for one of our experiment tables (ingested via MSQE), I noticed that all columns are being treated as dimensions, including numeric fields that logically should be metrics. From Druid’s design, dimensions get dictionary/bitmap indexes, whereas metrics are stored as simple numeric columns, so theoretically, this could add extra overhead. We haven’t tested the performance impact yet, so this is just based on observation from the segment meta and Druid architecture. So just wanted to confirm: Does ingesting everything as dimensions (via MSQE) have any known performance impact? Has anyone faced similar issues, especially on large datasources?
s
you can try aggregating the metrics and grouping by the required dimensions.