Hi All, We extended the pydruid connector to support Druid’s MSQ (Multi-Stage Query) engine, which means Superset dashboards and charts can now use the MSQ engine. In addition, Superset can finally cancel running Druid queries, preventing wasted resources and speeding up the user experience. These enhancements make Superset more responsive for analysts and more efficient for operators. We’ve written a blog on the changes here https://millersoft.co/blog Please reach out if you want to try the new driver? Calum Miller

Hi, Can someone please review this PR: https://github.com/apache/druid/pull/18634. This is to add a new task distribution strategy based on supervisor affinity. Thanks

Hi everyone ! I’m wondering if there are any plans to support native UUID types in Druid. Currently, we have to store UUIDs as strings, which makes querying them inefficient, and we have to rely on workarounds like lookups or mapping them to Long values. I couldn’t find any existing feature request about this on GitHub. Am I the only one encountering this issue, or is this a common limitation?

I am looking for new position as full stack developer now.

Hi, How do you run unit tests with Druid? I'm currently using a docker compose file with test containers, spawning the entire stack for the whole test suite. It's working fine, but it's slow to start and very slow to clean data between each test...

Hi everyone, I'm trying to understand the exact behavior when

tieredReplicants

is set higher than the number of historicals in a tier. Setup example: • Tier has 3 historicals • Datasource configured with

tieredReplicants: 5

Question: What actually happens in this case? 1. Does Druid cap the replicas at 3 (one per historical)? 2. Can a single historical load multiple copies of the same segment to satisfy the replication factor? 3. Does it fail/warn/queue the additional replicas? I couldn't find explicit documentation about this edge case. The architecture seems designed to distribute segments across different historicals, but I want to confirm the actual behavior when requested replicas exceed available nodes. Has anyone tested this scenario or can point me to the relevant code/docs that clarifies this? Thanks!

Hi team. Was wondering if there is any reason why editing load rules does not allow for deletion of entries for datasources? Our use case has a high throughput of datasources being added and deleted with specific load rules so the load rules file tends to get updated. Since there is no delete option for load rules : this causes the file to get bloated with "deleted-datasource":[] type of entries. Lmk if I've missed something here.

Hello Everyone Has anyone work on reading the datafrom Kafka protobuf from druid. and getting multiple error Can someone get on a call to help me to fix it ?? its bit urgent

Hi team, I'm still using druid (~ 5 years), I'm running Druid 27 and trying to separate the coordinator from the overlord (previously running in combined mode with

druid.coordinator.asOverlord.enabled=true

). Problem: After applying the new configuration with separate coordinator and overlord StatefulSets, the coordinator keeps crashing with OOM errors. What I've done: 1. Updated the Druid CR to have separate

coordinators

and

overlords

sections 2. Deleted the existing coordinator StatefulSet (required due to immutable field changes like port) 3. Manually created the missing task tables (

druid_tasks

,

druid_tasklogs

,

druid_tasklocks

) since they didn't exist - my metadata DB only had the 7 base tables (druid_audit, druid_config, druid_datasource, druid_pendingsegments, druid_rules, druid_segments, druid_supervisors) 4. Schema I used: https://gist.github.com/renatocron/8056649b67cc53b02a44a6f98fd30d5b - generated via claude from server/src/main/java/org/apache/druid/metadata/SQLMetadataStorageActionHandler.java - this file do not exists anymore in 35 Current state: • Tables are created, no more "relation does not exist" errors But coordinator is now OOM crashing, I reverted everthing and dropped tables now, I set druid.metadata.storage.connector.createTables=true on coordinator but even then it would not ceate table, was this a bug in the older version?

Anyone here actively working on the druid-operator ( https://github.com/datainfrahq/druid-operator) ?

Can an admin create a druid-operator slack channel in this workspace?

Could someone help take a look at this PR? Our company is very eager to have it merged into master as soon as possible. https://github.com/apache/druid/pull/18750

Hi group! I wanted to check whether Druid can query Iceberg tables directly without ingesting the data first. Are there any plans to support this capability in the future? Thank you.

Could someone help take a look at this PR? Our company is very eager to have it merged into master as soon as possible. https://github.com/apache/druid/pull/18750

I am trying to secure the communication between Druid and ZooKeeper using SASL + digest ACLs. However, I want to confirm the correct and recommended way to do this because Druid creates a large number of znodes dynamically under /druid (discovery, announcements, indexer tasks, segment metadata, etc.), and the structure changes frequently during normal cluster operation. Whats the best way forward

Hi team, I have a question regarding MSQE ingestion in Druid. While checking the segment metadata for one of our experiment tables (ingested via MSQE), I noticed that all columns are being treated as dimensions, including numeric fields that logically should be metrics. From Druid’s design, dimensions get dictionary/bitmap indexes, whereas metrics are stored as simple numeric columns, so theoretically, this could add extra overhead. We haven’t tested the performance impact yet, so this is just based on observation from the segment meta and Druid architecture. So just wanted to confirm: Does ingesting everything as dimensions (via MSQE) have any known performance impact? Has anyone faced similar issues, especially on large datasources?

Anyone ever thought of using S3 as Druid deep storage and use

standard-IA

s3 storage class on the s3 files when uploaded to s3?

Hey y'all, does druid publish any metrics specifically for ingestion autoscaling? I'd like to make an alert based around (for example) if autoscaling has increased the number of tasks more than 5 times over the last 24 hours. I could just go based on the number of tasks but that'd cause redundant alerts if it was manual action scaling them.

We are having issues with certain slow queries essentially starving all other queries. We have tried query laning (using 200 segments as the threshold) to some success but it isn't perfect. We are now running into an issue where that 200 segment threshold works for some datasources, but definitely not others. For example, we would like it to be 100 segments for some datasources and 500 segments for others. Is there any way to prioritize traffic based on some other performance metric (e.g. the # of rows or # of bytes)??? The cluster runs fine most of the time but every once in a while, some queries come along starving out all of our other queries, and then we get endless user complaints that Druid is unavailable / unreliable / etc...

Hi all, we've been experimenting with range partitioning, in which one of the columns we partition on is a long (epoch-milli) that we use comparators like

>,<,=

in the filter - just checking - is broker level segment pruning supported for longs? In the docs it seems to only refer to string column use case

Hello Druid experts, Need your suggestions on running performance issues in Druid 34 with reading Iceberg, Glue catalog. so help me. thanks in advance. https://apachedruidworkspace.slack.com/archives/C0309C9L90D/p1765644475297199

Need help in understanding these two historical metrics- query/segmentAndCache/time and query/segment/time. From the description query/segment/time is time to query individual segment including time taken to fetch it from disk. Then what is the utility of query/segmentAndCache/time metric

We did rolling restarts yesterday and today. In short; a busy broker took 2.5 hours to restart and a non-busy broker took 4 minutes. Is there any optimization that can be made during broker start sequence to avoid needing to wait hours for startup? We have 3 query nodes. All three are behind a LB but one of the nodes got hard-coded by a busy client. An obvious solution is to have all clients use the LB but there must be something else we can do to improve the startup time of a broker who got pinned by a client.

Hello Druid Experts, Has anyone performed benchmarks specifically for the “Virtual Storage” feature in Druid version 35? I’m interested in understanding the query latency metrics when Virtual Storage is enabled. Also, since this feature is marked as experimental in version 35, is it expected to be production-ready in the next release?

Is anyone using the druid-kubernetes-extensions in production? Its still labelled experimental, although there's an issue to take it out of that status (from 2022 😬). It doesn't look like its under active development based on the history.

Hi all, I am planning an upgrade from Druid 29.0.1 to 30.0.1. Normally we double capacity and replace historical nodes, but currently we can’t provision extra EC2 instances, and decommissioning/replacing historicals one by one would be very time-consuming (each node has ~10–28 TB of local segment cache on instance-store NVMe). Since this is an application-level upgrade, we’re considering an

in-place upgrade

for Historicals: • stop the Historical process • clean up tmp directories and old Druid 29.x binaries • keep the local

segmentCache

directory(data dir) intact • install Druid 30.x and restart the service The expectation is that on startup the Historical will re-announce and reuse existing local segments (no deep-storage re-downloads), so downtime per node is ~10–15 minutes. We have replicas, so no query downtime is expected. Is this approach supported for major version upgrades? Are there any known issues to watch out for (segment compatibility, tmp directories, map state, or other on-disk artifacts) when reusing the same instance and segment cache across 29.x -> 30.x? Any guidance or confirmation would be appreciated. Thanks!

Hi all, We are fine tuning few druid properties and had few doubts when going through docs, could someone please clarify 1. Query processing buffers/threads •

druid.processing.buffer.sizeBytes

•

druid.processing.numMergeBuffers

•

druid.processing.numThreads

I want to understand how are these used when processing the druid queries, couldn't get much clarity from docs. Could anyone please explain these? I see there are formulas in the docs, but wanted to understand the logic behind them. For example, when we increase

numThreads

, more query processing and row scanning happens in parallel, but at some point groupBy/aggregation phases require merge buffers. If merge buffers are insufficient, does query processing block waiting for a merge buffer? What are the key factors to decide appropriate values for these? https://druid.apache.org/docs/latest/configuration/#historical 2. Heap vs direct memory allocation Here, https://druid.apache.org/docs/30.0.1/configuration/#segmentwriteoutmediumfactory it mentions that for MiddleManagers, heap and direct memory should be the same size. Why is that requirement there? Also, does this recommendation apply to Historicals as well? Thanks

One detail. it was Druid + Superset +AI project. And chatbot prepared links to Superset dashboards. Druid MCP server is close to it but not the thing that I'm trying to find. Still can't find those videos...

Hello Druid Community, Do we know if there is workaround for this issue of Incorrect row signature order ? https://github.com/apache/druid/issues/18437

Do we have any news about this or in general about Druid working on Graviton Instance (Arm64 arc) (very old issue talking about it https://github.com/apache/druid/issues/11820)