hello. i have time series data in Druid and am using Superset to chart that data. some charts look at the entire time series, but for some charts i only want to look at the last day's data. i can't configure the time range to be the "Last Day" or any of the other time range filters because it's possible i don't have data from "yesterday" in relation to "today". can someone help me with a custom SQL that would get the last day's (in the data, not in time) data for my chart?

Hey everyone! 👋 Join us for a Real-Time Analytics & AI at Scale meetup in New York City on June 18! It’s a casual, in-person gathering for devs working on big data, distributed systems, AI infra, or just curious about how modern stacks scale real-time analytics. 📍New York City 🗓️ Wednesday June 18th 🍕 Talks + food + networking https://www.pingcap.com/event/real-time-analytics-and-ai-at-scale-meetup/ No fluff — just solid tech talks, cool people, and hands-on lessons from the field. Hope to see you there! 🙂

Hi everyone, does it make sense to create an MCP Server for Druid? Has anyone done it?

hey everyone, ive been using druid to write queries on a large datasource. but recently when i’ve tried to run it on a datasource with a size of 5.2GB its started showing bad gateway. i should mention im using a single server small set up. if anyone can help me out i would really appreciate it

Hi everyone, we have a usecase where we suspend and resume supervisors continuously to manage resources used for kafka ingestion . Lately we have seen that the overload uses a lot of memory (90GB plus ) so quick question does suspended supervisors add to overload memory ?

Hi everyone, getting this issue with msq

RowTooLarge: Encountered row that cannot fit in a single frame (max frame size = 1,000,000)

How does one increase the maxFrameSize, tried setting this in contextParam but of no use

Exception while seeking to the [earliest] offset of partitions in topic [schema_history_remote_postgres]: Timeout expired while fetching topic metadata while i'm trying to fetch data from kafka

idk much about druid i just started learning it

can anyone say me what i'm doing wrong here ?

ohh it worked with Docker container name instead of localhost

Im looking to assign different run priority to different ingestion tasks. (Meaning, which task gets to run, not related to task locking priority). We're ingesting data in a Lambda fashion: new data is ingested in streaming and then re-ingested after a week with batch. Since the batch re-ingestion has a much more relaxed SLA, we were thinking to let both workload share the same hardware resources, but protect the streaming by giving it higher run priority. But I couldn't find anything about such configuration - does this mechanism exist? I wonder if MM-less can support it, as Kubernetes has these mechanisms built-in.

Hi guys, Hope you're doing well, Is there any way to ingest data from 3 kafka servers into 1 datasource, docs says that we can only have 1 kafka per datasource and submitting another spec file would just replace the previous one, but is there any other way? Any help would be really appreciated (beginner on druid), Thank you

Any other alternatives would help aswell, where we can ingest data from mutiple kafka servers to 1 data source 🙂

is there a way i can check how much memory is being used during a query

Hi all, hope this is the place to ask, if not then could you point me in the correct direction, it would be appreciated. We’ve have two time fields, __time (which is the time the event is processed in our system) and the transaction time, when the event actually occurred. The transaction time is stored as is laid out in the docs (having it as millisecond timestamps stored as long types): https://druid.apache.org/docs/latest/ingestion/schema-design/#secondary-timestamps. The issue is that we are now wanting to filter on the transaction time which can differ to __time by years in the past and future, meaning if we want to filter on this time field we end up performing a full table scan, which ends up taking a long time to run. Reading previous questions around this and the documentation it looks like we should have a second partition on the transaction time field: https://druid.apache.org/docs/latest/ingestion/partitioning/#secondary-partitioning. I just wanted to check that is correct and I’m not overlooking something. Many thanks, Robert

Hi everyone, 🌉 Are you in San Francisco or the surrounding area? Come check out the Real-Time Data Lakes featuring ClickHouse® and StarRocks meetup on July 8! Featuring talks from Altinity, PostHog, and CelerData. • Tuesday July 8th | San Francisco • Great talks, networking, and food 🍻 Register: https://lu.ma/dccyqxyu

Are there any plans to add support to PromQL in Druid? Thanks

hi everyone. We are currently building an MCP Server for Druid and wonder if we can get some feedback here.

Team, does anyone implement druid emitter prometheus for monitoring ? can you help to share the some of the metrics?

Hi All, I wanted to check if there is a way to clean up only the older segment versions from metadata and deep storage, while retaining the latest versions safely in deep storage. We have a use case where we want to maintain retention in Druid as per policy. However, since we are running frequent compactions, we do not want to retain older segment versions unnecessarily. I came across the following properties: •

druid.coordinator.kill.on

•

druid.coordinator.kill.period

•

druid.coordinator.kill.durationToRetain

From what I understand, these properties delete all unused segments after a certain period, but not specifically older versions if newer versions exist. Is there a way to configure Druid to only delete older segment versions while keeping the latest one for each interval?

Hi All, We have a Druid cluster set up as follows: Node 1 – Master Server MySQL, Zookeeper, Coordinator, Overlord, Broker, Router Node 2 Data Server, Historical We’re currently handling 7 billion records in our Druid database. However, some queries are running indefinitely and eventually timing out. One example query that’s causing issues is:

SELECT COUNT(*)

FROM (

SELECT DISTINCT id

FROM table

WHERE responsetype IN ('Sent', 'SENT', 'failed')

)

We would really appreciate your help in tuning our Druid cluster for better performance.

👋 Hi everyone! I'm new to the channel and glad to be part the Druid community!

Is anyone here running Druid in docker on ARM? It doesn't look like there's an ARM image available on dockerhub, but I'm curious if anyone has had success building an image from source and running it on ARM hardware.

Hello everyone, I'm trying to extract data from deep storage with a query to the /druid/v2/sql/statements/ endpoint. The task runs fine, but at the end, the following error occurs, preventing me from extracting the query response. { "error": "druidException", "errorCode": "notFound", "persona": "USER", "category": "NOT_FOUND", "errorMessage": "Query [query-9578562a-94f0-452d-998a-e66e0f7d0ff5] was not found. The query details are no longer present or might not be of the type [query_controller]. Verify that the id is correct.", "context": {} } Does anyone know why this happens?

Hey everyone, we have released druid-mcp-server https://github.com/iunera/druid-mcp-server allowing to connect Druid to MCP Clients like claude desktop, copilot etc. to make Druid AI usable. We've tested it on our usecases and it looked pretty impressive how easy Druid became. Regardless if Query, MSQs, Ingestion or Ops. We'd tried to implement all Druid Features as MCP Tools. Nevermind, I would love to get some feedback on different scenarios due to the fact that there are a lot of diversity in them esp. whats missing and what didn't work properly.

Hello, anyone else noticed druid.struct.ai hasn't been available for the past few days?

hi all. how do you specify for a data source that it should monitor a directory for new files and ingest them? thanks!

i added this bit to the data source's spec file, but it doesn't seem to work.

"spec": {
    "ioConfig": {
        "watcher": {
          "type": "file",
          "pollPeriod": "PT10M"
        }

Hi folks, please review this PR to upgrade pac4j extension: https://github.com/apache/druid/pull/18259 This major upgrade is needed to fix CVE-2023-52428 which requires upgrading nimbus-jose-jwt to

9.37.2

which is not compatible with pac4j v4.

Druid combined with an AI. Natural language queries are then possible with Druid.