Hi, only one task in ‘RUNNING’ state, second always in ’PENDING”, on druid console, services tab

Could not get capacity info

Could not get detail info

MiddleManager contains

druid.worker.capacity=

where should I start debug?

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

Is more totalNumMergeTasks always better (faster)? If so, any reason why one would set maxNumConcurrentSubTasks > totalNumMergeTasks?

Hi, anyone facing “Still waiting for Handoff for [2] Segments” in Kinesis ingestion, it happens with some tasks only, I see no issues with historicals or coordinator.

We are seeing below error on broker node which causing 5XX error for short duration of time (~2-3mins, 5XX error count was 16k). please help here..

2023-07-02T18:30:21,785 WARN [ServerInventoryView-0] org.apache.druid.curator.inventory.CuratorInventoryManager - Exception while 
getting data for node /druid/segments/ip-10-xxx-xxx-xxx.ap-south-1.compute.internal:8100/ip-10-xxx-xxx-xxx.ap-south-1.compute.internal
:8100_indexer-executor__default_tier_2023-07-02T17:29:22.522Z_08cee3bb400
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /druid/segments/ip-10-xxx-xxx-xxx.ap-south-1.comp
ute.internal:8100/ip-10-xxx-xxx-xxx.ap-south-1.compute.internal:8100_indexer-executor__default_tier_2023-07-02T17:29:22.522Z_08cee3bb400
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[zookeeper-3.5.9.jar:3.5.9]
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.9.jar:3.5.9]
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.9.jar:3.5.9]
        at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:327) ~[curator-framework-4.3.0.jar:
4.3.0]
        at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:316) ~[curator-framework-4.3.0.jar:
4.3.0]
        at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:67) 
~[curator-client-4.3.0.jar:?]
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:81) ~[curator-client-4.3.0.jar:?]
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:313) ~[curator-framework-4.3.0.jar:4.3.0]
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:304) ~[curator-framework-4.3.0.jar:4.3.0]
        at org.apache.curator.framework.imps.GetDataBuilderImpl$1.forPath(GetDataBuilderImpl.java:107) ~[curator-framework-4.3.0.jar:4.3.0]
        at org.apache.curator.framework.imps.GetDataBuilderImpl$1.forPath(GetDataBuilderImpl.java:67) ~[curator-framework-4.3.0.jar:4.3.0]
        at org.apache.druid.curator.inventory.CuratorInventoryManager.getZkDataForNode(CuratorInventoryManager.java:188) [druid-server-0.22.1.jar:0.22.1]
        at org.apache.druid.curator.inventory.CuratorInventoryManager.access$400(CuratorInventoryManager.java:59) [druid-server-0.22.1.jar:0.22.1]
        at org.apache.druid.curator.inventory.CuratorInventoryManager$ContainerCacheListener$InventoryCacheListener.childEvent(CuratorInventoryManager.java:412) [druid-server-0.22.1.jar:0.22.1]

Hello, while running groupBy v2 queries we experience a

Resource limit exceeded

error:

DruidResourceError: Query failed <Response [400]> Bad Request b'{"error":"Resource limit exceeded","errorMessage":"Not enough dictionary memory to execute this query. Try enabling disk spilling by setting druid.query.groupBy.maxOnDiskStorage to an amount of bytes available on your machine for on-disk scratch files. See <https://druid.apache.org/docs/latest/querying/groupbyquery.html#memory-tuning-and-resource-limits>  for details.","errorClass":"org.apache.druid.query.ResourceLimitExceededException","host":"druid-historical-hot-ondemand-0.druid-historical-hot-ondemand.druid.svc.cluster.local:8083"}'.

This happens even with

druid.query.groupBy.maxOnDiskStorage

set to

1024M

. I'm experimenting with increasing this value right now, but I wonder whether adding

ephemeral-storage

for broker pods might be of use here? What do you think? And if yes, I wonder whether there is some additional setup required for the ephemeral-storage to be properly utilized by broker..?

This message was deleted.

Hi Druid cluster is running on 0.18.1. I want to upgrade to 25.x version. Since it's a major upgrade, I would like to know the breaking changes. Please provide a roadmap or things to consider before upgrading. Slack Conversation

Hi Druid Experts I am seeing the following exception in druid coordinator logs few times.

[_default_tier]: Replicant create queue stuck after 2+ runs!: {class=org.apache.druid.server.coordinator.ReplicationThrottler, segments=[segment_id ON histoical-ip:port]}

any idea when this error comes? Thanks in advance

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

Greetings, Humans! I'm your bot from the machine world, Struct AI 🤖 Responding to a beacon sent by one of you kind humans, I've landed here to capture your knowledge, enrich it, and make it effortlessly searchable. My first scan netted 281 thread(s), and I'm always on the lookout for more! Together, we're building an AI and vector-search powered Knowledge Base. Beam yourself over to https://apachedruidworkspace.struct.ai and don't forget to bookmark your coordinates! 🚀 By default, all this knowledge would be visible to members only. So, remember to log in. Let's boldly go where no Slack channel has gone before! 🖖

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.