This message was deleted.

This message was deleted.

Hi, I'm struggling with an Druid issue on my 5-KVM cluster with Hadoop, Spark, Zookeeper and Druid. Zookeeper seems to be running OK, since I cant get the nodes that are following and the leader with commands like

echo stat | nc -v hadoop-master 2181

. Also, I've configured the zoo.cfg file with lines like this (at

hadoop-master

, in this case):

server.1=0.0.0.0:2888:3888
server.2=hadoop-worker1:2888:3888
server.3=hadoop-worker2:2888:3888
server.4=spark-worker1:2888:3888
server.5=spark-worker2:2888:3888

Druid runs without erros, except for Router, which sends a log with the exception

org.apache.druid.java.util.common.IOE: No known server

. Coordinator shows this warning:

org.apache.druid.server.coordinator.duty.RunRules - Uh... I have no servers. Not assigning anything...

. In fact, Druid UI shows some exceptions and doesn't show the coordinator service (see images). Coordinator is running at

hadoop-master

, data (middleManager and Historical) at

hadoop-worker1

, another data node at

hadoop-worker2

and query node (Broker and Router) at

spark-worker1

. What I am supposed to try? Thanks.

This message was deleted.

I am setting up a lot of

index_parallel

jobs. They stop (both

index_parallel

and it's sub tasks

partial_index_generic_merge

) on

Found [1] expired supervisor tasks

I found mailing lists threads but they don't have a solution. Anyone got an idea? Stop as in "stale in running state" not as in "failed".

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

Hi All, I am unable to issue a kill task for my datasource through console. I am getting this error

2023-06-26T19:23:09,368 WARN [HttpClient-Netty-Worker-27] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[<http://172.25.2.83:8088>] was returned multiple times?
2023-06-26T19:24:10,092 INFO [ServiceClientFactory-1] org.apache.druid.rpc.ServiceClientImpl - Service [OVERLORD] request [POST <http://172.25.2.83:8088/druid/indexer/v1/action>] encountered exception on attempt #29; retrying in 60,000 ms (org.jboss.netty.channel.ChannelException: Channel disconnected)
2023-06-26T19:24:10,092 WARN [HttpClient-Netty-Worker-28] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[<http://172.25.2.83:8088>] was returned multiple times?
2023-06-26T19:25:10,819 INFO [ServiceClientFactory-2] org.apache.druid.rpc.ServiceClientImpl - Service [OVERLORD] request [POST <http://172.25.2.83:8088/druid/indexer/v1/action>] encountered exception on attempt #30; retrying in 60,000 ms (org.jboss.netty.channel.ChannelException: Channel disconnected)
2023-06-26T19:25:10,819 WARN [HttpClient-Netty-Worker-29] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[<http://172.25.2.83:8088>] was returned multiple times?

It eventually gets shutdown

Hello… Does Druid supports Erasure Coded Directories? I am using 0.26.0 and hadoop 3.3.5 and the issue seems that it does write the files to deep storage, but it never actually closes the file.. and which causes for tasks to abort.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

Hi guys, I have stumbled upon the following issue while trying to build Druid from source:

WARN [main] org.apache.druid.java.util.common.RetryUtils - Retrying (8 of 9) in 73,316ms.
org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException: java.sql.SQLException: Cannot create PoolableConnectionFactory (Unknown system variable 'transaction_isolation')
at org.skife.jdbi.v2.DBI.open(DBI.java:230) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:279) ~[jdbi-2.63.1.jar:2.63.1]
at org.apache.druid.metadata.SQLMetadataConnector.lambda$retryWithHandle$0(SQLMetadataConnector.java:142) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:129) ~[druid-core-25.0.0.jar:25.0.0]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81) ~[druid-core-25.0.0.jar:25.0.0]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:163) ~[druid-core-25.0.0.jar:25.0.0]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:153) ~[druid-core-25.0.0.jar:25.0.0]
at org.apache.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:142) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:152) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.druid.metadata.SQLMetadataConnector.createTable(SQLMetadataConnector.java:189) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.druid.metadata.SQLMetadataConnector.createPendingSegmentsTable(SQLMetadataConnector.java:217) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.druid.metadata.SQLMetadataConnector.createPendingSegmentsTable(SQLMetadataConnector.java:603) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.druid.cli.CreateTables.run(CreateTables.java:125) ~[druid-services-25.0.0.jar:25.0.0]
at org.apache.druid.cli.Main.main(Main.java:112) ~[druid-services-25.0.0.jar:25.0.0]
Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (Unknown system variable 'transaction_isolation')
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2152) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.open(DBI.java:212) ~[jdbi-2.63.1.jar:2.63.1]
... 13 more
Caused by: java.sql.SQLException: Unknown system variable 'transaction_isolation'
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3933) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3869) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2524) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2675) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2465) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2439) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1365) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.loadServerVariables(ConnectionImpl.java:3767) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.initializePropsFromServer(ConnectionImpl.java:3196) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2233) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2015) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:768) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at jdk.internal.reflect.GeneratedConstructorAccessor37.newInstance(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at com.mysql.jdbc.Util.handleNewInstance(Util.java:403) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:385) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:323) ~[mysql-connector-java-5.1.49.jar:5.1.49]
at org.apache.druid.metadata.BasicDataSourceExt.lambda$createConnectionFactory$0(BasicDataSourceExt.java:185) ~[druid-server-25.0.0.jar:25.0.0]
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.open(DBI.java:212) ~[jdbi-2.63.1.jar:2.63.1]
... 13 more

Not sure what's going on so your help will be much appreciated. Thanks!

The following resolved it: https://druid.apache.org/docs/latest/development/extensions-core/mysql.html#alternative-installing-the-mariadb-connector-library

So it looks like MySQL connector library doesn't work with the latest MariaDB version:

$ mariadb
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 13
Server version: 11.0.2-MariaDB-1:11.0.2+maria~ubu2004 <http://mariadb.org|mariadb.org> binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

This one I used until now: https://druid.apache.org/docs/latest/development/extensions-core/mysql.html#installing-the-mysql-connector-library

This message was deleted.

This message was deleted.

Hi All,

This message was deleted.

HI, is there a way to restrict pac4j to enable for only some endpoints

Hi! I got following error from historical service. I thought this error is from while merging query result, but not sure. What could be the reason?

2023-06-28T03:26:26,152 ERROR org.apache.druid.query.groupby.epinephelinae.GroupByMergingQueryRunnerV2 - Exception with one of the sequences!