This message was deleted.
# troubleshootings
Slackbot
09/22/2022, 10:23 PMThis message was deleted.
j
Julian Reyes
09/22/2022, 10:24 PMworth mentioning that I am using pulumi to make a deployment into EKS by installing helm chart, and I am doing following transformation to be able to download mysql connector driver:
Copy code
transformations: [(obj, opts): void => {
// Link service account to IAM role we created
if (obj.kind === 'Deployment' || obj.kind === 'StatefulSet') {
obj.spec.template.spec.serviceAccountName = serviceAccountName;
}
if (obj.kind === 'Deployment') {
const lifecycle: k8sInputs.core.v1.Lifecycle = {
postStart: {
exec: {
command: [
'cd /tmp',
'wget <https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.49.tar.gz>',
'tar xzf mysql-connector-java-5.1.49.tar.gz',
'mv mysql-connector-java-5.1.49/*.jar /opt/druid/extensions/mysql-metadata-storage/',
'cd -',
]
}
}
}
obj.spec.template.spec.containers[0].lifecycle = lifecycle
}
}],a
Adam Peck
09/22/2022, 11:53 PMLooks like you are loading the extensions
postgresql-metadata-storagemysql-metadata-storage➕ 1
j
Julian Reyes
09/23/2022, 6:06 AMAll right. What happens in the case where I need lookups that connect to external postgresql databases?
Julian Reyes
09/23/2022, 9:32 AMremoved postgresql extension however the coordinator does not seem to be getting healthy because of
Copy code
2022-09-23T09:19:10,573 INFO [main-EventThread] org.apache.curator.framework.imps.EnsembleTracker - New config event received: {server.1=druid-cluster-test-zookeeper-0.druid-cluster-test-zookeeper-headless.druid-cluster.svc.cluster.lo │
│ cal:2888:3888:participant, version=0, server.3=druid-cluster-test-zookeeper-2.druid-cluster-test-zookeeper-headless.druid-cluster.svc.cluster.local:2888:3888:participant, server.2=druid-cluster-test-zookeeper-1.druid-cluster-test-zookee │
│ per-headless.druid-cluster.svc.cluster.local:2888:3888:participant} │
│ 2022-09-23T09:19:10,573 ERROR [main-EventThread] org.apache.curator.framework.imps.EnsembleTracker - Invalid config event received: {server.1=druid-cluster-test-zookeeper-0.druid-cluster-test-zookeeper-headless.druid-cluster.svc.clust │
│ er.local:2888:3888:participant, version=0, server.3=druid-cluster-test-zookeeper-2.druid-cluster-test-zookeeper-headless.druid-cluster.svc.cluster.local:2888:3888:participant, server.2=druid-cluster-test-zookeeper-1.druid-cluster-test-z │
│ ookeeper-headless.druid-cluster.svc.cluster.local:2888:3888:participant}Julian Reyes
09/23/2022, 1:50 PMsome logs from zookeeper
Copy code
2022-09-23 13:41:14,722 [myid:] - WARN [NIOWorkerThread-2:o.a.z.s.NIOServerCnxn@371] - Unexpected exception │
│ org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client, it probably closed the socket: address = /10.3.153.21:41132, session = 0x100006dc9ab0000 │
│ at org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:170) │
│ at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:333) │
│ at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:508) │
│ at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:153) │
│ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) │
│ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) │
│ at java.base/java.lang.Thread.run(Unknown Source) │
│ 2022-09-23 13:41:18,598 [myid:] - INFO [CommitProcessor:1:o.a.z.s.q.LearnerSessionTracker@116] - Committing global session 0x100006dc9ab0001 │
│ 2022-09-23 13:41:21,260 [myid:] - INFO [NIOWorkerThread-2:o.a.z.s.NIOServerCnxn@514] - Processing srvr command from /127.0.0.1:45138 │
│ 2022-09-23 13:41:26,792 [myid:] - INFO [NIOWorkerThread-1:o.a.z.s.NIOServerCnxn@514] - Processing ruok command from /127.0.0.1:45146 │
│ 2022-09-23 13:41:51,250 [myid:] - INFO [NIOWorkerThread-2:o.a.z.s.NIOServerCnxn@514] - Processing srvr command from /127.0.0.1:40150 │
│ 2022-09-23 13:41:56,776 [myid:] - INFO [NIOWorkerThread-1:o.a.z.s.NIOServerCnxn@514] - Processing ruok command from /127.0.0.1:40154 │
│ 2022-09-23 13:42:21,245 [myid:] - INFO [NIOWorkerThread-2:o.a.z.s.NIOServerCnxn@514] - Processing srvr command from /127.0.0.1:45368 │
│ 2022-09-23 13:42:26,784 [myid:] - INFO [NIOWorkerThread-1:o.a.z.s.NIOServerCnxn@514] - Processing ruok command from /127.0.0.1:45372 │
│ 2022-09-23 13:42:34,726 [myid:] - WARN [NIOWorkerThread-2:o.a.z.s.NIOServerCnxn@371] - Unexpected exception │
│ org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client, it probably closed the socket: address = /10.3.153.21:56854, session = 0x100006dc9ab0001 │
│ at org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:170) │
│ at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:333) │
│ at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:508) │
│ at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:153) │
│ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) │
│ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) │
│ at java.base/java.lang.Thread.run(Unknown Source)10.3.153.21Julian Reyes
09/23/2022, 6:25 PMI switched to postgresql and the problem went away