This message was deleted.
# troubleshootings
Slackbot
01/25/2023, 8:36 PMThis message was deleted.
s
Sergio Ferragut
01/25/2023, 8:55 PMI think the best person to answer that is @Cory Johannsen. Cory has been working on such a deployment and made improvements to the extension.
y
Younes Naguib
01/25/2023, 8:55 PMIf I can do without zk, I will 🙂
c
Cory Johannsen
01/25/2023, 8:56 PMThere's an open PR with a fix for a resource leak. We run a locally built version of druid that includes this fix. It's not perfect, but it does work around the underlying issue.
Cory Johannsen
01/25/2023, 8:57 PMWe automatically cycle the coordinator pods every ~12 hours to force a leader change and clean up any leaked http connections
Cory Johannsen
01/25/2023, 8:58 PMWe run this configuration in our staging and production environments and so far it has been stable.
y
Younes Naguib
01/25/2023, 8:59 PMNice! I want to use in an early stage Prod.
Younes Naguib
01/25/2023, 8:59 PMWill only start being customer facing in ~ 3months
c
Cory Johannsen
01/25/2023, 9:00 PMIf you use the extension as-is (without the fix in place) you will see the leader coordinator leak an http connection every time an indexer pod is removed.
y
Younes Naguib
01/25/2023, 9:01 PMWhen will the fix be GA?
Younes Naguib
01/25/2023, 9:02 PMWhich PR are you referring to? We could just build a version for us
s
Sergio Ferragut
01/25/2023, 9:02 PMAdditionally, there are also options to avoid ZK for certain functions. In Apache Druid 25.0 most of these are now the defaults. Check out the PR: https://github.com/apache/druid/pull/13092
y
Younes Naguib
01/25/2023, 9:03 PMGreat. Thanks, I'll check it out
c
Cory Johannsen
01/25/2023, 9:03 PMThis the PR https://github.com/apache/druid/pull/13667
Cory Johannsen
01/25/2023, 9:04 PMI would love to test is using the extension that removes the middlemanager actually works around this issue, I believe it may. I have not had time.
y
Younes Naguib
01/25/2023, 9:05 PMIf we can help, let me know