https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
a
Hi @Mark Gee, happy to help you here. Can you please let us know which core extension you are using that allows native queries?
m
its the distinct-count extension
i can get the full extension name in a sec
a
yes, pls, thank you
m
a
Right! I was there and reading that.
Let me check and get back here.
m
amazing thanks
a
Druid SQL has a way to turn off the approx count distincts (via druid.sql.planner.useApproximateCountDistinct=false). I am new to this other community extension. I’m not sure if I’m missing something here.
m
yea so distinctcount only supports native queries but we are using SQL and the useApproximageCountDistinct is for the calculation when running 
COUNT(DISTINCT
so different process
a
m
yea so we use that at the moment 
useApproximateCountDistinct
but this plugin provides a function called 
distinctCount
rather then using the build in 
COUNT
combined with 
DISTINCT
a
ah! alright. thank you
Well, have you gone through the “How to Contribute” page: https://github.com/apache/druid/blob/master/CONTRIBUTING.md? If you have specific questions, pls let us know.
m
Yea i just didn't know if this would also apply to extensions :
a
@Vijay Narayanan / @Sergio Ferragut / @Mark Herrera: would you guys know? I do see some code here: https://github.com/apache/druid/tree/master/extensions-contrib/distinctcount/src/main/java/org/apache/druid/query/aggregation/distinctcount
m
Hi @Mark Gee. This might be a solution:
If you'd like to take on maintenance for a community extension, please post on dev@druid.apache.org to let us know!
@Charles Smith as I read the highlighted portion of that doc:
Community extensions are not maintained by Druid committers, although we accept patches from community members using these extensions. They may not have been as extensively tested as the core extensions.
Do you happen to know the mechanism? Would someone be able to post to dev@druid.apache.org to offer a patch?
m
@Mark Herrera We have added a SQL interface so the extension can be called from SQL query rather than being limited to native - I am happy to post this to you with our patch offering
s
@Mark Gee Sounds like a good contribution. The process for contributing is the same for community extensions. Where I'm still at a loss is why is a different distinctCount needed? How does this one differ regular COUNT(DISTINCT ) in exact or approximate calculations. The extension's description doesn't say anything about what it does different.
✅ 1
a
Agreed. Still do not understand what the flow within Druid would be. However, I think the code I pasted above will offer clues @Sergio Ferragut. I’m thinking that @Mark Gee’s new code would perhaps just call one of the already available interfaces with the native query. Is that right, @Mark Gee?
m
Thats right @Ajith Ramanath
a
Cool, I guess we’d know more once you contribute it in the PR!
s
👍
g
fyi for dev questions you can also post to #C030CMF6B70 or dev@druid.apache.org
@Mark Gee as to your specific question: 
CONTRIBUTING.md
applies equally to extensions. You would raise a PR (hopefully with some tests 🙂) and it would be reviewed and merged. When you do raise the PR, feel free to post it in #C030CMF6B70 to remind us
@Sergio Ferragut it's a limited-purpose special-case aggregator; basically it does a local per-segment count and then totals up the local counts. The docs mention that all rows with a particular value for a dimension must go into the same segment; this is because local counts are simply totaled rather than deduped. It's quite fast and memory efficient but only useful when your data is very carefully laid out in a very specific way. Hence it isn't super widely used; it's kind of a hassle to get that right. But, some people do use it!
s
That makes sense, thanks.
m
Open in Slack
PreviousNext
Powered by