Hey, we are using presto on top of pinot. And we w...
# general
n
Hey, we are using presto on top of pinot. And we want to build star-tree index on the table. The aggregation function is DistinctCountHLL. And I will also use approx_distinct in prestoDB which is also back by HLL. I am wondering will presto respect this star-tree index in pinot?
m
From presto code I see it might be supported:
Copy code
private String handleApproxDistinct(CallExpression aggregation, Map<VariableReferenceExpression, Selection> inputSelections)
        {
            List<RowExpression> inputs = aggregation.getArguments();
            if (inputs.isEmpty() || inputs.size() > 2) {
                throw new PinotException(PINOT_UNSUPPORTED_EXPRESSION, Optional.empty(), "Cannot handle approx_distinct function " + aggregation);
            }
            Selection selection = inputSelections.get(getVariableReference(inputs.get(0)));
            if (inputs.size() == 1) {
                return format("DISTINCTCOUNTHLL(%s)", selection);
            }
            RowExpression standardErrorInput = inputs.get(1);
            String standardErrorString;
            if (standardErrorInput instanceof ConstantExpression) {
                standardErrorString = getLiteralAsString((ConstantExpression) standardErrorInput);
            }
@User to also confirm.
In the meanwhile @User could you
explain
the query on presto side? It might show the query being sent to Pinot, where you can verify if it sent DistinctCountHLL to Pinot
x
Try to use explain to see the query plan
We have rewritten the aggregation and filter parts to Pinot query and push down
But yeah, explain the query should tell the actual pinot sql
x
Yes, approx_distinct will be converted to distinctCounthll
👍 1
n
Thank you guy! Sorry for the late reply. my company's vpn does not white list slack.
I will look into the presto explain.
it does convert the presto's approx_count to pinot's countdistinctHHL. Thanks.
👍 1