This message was deleted.

so k1024 is the column that has the sketch?

Yes, when running

SELECT "ORDINAL_POSITION", "COLUMN_NAME", "IS_NULLABLE", "DATA_TYPE", "JDBC_TYPE"
FROM INFORMATION_SCHEMA.COLUMNS
where "COLUMN_NAME" = 'k1024'

I got the attached result But running the

ds_histogram

on the column like use suggesting:

SELECT 
  DS_HISTOGRAM("k1024",10,20,30) AS median_s
FROM "wikipedia_new"

Gives me the following error:

Error: SQL query is unsupported

Query not supported. Please check Broker logs for additional details. SQL was: SELECT DS_HISTOGRAM("k1024",10,20,30) AS median_s FROM "wikipedia_new"

org.apache.calcite.plan.RelOptPlanner$CannotPlanException

looks like I was not correct I tried the following

select DS_HISTOGRAM(DS_QUANTILES_SKETCH("sum_added"),20,30,40) from "wikipedia_quantilesketch"

and got

[14386.0,1152.0,1276.0,7619.0]

sum_added is a quantilesdoublesketch created in the druid ingestion

Yes, fix me if I am mistaken: Using

DS_QUANTILES_SKETCH

causes druid to recalculate the sketch and not use the pre-calculated sketch(by the plan I attached)

actually...I did a rollup at ingestion and hence the sum_added has only the sketch not the actual numerical value. So druid cannot recalculate the sktech. What does happen is a sktech merge

Cool, thank you for the answer. Is it possible to see how the query runs under the hood and see that it does sketch merge?

quick note here, if you want to compute a median (as the

AS

in on of your queries implied) you want to use

APPROX_QUANTILE_DS(expr, probability, [k])

with the second argument as 0.5, the first argument could be a sketch column or a numeric column (see https://druid.apache.org/docs/latest/querying/sql-aggregations.html)

Cool, thank you for the answer.

Is it possible to see how the query runs under the hood and see that it does sketch merge?

☝️ for this, if you see a sketch type in

aggregations

with

fieldName

set to the name of a sketch column, then it is doing a sketch merge

@Vadim, you right i forgot to delete the

AS

it's doing one if

k1024

is a sketch field (the JSON won't tell you that; but if it is, then you would get a sketch merge)

Thank you very much for the answer @Gian Merlino It could be nice to see the actual plan of what Druid is doing under the hood. It could help with understanding it better and for optimization.

i agree!! hope to deliver something like that in the future 🙂