https://linen.dev logo
Join Slack
Powered by
Would it be possible to have the SegmentMetadata q...
# dev
m
Would it be possible to have the SegmentMetadata queries return distinct values of a column? Similar to how it already supports cardinality, this would just be looking at the keys in the string column dictionaries?
g
it could be a new 
analysisType
m
The use case for this is to populate the drop down for the UI (when user is selecting on a dimension)
This is similar to the Druid UI Explore page
Which I think is issuing a topN right now
i.e.
Copy code
SELECT
  dimension
  COUNT(*)
FROM tbl
GROUP BY 1
ORDER BY 2 DESC
LIMIT 100
I guess we can also issue a groupBY i.e.
Copy code
SELECT
  dimension
FROM tbl
GROUP BY 1
^ would the above groupBy query time be dependent on the number of rows?
I guess a SegmentMetadata query which just look at keys in the string column dictionaries would be independent of the number of rows in the segment
g
yeah the 
groupBy
would be dependent on the # of rows
in theory, there could be an optimization in the 
groupBy
engine to detect queries like this (i.e. dimension with no aggregators, or just one 
count
aggregator) and compute the result based on indexes instead of scanning the data
there isn't currently that optimization tho. but it would be cool
m
Looking a bit more… Would I already be able to accomplish this by using the search query (without a search query aka AllSearchQuerySpec)? i.e.
Copy code
{
  "queryType": "search",
  "dataSource": "my_datasource",
  "searchDimensions": [
    "my_dimension"
  ],
  "intervals": [
    "2013-01-01T00:00:00.000/2025-01-01T00:00:00.000"
  ]
}
This would only look at the bitmap index and not the actual rows right?
also seems like we don’t have SQL equivalent of search query right?
g
yeah the search query would look at the index only; it's not exposed in SQL currently
2 Views
Open in Slack
PreviousNext
Powered by