https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# troubleshooting
s
This message was deleted.
u
Or i am missing something here ?
l
What are the results that you are getting b/w 1 and 2. Sketches are supposed to be approximations so minor variations are expected.
As per my understanding the subquery should have finalized the aggregations. Therefore for the v1, the aggregation should be applied on the finalized version of sub_query (which would already have some approximated and aggregated data) whereas for v2 it should be applied on the whole table (which would be all the raw data). Therefore some variance is expected. This is my understanding of the sketches.
u
My bad: I did not do any post agg (p95 on the ) in sub-query , so i was expecting the aggregation that i will get is raw sketch from sub-query. 
sub_query = quantile_sketch(event.duration) by service, api
results from the queries
Copy code
quantile_p95_v1 = 3767
quantile_p95_v2 = 5000
g
are you using native queries or sql? in sql i believe if you use DS_QUANTILES_SKETCH in the sub query, then the result will remain a sketch and can be further aggregated in an out query
u
I am using native queries.
g
in native, subquery aggregations are by default not finalized. (finalization refers to converting sketch to estimate.) so if you refer to the aggregation from the subquery directly then it should work as you expect
what's your query look like?
u
something like this
Copy code
{
  "queryType": "groupBy",
  "dataSource": {
    "type": "query",
    "query": {
      "dataSource": "events",
      "queryType": "groupBy",
      "filter": {
        "type": "and",
        "fields": [
          {
            "type": "bound",
            "dimension": "create_time",
            "lower": 1666071966,
            "lowerStrict": "false",
            "upper": 1666158366,
            "upperStrict": "false",
            "ordering": "numeric"
          }
        ]
      },
      "intervals": [
        "2022-10-18T05:30:00Z/2022-10-19T06:00:00Z"
      ],
      "granularity": "all",
      "virtualColumns": [],
      "aggregations": [
        {
          "type": "quantilesDoublesSketch",
          "name": "duration_quantile",
          "fieldName": "duration",
          "k": 32768
        }
      ],
      "postAggregations": [
        {
          "type": "expression",
          "name": "duration_quantile_post_agg",
          "expression": "duration_quantile"
        }
      ],
      "dimensions": [
        {
          "type": "extraction",
          "dimension": "create_time",
          "outputName": "__ts",
          "outputType": "LONG",
          "extractionFn": {
            "type": "bucket",
            "size": 60000000,
            "offset": 1666071966
          }
        }
      ]
    }
  },
  "intervals": [
    "2022-10-18T05:30:00Z/2022-10-19T06:00:00Z"
  ],
  "granularity": "all",
  "context": {
    "queryId": "f93ff520-d4b2-4493-8c4c-6ede6d9393f8_1666071966741_1d",
    "timeout": 60000,
    "finalize": true
  },
  "aggregations": [
    {
      "name": "duration_quantile_post_agg_agg",
      "fieldName": "duration_quantile_post_agg",
      "type": "quantilesDoublesSketch"
    }
  ],
  "dimensions": [],
  "postAggregations": [
    {
      "type": "quantilesDoublesSketchToQuantile",
      "name": "quantile_over_time_0.95",
      "field": {
        "type": "fieldAccess",
        "fieldName": "duration_quantile_post_agg_agg",
        "name": "duration_quantile_post_agg_agg_p95"
      },
      "fraction": 0.95
    }
  ],
  "subtotalsSpec": [
    []
  ]
}
g
try reading duration_quantile from the subquery directly
when you read it through the expression postaggregator (duration_quantile_post_agg) it gets finalized
u
thanks for the help, It worked .
g
great to hear 🙂
Open in Slack
PreviousNext
Powered by