This message was deleted.

you can add a filter in the ingestion in the transformSpec

thanx, that's what I thought of. but by which column? and won't it fail on the parsing before the filter anyway?

add a placeholder value for the __time and then filter out rows with that placeholder value

Like Vijay said: order of operations is important

Conceptually, after input data records are read, Druid applies ingestion spec components in a particular order: first

flattenSpec

(if any), then

timestampSpec

, then

transformSpec

, and finally

dimensionsSpec

and

metricsSpec

. Keep this in mind when writing your ingestion spec.

So you need to have a default time set to something, then you can filter it with a

transformSpec

got it - nice idea. thanx!!

That filter says you only want

null

rows - you probably want a

not

"filter": {
    "type": "not",
    "field": {
      "type": "selector",
      "dimension": "column3",
      "value": null
    }
  }

worked - thanx guys!!