This message was deleted.

at present you have to define the metadata on the extern. If you have your data on the filesystem you can use the console wizard to sample the data and write out the metadata for you

For the regular ingestion even if we don’t set

DimensionsSpec

it identifies dynamically from the file. Why is that not true for SQL based ingestion ?

It's because the planning for the SQL statement needs to know about the columns available in the input files It's mentioned in the known issues for

EXTERN

here: https://druid.apache.org/docs/latest/multi-stage-query/known-issues.html#extern We will be addressing this in a future release by making schemaless + SQL = ❤️, so stay tuned!!