Hi, Question around projections. - Do i have to d...
# general
u
Hi, Question around projections. • Do i have to define the projection as part of ingestion spec if so why ? i was wondering why not like sql views so that it is available for older segments as well and older segment can build it at the time of loading ? may there are some cons doing that which i might have missed to understand. • if i create a projection and if i want to later delete it how do i do that ? do i have to re-compact with projection spec deleted from it ? • I could not find documentation for trying this out, is it not recommended to use at this moment ?
k
• Projections are build as part of segment generation hence they are part of the ingestion spec. • If you need to delete the projected col from the data source yes, you would have to replace the datasource again without the projection settings.
This is some documentation mentioned here : https://github.com/apache/druid/issues/17481
g
one more note; the direction we're heading is to use the catalog to store projection configuration, rather than the ingest spec -- some work towards that in https://github.com/apache/druid/pull/17803
c
slight correction, https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/data/input/impl/AggregateProjectionSpec.java is the spec type, not
AggregateProjectionMetadata
(the latter is what the spec is computed into and stored inside of the segment)
u
Sorry i was away last week. let me check these.