Apache Pinot

It’s WIP. There is a concept of applying functions during ingestion.

We should probably allow a custom field extractor that extracts a target schema from a source schema 

For example, users can provide a custom class that extracts a field say f1.f2.f3 from input schema, applies a transform to their value and puts it in field f of output schema

That custom class takes an input record in input schema and translates to a record in output schema , and has the necessary logic to do so 

And there can be inbuilt extractors/transformers for common requirements 

so the extractors will be specific to each format?

No, we can have a format -&gt; genericRow converter in front. And the extractor can be genericRow -&gt; genericRow