Hi team, I am working on a design where I wanted to fetch the changes happening at the source side and put it the datahub without re-running the recipe file. Traditionally what we have in the datahub is like we have to re-run our recipe to get the changes from the source and it gets reflected on the datahub in a timeseries way. But can we have something where we do not require to re-run our recipe to get the changes to the datahub. Current Scene: We are fetching pull-based metadata from source ( we need to re-run the recipe ) and reflecting to the datahub UI. Required Scene: Data Source should be able to send the changes to the Datahub, whenever there is a change in their source. It's like *push-based ingestio*n from source to the Datahub. Any thoughts and inputs are welcome.

hey there! yeah this is a great question. right now, most of our out of the box ingestion sources only support pull-based ingestion. however, we do support push-based integration as a concept for the other sources, but you'll have to do some work on your end to set that up. the ones we support right away are Airflow, Spark, Great Expectations, and Protobug Schemas. here's some documentation on push-based ingestion: https://datahubproject.io/docs/metadata-ingestion/#push-based-integration https://datahubproject.io/docs/architecture/metadata-ingestion/#push-based-integration