Hi Good people of Datahub community. We are looking to extend Postgress Ingestion support so that DH can extract spatial metadata from Postgres (Postgis enabled) tables. Our current workflow involves first ingesting metadata for PG tables in Datahub, and then using then either programatically using Java Client API updating the extracted dataset metadata properties with spatial info that we extract from the same PG table using a python script. Alternatively we are using the REST API to update the dataset metadata. So far so good. But we would like to wrap all this up in the ingestion, so that on ingesting a PG source, both normal metadata and spatial metadata(via python extension) would be generated. Is this possible, does it make sense?

Have you taken a look at transformers ? If you are able to wrap your custom logic as a transformer, it can be used in ingestion recipe.

@hundreds-photographer-13496 thanks for the feedback. I once tried that route but it didnt work. Or let me say, I dint know how to go about it. I followed the docs, but was not successful to make it work. Any chance for a session to walk me through this?

Hi @brief-toothbrush-55766! Apologies for the delay here - we have office hours every Tuesday at 9am US PT - that’s a great spot to bring questions like this for live help! We post reminders in #office-hours 🙂