Does Datahub expose Ingestion related metrics? We ...
# ingestion
o
Does Datahub expose Ingestion related metrics? We want to keep an eye on how many Datasets, fields etc got ingested recently. And also as a way to look out for errors and suspicious patterns.
s
https://datahubproject.io/docs/advanced/monitoring/ This gives an overview of the monitoring that we have
o
Ok. That looks like monitoring of whether Datahub is running smoothly. I'm more interested in knowing what kind of stuff got pushed into Datahub, especially using the datahub ingest command.
s
Some telemetry is being added to OSS starting https://github.com/acryldata/datahub/releases/tag/v0.8.17.5 but I don't think there is any way currently for what you are looking for other than going to Elasticsearch or Mysql.
Can you raise a feature request for this?
m
Querying DataHub metadata table "metadata_aspect_v2" can be a workaround.
o
Thanks. I got to know that I could query the database. But hadn't looked into finding the relevant tables.