Hi! I've been debugging my dataHub deployment on GKE for a few days, and I noticed that most (if not all) of the GMS queries are going straight to the database. I suppose some of them should be done in elasticsearch. Does anyone have an idea what it could be?

Hello Patrick, What sort of queries do you expect to see done by elasticsearch? We use elastic for search and graph-based queries. For example if you use the search bar in DataHub, elastic search will be used. If you want on the other hand to get details regarding a given entity, then we use the database. That's where the metadata model is stored.

Hello @incalculable-ocean-74010, thanks for your response! What I'm worried about is the poor performance that my DataHub deployment has been showing. About 11 seconds to load the information in the frontend. I don't see this same delay in the demo version of the datahub, for example.

What version of DataHub are you running?

Also, I have come across a series of errors in the search and these same errors appear with status code 200 coming from graphql. This makes it difficult to track problems. Even these same errors are not being caught in the monitoring done by Prometheus.

We are actively working on improvement monitoring. It should be coming out in the next release:

0.9

cc @microscopic-australia-66302

That sounds great! It would be incredible if had some documentation about the metrics exposed by OpenTelemetry. I'm building my own Grafana dashboard with distinct sources and recently I enabled the GMS monitoring with Prometheus.