Hello All, I am pretty new to the data hub tool and started off with the docker installed on my laptop. I was able to import some of the metadata from trino datasource into data hub. Have couple of issues. Would appreciate if some one can help me here. 1. I turned on the profiling for trino data source but I still dont see any stats around it. The stats and lineage tabs are disabled always. 2. Is there a way to setup lineage manually between data sources?

on #2, I had set up lineage by calling DataHub API with custom codes

for 2, here are some examples: https://datahubproject.io/docs/lineage/sample_code/

Swaroop, Thanks for response. for 1, I did not see any reference for profiling in the log. it doesnt look like the profiling started for trino view. I was actually digging more on it and I tried with importing oracle(different source ) table as dataset and the profiling worked for a table. And then I tried for an oracle view, it did not work. In trino datasource, I have views as my source and thats probably why the profiling is not running I guess.

It is probably correct that profiling doesn’t work on views. I will let @dazzling-judge-80093 confirm

okay. thank you.

Views should not be profiled. Is it generate profile for other tables?

@dazzling-judge-80093 Thanks. Any reason why view should not be profiled? Our use case is such that we have glue tables on S3 and trino views on top of glue tables. The views are the ones that are exposed to business users which hides lot of complexity from the users.

The reason I think is more from performance wise. For normal tables table stats can help in count and various metrics but at views this can’t be used and these queries can be even more expensive on them.

ok. Honestly, I don't see any difference between profiling a table vs view. I understand profiling activity is expensive but it the same cost for both tables and views in my opinion. Anyway, that's fine if that's a conscious decision to ignore views .

Views can be constructed using joins across multiple tables and can involve aggregations etc . Profiling a non-materialized view can involve running much more expensive queries than single table scans

ok thanks @loud-island-88694. I get the complexity involved with views.