hi guys, where i can find statistics/usage in dat...
# ingestionf
few-air-56117
02/16/2022, 2:22 PMhi guys, where i can find statistics/usage in datahub mysql?
d
dazzling-judge-80093
02/16/2022, 2:23 PMUsage is not supported on mysql.
column/table stats can be collected with profiling -> https://datahubproject.io/docs/metadata-ingestion/source_docs/sql_profiles
f
few-air-56117
02/16/2022, 2:29 PMi mean, after i insert the profiles in datahub
few-air-56117
02/16/2022, 2:30 PMthe data are stored in mysql right?
few-air-56117
02/16/2022, 3:30 PMi find the lineage and urn in datahub.metadata_aspect_v2, but i cant find the stats
d
dazzling-judge-80093
02/16/2022, 3:56 PMyou should be able to find it in Elasticsearch
b
big-carpet-38439
02/16/2022, 6:39 PMProfiles in particular are only persisted to elastic (as are all timeseries metadata)
f
few-air-56117
02/17/2022, 9:01 AMOh, got it, thx😇. Do you have any example of how can i extract some data from eleasticsearch using python? I never use elasticsearch before
😅
d
dazzling-judge-80093
02/17/2022, 9:05 AMYou can also use the graphql interface if it is better -> https://datahubproject.io/docs/graphql/objects#datasetprofile
f
few-air-56117
02/17/2022, 9:06 AMits a way to get all the tables stats using 1 graphql call?
I try a python script to gett all the tables, but i need to iterare over project, after that over datase, and after that over tables
2 Views