How does datahub deal with “column level access control” in bigquery? For that matter in other systems too.
o
orange-night-91387
01/31/2022, 11:14 PM
In what context? DataHub does not access any column data by default. For profiling we allow configuration of what fields should be profiled and if sample values should be shown:
https://youtu.be/GrS_uZhYNm0?t=2832▾
Profiling demo
c
calm-airplane-47634
02/01/2022, 5:24 AM
I wanted to know how they show up in the model. Would there be any annotations denoting that something has restrictions etc
o
orange-night-91387
02/01/2022, 5:00 PM
Right now I don't think we pull them in. It is possible to augment the BigQuery or other ingestion sources to pull extra information by customizing the ingestion Python script. Full details of what metadata is currently extracted from sources can be found here: https://datahubproject.io/docs/metadata-ingestion/source_docs/bigquery/