Hi! I’m doing some tests with the profiling in the redshift connector and I found the following: filtering out (deny) some tables in the

table_pattern

does not prevent them to be profiled, unless they are also filtered out (deny) in the

profile_pattern

too. Is that the expected behaviour? Shouldn’t denied tables for ingestion also be denied for profiling? If I’m not wrong, this is noted here https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/sql/sql_common.py#L1089-L1092 where table pattern is not included. Thanks!

I think you are right and we should not profile ignored tables. Let me ask internally if there was any reason to work the other way.