Hello Everyone, Just wanted to check is there any way to disable automated lineages created in Redshift in DataHub. Thanks!

Yes, table lineage, view lineage and column-level lineage are all configurable within the provided redshift integration: https://datahubproject.io/docs/generated/ingestion/sources/redshift/#config-details

Hello @acoustic-lawyer-52945, Thanks for the reply. Could you please highlight which specific option will help me in disabling Redshift automatic lineage created with views and how can we disable it. I have tried to exclude "include_table_lineage" option but its still creating the automatic lineage. Also have tried all the below 3 lineage collector types :- stl_scan_based sql_based mixed But its still not working. Also I have tried to exclude "include_view_lineage" option in YAML recipe as below but its still not working. I am getting the error as "Failed to configure Redshift. Extra fields not permitted "include view lineage". Kindly suggest if there is any specific thing which is required to be performed.

Extra fields not permitted "include view lineage".

- I usually get this error when I accidentally add a field outside of the

config

section. After clearing your current redshift data with the CLI, Try the following:

source:
    type: redshift
    config:
        include_table_lineage: false
        include_table_location_lineage: false
        extract_column_level_lineage: false
        include_unload_lineage: false
        include_view_lineage: false
        include_view_column_lineage: false
        include_copy_lineage: false
        {...}

Hello @acoustic-lawyer-52945, Thanks for the help, I have tried to delete the complete Redshift data and included the above recipe in the config section and then run the ingestion still I am getting the below error.

Failed to configure the source (redshift): 3 validation errors for RedshiftConfig
extract_column_level_lineage
  extra fields not permitted (type=value_error.extra)
include_view_column_lineage
  extra fields not permitted (type=value_error.extra)
include_view_lineage
  extra fields not permitted (type=value_error.extra)

hmm, the docs may need an update then. With those removed, does the view lineage still render? To me, the code reads like having include_table_lineage and include_copy_lineage set to false will skip all lineage, even views: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/redshift/redshift.py#L420C61-L420C81

Thank you so much @acoustic-lawyer-52945! This solution worked for me.

Nice! Glad we were able to figure this one out 👍