Hi guys. I need advice on creating lineage relatio...
# advice-metadata-modeling
c
Hi guys. I need advice on creating lineage relations between different sources as in the example. My looker looks like this in the DataHub (screenshot left), but I would like to have this one as in the example (screenshot right). I don’t understand how I can do it. For Looker I use LookML and Looker ingestions
a
Hi @chilly-engineer-73807, could you share your configuration? This is for column-level lineage correct?
c
LookML:
Copy code
source:
    type: lookml
    config:
        parse_table_names_from_sql: true
        stateful_ingestion:
            enabled: true
        api:
            client_secret: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_SECRET}'
            client_id: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_ID}'
            base_url: ''
        github_info:
            deploy_key: ""
            repo: ""
Looker:
Copy code
source:
    type: looker
    config:
        base_url: ''
        client_secret: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_SECRET}'
        client_id: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_ID}'
        extract_column_level_lineage: true
        extract_usage_history: true
        extract_owners: true
pipeline_name: 'urn:li:dataHubIngestionSource:ff31a294-9fec-4829-af19-fc5525951321'
a
CC: @bulky-soccer-26729
c
And the main task for me now - is to get lineage between Looker and Snowflake. As I understand, I need to use the common pipeline name in both recipe files and extract columns in both recipes. I did it: Snowflake:
Copy code
source:
    type: snowflake
    config:
        account_id: ''
        warehouse: EXO_ANALYST_MD
        username: '${EXO_DEVELOPMENT_SNOWFLAKE_RW_USER}'
        password: '${EXO_DEVELOPMENT_SNOWFLAKE_RW_PASSWORD}'
        role: ''
        include_column_lineage: true
pipeline_name: 'urn:li:dataHubIngestionSource:ff31a294-9fec-4829-af19-fc5525951321'
Looker:
Copy code
source:
    type: looker
    config:
        base_url: ''
        client_secret: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_SECRET}'
        client_id: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_ID}'
        extract_column_level_lineage: true
        extract_usage_history: true
        extract_owners: true
pipeline_name: 'urn:li:dataHubIngestionSource:ff31a294-9fec-4829-af19-fc5525951321'
LookML:
Copy code
source:
    type: lookml
    config:
        parse_table_names_from_sql: true
        stateful_ingestion:
            enabled: true
        api:
            client_secret: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_SECRET}'
            client_id: '${EXO_DEVELOPMENT_LOOKER_R_CLIENT_ID}'
            base_url: ''
        github_info:
            ""
            repo: ""
        extract_column_level_lineage: true
pipeline_name: 'urn:li:dataHubIngestionSource:ff31a294-9fec-4829-af19-fc5525951321'
But now I don’t see the lineages between Looker and Snowflake ((( I read something about “schema mapping” for connecting between them, but don’t understand where I can turn on this parameter…