A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

image.png

<@U0348BYAS56> yes, and there it shows the workspace_id_as_urn_part , so wondering why it does not work

Hi <@U04JEARDD6D>
I am on `acryl-datahub, version 0.10.2.3` . It is working fine at my end.

<@U04JEARDD6D> which version are you using?

Sure, I'll try, will get back once tested

hmm, that seem to have messed up my Python env. now getting: Command failed: Failed to find a registered source for type powerbi: mutable default &lt;class 'datahub.metadata.schema_classes.NullTypeClass'&gt; for field datahubDataType is not allowed: use default_factory

wiped my venv, reinstalled, but same issue, guessing it must be something else, e.g. Python related. <@U0348BYAS56>which version of Python are you running?

Hm... Curious, do you have correct indentation with your yaml?

<@U030H0S9QUV> yes, if I remove that one entry it works (well not know since my env is messed up somehow), but before that it did. Also tried to put the entry elsewhere in the yaml, same result. Once I get my env. working again I'll test some more

If you can paste your recipe again in code quote format or pastebin, it will be easier to see whether it is a malformed format or something else

`pipeline_name: powerbi_test`
`source:`
  `type: "powerbi"`
  `config:`
    `# Your Power BI tenant identifier`
    `tenant_id:` 
    `#leita`
    `#tenant_id:` 
    `platform_instance: test`
    `env: test`

    `# Azure AD Application identifier`
    `client_id:` 
    `#leita`
    `#client_id:` 
    
    `# Azure AD App client secret`
    `client_secret:` 
    `#leita`
    `#client_secret:` 
    
    `# Ingest elements of below PowerBi Workspace into Datahub`
    `workspace_id_pattern:`
      `allow:` 
        `- 8d85743b-432e-4c87-9ec8-9ec08f94bec0`
        `- 63c88901-7861-4736-8144-044cf179d296`
      `# deny:`
    `dataset_type_mapping:`
        `PostgreSql: postgres`
        `Oracle: oracle`
        `Sql: mssql`

    `#ingest using admin api`
    `admin_apis_only: true`

    `# Enable / Disable ingestion of ownership information for dashboards`
    `extract_ownership: true`

    `# Enable / Disable ingestion of lineage information`
    `extract_lineage: true`
   
    `# Enable/Disable extracting workspace information to DataHub containers`
    `extract_workspaces_to_containers: true`
    
    `# Enable / Disable ingestion of endorsements.`
    `# Please notice that this may overwrite any existing tags defined to ingested entities!`
    `extract_endorsements_to_tags: false`
    
    `workspace_id_as_urn_part: true`

Okay, so I rolled back to 0.10.2 and now it works

0.10.2.3 did not work and when I saw the error originally posted I think I might have been on an older version, but for now all is good and I can continue investigating why the lineage does not get extracted (another issue).

back to not working after running once successfully, so it must be my Python setup, will do a complete reset.....

There are some issue with 10.2, might as well upgrade to 10.3

I'll get our setup upgraded to 10.3 and test and see :slightly_smiling_face: thanks for the assistance so far.....