I am trying to use Python Emitter to emit databric...
# getting-started
l
I am trying to use Python Emitter to emit databricks notebook details to Datahub. Any Examples?
c
@little-spring-72943 What kind of details you want to send? If you are looking at sending lineage information Please refer to https://github.com/datahub-project/datahub/blob/3a2fec341aa2dac2d59eb51290a4f3e5756ca092/metadata-ingestion/examples/library/lineage_job_dataflow.py You can send datasets' used in there as well. Refer to more examples here: https://github.com/datahub-project/datahub/tree/3a2fec341aa2dac2d59eb51290a4f3e5756ca092/metadata-ingestion/examples/library
l
I would like to send notebook itself to document. Using entityType="notebook" . Is it possible?
@careful-pilot-86309 - Any suggestions please?
c
You can check Notebook entity. There is list of all available entities and information about the meatdata they can hold. Please check which one suits you most.
l
Thanks, Any working example will be great as I am struggling to build metadata_event based on NotebookInfoClass
@careful-pilot-86309 - The datum NotebookInfoClass({'customProperties': {}, 'externalUrl': None, 'title': 'This is a title', 'description': None, 'changeAuditStamps': {}}) is not an example of the schema
Copy code
notebook_properties = NotebookInfoClass(title="This is a title", changeAuditStamps={},
     customProperties={})
metadata_event = MetadataChangeProposalWrapper(
    entityType="notebook",
    changeType=ChangeTypeClass.UPSERT,
    entityUrn=builder.make_dataset_urn(platform="Databricks", name="test"),
    aspectName="notebookInfo",
    aspect=notebook_properties,
)
c
You can refer to sample examples here. But there is no direct code ready for Notebookclass with me at this point. If you are stuck even with above reference let me know. I will try to get it.
l
@careful-pilot-86309 - Thanks , I tried the example, it seems we are stuck and not moving forward.. any help will be appreciated
b
I am also interested in the answere 🙂
c
@little-spring-72943 Apologies, i missed this thread. @brainy-intern-50400 Here is the example:
Copy code
notebook_urn = "urn:li:notebook:(querybook,1234)"

notebook_properties = NotebookInfoClass(title="This is a title",
                                        changeAuditStamps=ChangeAuditStampsClass())
metadata_event = MetadataChangeProposalWrapper(
    entityType="notebook",
    changeType=ChangeTypeClass.UPSERT,
    entityUrn= notebook_urn,
    aspectName="notebookInfo",
    aspect=notebook_properties,
)
l
Thanks Mugdha - This is working but cannot search the notebook added by this code
c
Can you try adding browse path aspect?
b
Thanks! Now the Api shows the Entity, but the Frontend homepage turns white, I will try to find out more
I have the following error in the GUI
Copy code
Error: Unrecognized key NOTEBOOK provided in map {}