A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hello, I'm working and creating a custom Kafka emitter (Python) , please someone here could you please provide or share one example where I can construct MySQL object ( columns, schema, etc..) please
# Construct a MetadataChangeProposalWrapper object.
metadata_event = MetadataChangeProposalWrapper(
    entityType="dataset",
    changeType=ChangeTypeClass.UPSERT,
    entityUrn=builder.make_dataset_urn("mysql", "emmiter_project.source.user-compras", "DEV"),
    aspectName="datasetProperties",
    aspect=dataset_properties,

Hi <@U02QK58SYKC>, I'll get back to you soon.

Hi <@U02QK58SYKC>,   you can set `aspectName=schemaMetadata`, and populate the SchemaMetadata aspect following this <https://github.com/linkedin/datahub/blob/a56f0661d5dfc31d1fc9393df94aa0ff2ee199ec/metadata-ingestion/src/datahub/ingestion/source/sql/sql_common.py#L333|example> and pass it as the aspect value. The entityType, changeType and entityUrn look good.

Hi <@U028L1V9BE1>, tks a lot and I will see and test again.  Best

fabiocastro_kafka_emitter.py

Captura de tela de 2022-01-10 16-27-13.png

Hi <@U028L1V9BE1>, may you please help me again, because I stop and didn`t know about -  I did what you said but I didn`t have idea about pass schema values into this kafka emitter - could you provide me a litte example please where possible pass sql schema columms into datatub  to use in image below
my py code

You are along the right track. You just need to instantiate the schema_metadata aspect object. Populating the `fields` member should display the columns under the `schema` tab in the image that you have shared. Please follow the example <https://github.com/linkedin/datahub/blob/a56f0661d5dfc31d1fc9393df94aa0ff2ee199ec/metadata-ingestion/src/datahub/ingestion/source/sql/sql_common.py#L333>.

Hi <@U028L1V9BE1>, may you help me again please?  I did your recommendation like below but experience issue yet  - I will appreciate
# Construcao dataset propriedades do objeto
schema_metadata = SchemaMetadata(
    schemaName="dataset_name",
    platform="sql"
    version="0",
    hash="",
    platformSchema=MySqlDDL(tableSchema=""),
    fields="idproduto",
)


# Construcao do MetadataChangeProposalWrapper objeto
metadata_event = MetadataChangeProposalWrapper(
    entityType="dataset",
    changeType=ChangeTypeClass.UPSERT,
    entityUrn=builder.make_dataset_urn("mysql", "fabio-mysql.fabio-dataset.user-table"),
    aspectName="schemaMetadata",
    aspect=schema_metadata,
)