hi experts, How can i ingest an updated table? Such as: I origin ingest with this, and I update “description”: null to “description”: “a new description” and set “version”: 1 to “version”: 2, but it’s not working. The description still empty. Any thing I am miss? Thanks for your help.

[
{
    "auditHeader": null,
    "proposedSnapshot": {
        "com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot": {
            "urn": "urn:li:dataset:(urn:li:dataPlatform:mysql,abtest.abtest.abtestv3_allocation,PROD)",
            "aspects": [
                {
                    "com.linkedin.pegasus2avro.schema.SchemaMetadata": {
                        "schemaName": "abtest.abtest.abtestv3_allocation",
                        "platform": "urn:li:dataPlatform:mysql",
                        "version": 1,
                        "created": {
                            "time": 0,
                            "actor": "urn:li:corpuser:unknown",
                            "impersonator": null
                        },
                        "lastModified": {
                            "time": 1641798176000,
                            "actor": "urn:li:corpuser:unknown",
                            "impersonator": null
                        },
                        "deleted": null,
                        "dataset": null,
                        "cluster": null,
                        "hash": "",
                        "platformSchema": {
                            "com.linkedin.pegasus2avro.schema.MySqlDDL": {
                                "tableSchema": ""
                            }
                        },
                        "fields": [
                            {
                                "fieldPath": "id",
                                "jsonPath": null,
                                "nullable": false,
                                "description": null,
                                "type": {
                                    "type": {
                                        "com.linkedin.pegasus2avro.schema.NumberType": {}
                                    }
                                },
                                "nativeDataType": "INTEGER(display_width=11)",
                                "recursive": false,
                                "globalTags": null,
                                "glossaryTerms": null
                            }
                        ],
                        "primaryKeys": null,
                        "foreignKeysSpecs": null
                    }
                }
            ]
        }
    },
    "proposedDelta": null,
    "systemMetadata": {
        "lastObserved": 1629696884482,
        "runId": "d2584674-03d3-11ec-8de4-9ae590158f91",
        "properties": null
    }
}
]

Hi @salmon-rose-54694, seems like you are using the mysql source. So when a table gets updated you need to run the ingestion again and you should see the new metadata reflected. Are you saying that's not working? Could you please paste your recipe file?

Hi @salmon-rose-54694! Gentle reminder to please post large blocks of code in message threads - it’s super helpful for us to manage all of the open/unaddressed community questions across all of our channels! teamwork

@miniature-tiger-96062 Thanks for your help. Sorry I didn’t use mysql direct pull, instead I pull tables from mysql to file, and then update some properties manually and ingest to datahub as file format. I did this way because i need to understand how it works and then i can do some extra work. my question is: I update the version in json file and suppose table should update, is that correct? Or other extra work need to do?