if we were to use airflow or cron to schedule peri...
# ingestion
b
if we were to use airflow or cron to schedule periodic ingest of tables in databases, and if there are no changes to the tables, the mysql will accumulate nearly identical rows of SchemaMetaData, (except for the timestamp which keeps updating). Whats the best practice here, to periodically delete away older versions of the aspects? can we directly delete away older versions of the aspects on the DB?
b
Yes! We are working on a fix for this right now. cc @gray-shoe-75895 We will followup once it's been published as a PR
m
This fix was merged in yesterday. I’ll cut a release today.
1
b
thanks!
m
@better-orange-49102 took a bit longer -> release is here (https://github.com/linkedin/datahub/releases/tag/v0.8.6)
👍 1
b
thanks! So does that mean that if there is an existing row with everything being the same (since audit time stamp is the same now), GMS will just ignore and not store the new row?
m
correct
🥰 1