does anyone here have some experience with

dbt

? I have seeing an issue when I try to ingest a dbt project, it causes a

DuplicateKeyException

When I disable the dbt node creation using

disable_dbt_node_creation: True

it works fine. So it must be related to the dbt specific metadata. Oddly, I can disable the dbt nodes, do the ingestion successfully, re-enable the dbt nodes, and now ingestion works without any errors The specific error I see in the stack trace is this

'Caused by: java.sql.BatchUpdateException: Duplicate entry '
"'urn:li:dataset:(urn:li:dataPlatform:snowflake,citibike_tripdata.' for key 'metadata_aspect_v2.PRIMARY'\n"

+ @mammoth-bear-12532

Hi @crooked-van-51704 we'll get back to you once we have reproduced and have a solution to this. Thanks for reporting!

if you want i can write up a github issue and create a reproduction repo, i was able to reproduce it with postgres and datahub in docker-compose

Hi Lucas. Steps to reproduce it in a github issue would be helpful. Would help team get to a solution faster

Same here. @crooked-van-51704, thanks a lot for explaining the way to fix it.

I am creating a reproduction repo now, just verifying all of the steps before i submit the issue

I am seeing this error as well when datahub was updated to v 0.8.25. I am using the rest emitter with acryl-datahub python lib. I updated this lib from 0.8.170 -> 0.8.25 but still getting the error. Has the DataHub behaviour changed by not overwriting datasets and datajobs now? Perviously it would overwrite the entities. I can’t find how to add a version to the entity if that is required now? The only version I can see is within the dataset schema which I am using. Can anyone from the datahub assist?

Hi folks : @crooked-van-51704 @silly-boots-14314: we believe this is fixed now, could you give

0.8.26.8

a whirl and let us know if it fixed it for you?

Hello @mammoth-bear-12532, what components need to be updated to 0.8.26.8. Is it the server & client?

the client if you are on

0.8.26.x

@bitter-family-50639 ^