Hey team I am running dbt recipe that uses both Ka...
# all-things-deploymentc
creamy-van-28626
08/07/2023, 6:38 PMHey team I am running dbt recipe that uses both Kafka and gms , I can see for most of the aspects MCP is getting created but for siblings aspect no MCP is getting created . Why all the dbt aspects not getting created via Kafka ?
What is the purpose of this sibling aspect or can we reroute via MCP?
creamy-van-28626
08/08/2023, 3:43 PMThis is the current recipe that we are using ;
We are using transformers and doing api call and also using sink as Kafka
creamy-van-28626
08/08/2023, 3:44 PMAnd with this recipe what we have observe that a mostly all aspects MCP or metadata change event is getting created but for sibling no MCP is getting created or it going through api
creamy-van-28626
08/08/2023, 3:44 PMWhy is it so ?
creamy-van-28626
08/08/2023, 6:21 PMAny update on this ? @dazzling-yak-93039
a
aloof-gpu-11378
08/08/2023, 8:58 PM@famous-waitress-64616 have you seen something like this where the siblings aspect only is not getting created? I believe this would be in ingestion?
c
creamy-van-28626
08/09/2023, 4:48 AMWe are ingesting dbt by running this recipe and I can see only metadatachangelog_versioned topic only sibling aspect is getting create. How we decide in which case (aspects) it will do synchronous ingestion (gms) and in which cases it will go through asynchronous ingestion (Kafka)?
a
aloof-gpu-11378
08/09/2023, 5:59 AM@gray-shoe-75895 do we still have the siblings bug around ingestion order?
c
creamy-van-28626
08/09/2023, 6:01 AMIs it a bug ?
creamy-van-28626
08/09/2023, 7:46 AMBut one thing I want to understand how we decide which aspect will be cover via Kafka or which one with rest/gms ?
creamy-van-28626
08/10/2023, 6:56 PM@gray-shoe-75895 and @famous-waitress-64616 any update on this
a
aloof-gpu-11378
08/14/2023, 8:52 PMSiblings are created by a hook on the gms side, not directly from ingestion
aloof-gpu-11378
08/14/2023, 8:52 PMBut one thing I want to understand how we decide which aspect will be cover via Kafka or which one with rest/gms ?Bit confused by this question. You can use either the rest or the kafka sink in ingestion and things will work fine
c
creamy-van-28626
08/15/2023, 6:04 AMI am using this dbt recipe and giving sink as datahub Kafka and I can see only siblings aspect getting created via mae consumer and for other aspects MCP is getting created