Hi. In the new browsing Experience I see that Airflow Pipelines got a new dimension - some 'Default' container. But how do I create more of such containers and divide pipelines into them? I have several Airflow instances connected to Datahub, each in its own ingest recipe. Before I couldn't find a way do separete them in UI, but maybe this new functionality can help me?

@big-carpet-38439 might be able to speak to this!

hey @elegant-salesmen-99143! yes I think you can indeed use this new functionality for your use case. what's going on here is we have a new

browsePathsV2

aspect that we will use to show these folders underneath the platform - right now all of your airflow entities have a

browsePathsV2

aspect that looks like this:

{"path":[{"id":"Default"}]}

- you have a few options here. (1) I would upgrade your ingestion version to get better browse paths here and (2) I believe if you use a data-platform-instance that is different on each of your ingestion recipes (can define these on the recipe) then we should create a top level path that splits between them

@bulky-soccer-26729 (1) We are now on 10.5, do I need further upgrade to get better browse paths? (2) but Airflow connection is push-based, not pull-based, so there's no ingestione recipe in Datahub for it, but I believe there is a Datahub plugin installed on Airflow's side. I'm not very familiar with this plugin, can those paths be set there the same way?

ahh okay sorry i totally forgot airflow was different than most other ingestion sources. Let me tag @dazzling-judge-80093 our resident expert in airflow to get his take. There's a chance we may not be generating these aspects for airflow which makes sense why you'd be seeing this "Default" folder

@dazzling-judge-80093 do you have any advice on this?

We don’t generate browse path v2 from the Airflow plugin at the moment

@dazzling-judge-80093 is it in the roadmap? or cah we create them on our own using some transformer/mutation/Python SDK?

I need to look after

@dazzling-judge-80093 hi. Did you manage to check of it's possible to specify Airflow containers here?