Slightly relating to the post above: I wonder if it's possible (or will be possible after the update mentioned in the thread above) to ingest into Datasets without defining the env in the .yml config. I want to have a structure in Datasets without dividing between prod, dev,...

You could edit the browsepaths as a workaround, which will change the way the datasets are arranged in the UI. Transformers that can do the job.

Hi Andrew, Do you mean to skip specifying the "env" part for datasets? If you don't define the env in yaml recipe, then by default it will be picked as "prod". We do require env to be non empty. The update from above thread is only to resolve conflicts across multiple instances by including instance information in the urns.

Yes I want to skip specifying the "env" part for datasets. Is this possible with a workaround (such as the one xL mentioned)?

my workaround only addresses how the dataset is viewed in UI. Currently by default, the datasets are grouped by env when you list the datasets. you can work around that viewing hiearchy with browsepath. Env is currently part of the dataset "key", and cannot be omitted, ie, you have to pick prod, dev etc as a env (or default to prod)

Thank you for the explanation. I only want to omit it in the UI and don't mind specifying what goes to dev, prod, etc. Is editing the browsepath like you mentioned something you edit in the datahub-web-react code or something else?

For existing datasets, u could ingest a dataset aspect that modifies the browsepath for those dataset - refer to lines 121-124 in https://github.com/linkedin/datahub/blob/master/metadata-ingestion/examples/mce_files/bootstrap_mce.json If you want to affect new ingested datasets, a transformer would work better, like this one https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/transformer/add_dataset_browse_path.py

Perfect, thank you