Hi Guys Is it possible to set env by custom string, not one of [PROD, DEV, CORP, IE] ? Because we have several departments, each has his own prod\dev kafka for instance and sometimes topics have the same names. And I wanna to ingest all envs, but divide them somehow

Newbie to datahub. But still trying to answer.. If we want more types we want to add here https://github.com/linkedin/datahub/blob/master/li-utils/src/main/pegasus/com/linkedin/common/FabricType.pdl and rebuild the custom version of gms server and use it...

yep, possible solution but don't want to fork the project and not always know all possible cases for future envs now I see a solution by adding suffix for dataPlatform

@lively-jackal-83760 : would instance specific dataset identifiers (same dataset in different kafka cluster gets a different id) be sufficient for you?

Hey @lively-jackal-83760 — we will have an update on this front during town hall… but we are working on a second version of URNs which will remove this and replace it with a data platform instance identifier (as string)

@mammoth-bear-12532 not sure I got it look we have something like this urnlidataset:(urnlidataPlatform:kafka,some_topic,DEV) it was ingested from some cluster. And I want to ingest another cluster where we have the same topic name. And I see several options: • create new dataPlatform like urnlidataPlatfrom:kafkaNew • add suffix or prefix for topic which will specify cluster • change env (now there are just 4 possible options) I choose first option

@lively-jackal-83760 Agree these are the only options right now. What we intend to do is add a new format of URN that includes a freeform "instance id" urnlidataset:(kafka,some_topic,<instance_id>)