Hi Team, I receive the following warning in my ing...
# ingestion
g
Hi Team, I receive the following warning in my ingestion recipe for our Athena source: '''Global Warnings: ['env is deprecated and will be removed in a future release. Please use platform_instance instead.']''' However, I think this warning is misleading. Since env points to the 'general' environment the datasource is part of (so: urnlidataset:(urnlidataPlatform:athena,mytablename,PROD) whereas platform_instance is refering to a subset within the platform (so: urnlidataset:(urnlidataPlatform:athena,PROD.mytablename,PROD) Is this a bug or a misunderstanding on my side?
1
g
@gray-shoe-75895 Could you please clarify this
g
This is intentional. We’ve found that
env
was too limiting a concept, and platform instance is a generalization of env. While env makes sense on first glance, it’s currently implemented as an enum. Folks very quickly would start asking for “custom env” support - while we added a bunch, there’s always more e.g. for “CI” env. At the same time, folks would regularly have multiple “instances” of a platform (e.g a finance redshift instance and a marketing redshift instance) and wanted a way to distinguish between these. So, we introduced platform instance as a new top-level concept to handle both of these use cases. Platform instance names can be arbitrary strings, but we recommend using some sort of convention when naming them Docs on platform instance https://datahubproject.io/docs/platform-instances/