Platform instance feature is not implemented for t...
# ingestion
w
Platform instance feature is not implemented for the bigquery connector https://datahubproject.io/docs/generated/ingestion/sources/bigquery#module-bigquery
BigQuery doesn’t need platform instances because project ids in BigQuery are globally unique.
While the feature is not required in terms of uniqueness (project id is already included in the URN), setting the
DataPlatformInstace
aspect with the project id would enable the use of the project id for the platform instance faceting (as a filter in the searches). WDYT about this? I could do a PR if some agreement on this.
c
@witty-butcher-82399 I will check on this and get back
w
I have noted that project id is also ingested as a container and so it can be filtered with the container faceting. So I guess the discussion becomes more existencial, what’s more natural for a project id, to be represented as a data platform instance or a container? 😅
c
@mammoth-bear-12532 Please take a look
m
Hey Sergio, our current thinking on this is that
project-id
should be used as a filter provided by the container entity.
Essentially for all warehouses -> you get filtering on
database
/ `schema`/ ... using the containers
for BigQuery this would map to `project-id`/`dataset`/
do people usually think about the "project-id" as an independent installation of bigquery?
w
Hard to know what people usually think 🙂 I initially raised the question on setting the platform instance aspect for bigquery because I wanted to have ability to filter by project id. I can do that with the container faceting. So 👌 Whether project id should be a platform instance or a container. I have no strong opinion, so the current criteria is good to me. Actually, being BigQuery serverless, the “instance” concept may not apply.
m
Actually, being BigQuery serverless, the “instance” concept may not apply.
This is our current thinking on this as well.. but of course strong opinions loosely held 🙂