A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi everyone, I have ingested parquet files into DH from GCS. I is there a cleaner way of ingesting lineage data for parquet files that have been partitioned than my example in the comments?

```lineage:
  - entity:
      name: filepath.artifacts.inventory.parquet.dc000000000000.parquet
      type: dataset
      env: PROD
      platform: gcs
    upstream:
      - entity:
          name: analytics_processed.inventory
          type: dataset
          env: PROD
          platform: bigquery
  - entity:
      name: filepath.artifacts.inventory.parquet.dc0000000000001parquet
      type: dataset
      env: PROD
      platform: gcs
    upstream:
      - entity:
          name: analytics_processed.inventory
          type: dataset
          env: PROD
          platform: bigquery```

Hi <@U058S8EAJ01>
As per doc this source doesn't support lineage <https://datahubproject.io/docs/generated/ingestion/sources/gcs>

<@UV14447EU> Could you please take a look

<@U03QF8Q1M8A> I know - I am importing the lineage

<https://datahubproject.io/docs/generated/ingestion/sources/file-based-lineage>