A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Is there an easy (=user friendly) way to undo a soft delete via the datahub tool? We have currently the problem that a table was soft-deleted a few days ago (because it was removed from DB). Now the table is back on the DB but won't show up in DataHub after scanning/ingesting this DB. Probably because the soft delete flag is still set and the ingestion process does not remove it?
(DataHub 0.8.43, stateful ingestion)

<@UV0M2EB8Q>
The ingestion recepie we are using is listed below.
Adding and removing tables from the table_pattern.allow section has the same effect.
Tables not listed there get soft-deleted in DataHub (this one was also unexpected but seems logical in hindsight) but if you add them later again to the table_pattern.allow section the tables remain soft-deleted.

```source:
  type: trino
  config:
    # Coordinates
    host_port: ${STARBURST_BASE_URL}:443
    database: deltalake
    schema_pattern:
      allow:
        - dwh
    table_pattern:
      allow:
        - ".*_data01$"
        - ".*_data07$"
        - ".*_data08$"
    env: ${DH_STARBURST_DATA_ENV}

     # Credentials
    username: ${STARBURST_USER}
    password: ${STARBURST_PASSWORD}

    stateful_ingestion:
      enabled: True

    profiling:
      enabled: True

    domain:
      "urn:li:domain:dwh-demo":
        allow:
        - ".*"

pipeline_name: "my_pipeline_test_1"

sink:
  type: "datahub-rest"
  config:
    server: "${DATAHUB_BASE_URL}/api/gms"
    token: ${DATAHUB_ACCESS_TOKEN}```
