sparse-barista-40860
06/22/2022, 6:03 PMnano /root/datahub/metadata-ingestion/examples/demo_data/recipe.dhub.yaml
##
source:
type: bigquery
config:
# Coordinates
project_id: my_project_id
# `schema_pattern` for BQ Datasets
schema_pattern:
allow:
- finance_bq_dataset
table_pattern:
deny:
# The exact name of the table is revenue_table_name
# The reason we have this `.*` at the beginning is because the current implmenetation of table_pattern is testing
# project_id.dataset_name.table_name
# We will improve this in the future
- .*revenue_table_name
sink:
# sink configs
##
datahub ingest -c /root/datahub/metadata-ingestion/examples/demo_data/recipe.dhub.yaml
[2022-06-22 13:00:20,415] INFO {datahub.cli.ingest_cli:99} - DataHub CLI version: 0.8.38.3
1 validation error for PipelineConfig
sink
none is not an allowed value (type=type_error.none.not_allowed)
purple-tailor-57675
06/22/2022, 8:17 PMsink
attrsparse-barista-40860
06/22/2022, 9:11 PMsparse-barista-40860
06/22/2022, 9:11 PMsparse-barista-40860
06/22/2022, 9:11 PMpurple-tailor-57675
06/22/2022, 9:13 PM