https://datahubproject.io logo
#ingestion
Title
# ingestion
w

wonderful-egg-79350

06/16/2022, 5:55 AM
Hello Team. I am trying to ingest specific bigquery data but all data is being ingested. Where could I change? This is my yaml:
Copy code
source:
type: bigquery
config:
   project_id: "my-project-id"
   options:
     credentials_path : "./gcp-credential.json"
   table_pattern :
     # Allow ony one table
     allow :
       - "my_dataset.my_table"
 
sink:
type: "datahub-rest"
config:
   server: <http://localhost:8080>
h

hundreds-photographer-13496

06/16/2022, 6:31 AM
Hey @wonderful-egg-79350 can you try to use table_pattern as shown below ?
Copy code
table_pattern :
     # Allow ony one table
     allow :
       - "my-project-id.my_dataset.my_table"
I believe datahub validates table_pattern against qualified table name formed by {project_id}.{schema}.{table_name}
b

big-carpet-38439

06/16/2022, 4:04 PM
Did this work @square-lawyer-36076?
2 Views