Hello Team. I am trying to ingest specific bigquer...
# ingestion
w
Hello Team. I am trying to ingest specific bigquery data but all data is being ingested. Where could I change? This is my yaml:
Copy code
source:
type: bigquery
config:
   project_id: "my-project-id"
   options:
     credentials_path : "./gcp-credential.json"
   table_pattern :
     # Allow ony one table
     allow :
       - "my_dataset.my_table"
 
sink:
type: "datahub-rest"
config:
   server: <http://localhost:8080>
h
Hey @wonderful-egg-79350 can you try to use table_pattern as shown below ?
Copy code
table_pattern :
     # Allow ony one table
     allow :
       - "my-project-id.my_dataset.my_table"
I believe datahub validates table_pattern against qualified table name formed by {project_id}.{schema}.{table_name}
b
Did this work @square-lawyer-36076?