Hi, i tried to ingest data from biquery using reci...
# ingestion
f
Hi, i tried to ingest data from biquery using recipe
Copy code
source:
  type: bigquery-usage
  config:
    # Coordinates
    projects:
      -project

sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"
but it gives me just 1 dataset and only 2 tables. Dose anyone know what i am doing wrong?
also the status is Pipeline finished successfully
d
You should use
type: bigquery
instead of the usage which is only for getting usage
f
Hi Tamas, thx, i will try today 😄
i got this
Copy code
projects
  extra fields not permitted (type=value_error.extra)
for
Copy code
source:
  type: bigquery
  config:
    # Coordinates
    projects:
      - <project>


sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"
d
I think it should be project_id and not projects (https://datahubproject.io/docs/metadata-ingestion/source_docs/bigquery/)
Copy code
source:
  type: bigquery
  config:
     project_id: myproject
sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"
f
yep, i cant do it for multiple project in the same time
thx 😄
b
Were you able to get this working?