Hi all! I’m having trouble ingesting sample queries from BigQuery. Monthly queries and Top Users are successfully shown in the UI, but “No Sample Queries” in the “Queries” tab. Does anyone know what the reason might be? I made sure the table was queried in the past day, and I have access to the Project History Query in the BigQuery GCP console. Here’s the yaml I used:

source:
  type: bigquery-usage
  config:
    env: "DEV"
    projects:
      - <project-id>
    top_n_queries: 10

sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"

Hi @agreeable-hamburger-38305, if we don't specify start_date and end_date the end_date get picked up as current time in UTC and start_day is a bay behind as default.

Hi @miniature-tiger-96062 , I ran it again with this, but still no queries shown in UI

source:
  type: bigquery-usage
  config:
    env: "DEV"
    projects:
      - <project_id>
    top_n_queries: 10
    start_time: '2021-10-01T00:00Z'
    end_time: '2021-10-05T00:00Z'

sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"

Before I try to reproduce, could I know your permission level that you have set for accessing the logs?

Please excuse the noob question: how do I check that?

@agreeable-hamburger-38305: did you get unblocked on this?

Awesome, let me give it a try

large queries were getting truncated and lost (on the query pathway, not on ingestion)