Hi everyone! I need some help. I am getting this error:

Dataset query failed with error: 400 INFORMATION_SCHEMA.PARTITIONS query attempted to read too many tables. Please add more restrictive filters. Location: EU Job ID: fb2b9691-cb6bb7

I tried to filter this query and reduce the amount of data to fetch but it looks like didn't work. Does datahub parse the entire project? I need to ingest only a specific dataset, so I add a filter and included my dataset name in allow patterns but it looks that is not working or taking it in consideration. Thank you

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ button ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? ✅ button Did you find a solution to your issue? Yes button No button

Is it on bigquery? Which version of datahub client are you using?

Yes, BigQuery... datahub 0.10.2

You can do two things, I think, 1. set

number_of_datasets_process_in_batch_if_profiling_enabled

to a lower number than 200 2. disable profiling