A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Where do I find all the fields required for a recipe . I want to know what I can and cannot put in the reporting config

Hi <@U04JPNR3752>,

you can find them in the documentation. Here is the example for BigQuery: <https://datahubproject.io/docs/generated/ingestion/sources/bigquery#config-details>

Thanks <@U049WUH4155> I have seen those.. I was just wondering if there was some spec on the level above that config. i.e.

```source:
   type: bigquery
   ... &lt;- Are there other parameters I can use here
   config: &lt;- I know this part```

Ah okay, so maybe the class `PipelineConfig` is what you are looking for:
<https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/run/pipeline_config.py#L40>
This describes the whole pipeline config

If you are interested in other fields than `type` and `config` within `source`: I haven't seen any other  :thinking_face:

So it does not show up in the UI as bigquery/dbt/kafka all the time

Ah OK got it thank you <@U049WUH4155> I guess I can try platform_instance?

sure you can, but this does overwrite the type of the config in the UI

Do you want to get it rid from the click/browse path when accessing the assets or everywhere?

No I don't I want to affect any other behaviour other than being clear what ingest has run and which ones I need to pay attention to

I have multiple bigquery accounts I need to ingest and if they all show up under `bigquery` it's difficult to know which ones "at a glance" that I need to troubleshoot

Then `platforrm_instance` might be a good choice for you :slightly_smiling_face:

Yea... this is what I was thinking. Thanks Tim... you have been of great help