Hi all, when creating a custom connector to an API that returns a JSON with nested fields, is it possible to configure somewhere in airbyte for it to create record and repeated fields in BigQuery? It's currently putting a JSON as a string in one single column instead of using the nested capabilities of BigQuery tables.

Airbyte can represent nested data structurally, but whether you see nested / repeated fields in BigQuery (instead of a single JSON column) depends on: 1. How the source emits the schema and records, and 2. Whether normalization is enabled on the destination. From the docs, Airbyte’s internal catalog and streams are explicitly designed to support arbitrarily nested JSON schemas using

type: object

with nested

properties

– e.g. the “flights” example with

origin

and

destination

objects inside a record.[\[Nested schema\]](https://docs.airbyte.com/platform/understanding-airbyte/beginners-guide-to-catalog#nested-schema-example) BigQuery as a source also has explicit work to handle nested/RECORD and REPEATED fields, which shows that Airbyte’s model is compatible with BigQuery’s nesting semantics.[\[BigQuery changelog\]](https://docs.airbyte.com/integrations/sources/bigquery#changelog) However, the knowledge sources you provided do not contain explicit, step‑by‑step instructions for: - configuring a custom source connector so that its nested JSON is loaded into BigQuery as RECORD/REPEATED columns, nor - changing the current behavior where everything is ending up in a single

_airbyte_data

(or similar) JSON/string column.[\[Data consolidation GH issue\]](https://github.com/airbytehq/airbyte/issues/53661) There is also a forum post indicating that to change how nested data is materialized into tables/columns, users sometimes need to adjust or replace the default dbt normalization project, but it doesn’t document the exact configuration for your case.[\[Unnesting forum\]](https://discuss.airbyte.io/t/638) Given what’s in the sources: - Yes, Airbyte’s schema model can represent nested data in a way that is conceptually compatible with BigQuery nested fields. - But the provided documentation does not clearly describe how to force a custom API source’s nested JSON to be written as native RECORD/REPEATED fields in BigQuery instead of a single JSON/string column. So I’m not able to confidently give you the exact configuration steps from these documents alone. The behavior you’re seeing (consolidation into one JSON column) matches the “data consolidation” issue referenced above, and addressing it likely involves normalization configuration or custom dbt, which is only mentioned at a high level in the sources.