I have a GCS bucket that I want to replicate to BigQuery. All files are in .jsonl.gz . The problem is that the connection creates a table with the following columns :

_ab_source_file_last_modified

,

_ab_source_file_url

,

data

. This prevents from deduplicating for example. How can I fix it and have the actual data as columns ?

The behavior you're seeing — where data lands in a single

data

column rather than as individual columns — is expected for JSONL files synced from GCS to BigQuery. The GCS source connector wraps the raw JSON in a

data

field along with metadata columns like

_ab_source_file_last_modified

and

_ab_source_file_url

. To get the actual fields as individual columns, you have a couple of options based on the knowledge sources: Option 1: Provide a User Schema on the GCS Source You can define an explicit schema in the GCS source connector configuration. This tells Airbyte which columns to expect and their data types. The schema must be provided as valid JSON in the format

{"column": "datatype"}

. For example:

{
  "id": "integer",
  "location": "string",
  "longitude": "number",
  "latitude": "number"
}

Supported data types are:

string

,

number

,

integer

,

object

,

array

,

boolean

,

null

. Without a provided schema, columns and datatypes are inferred from the first file matching your path pattern. [GCS User Schema] Option 2: Check the JSONL "Schemaless" Setting The GCS source connector has a Schemaless option for JSONL format: "When enabled, syncs will not validate or structure records against the stream's schema." [GCS File Format Settings] Make sure this is disabled if you want structured columns. Important caveat: The knowledge sources do not explicitly confirm that providing a user schema will fully resolve the deduplication issue or guarantee that all fields appear as top-level columns in BigQuery for JSONL files. If after providing a schema the data still lands in a

data

blob, you may need to apply transformations on the BigQuery side, as the GCS-to-BigQuery pipeline for file-based sources has inherent limitations around schema inference and column flattening. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.