how to change data in messages.jsonl into columns
# help-connector-development
p
how to change data in messages.jsonl into columns
k
A message from kapa.ai
Airbyte provides a process called "Normalization" to transform the raw JSON data into a relational table where each field is its own column. This is part of the ELT (Extract, Load, Transform) process that Airbyte handles. After the extraction and loading of data, the transformation step is applied, which includes normalization. Here's an example of how you can transform JSON data into columns using SQL:
Copy code
sql create table "postgres".quarantine."covid_epidemiology_f11__dbt_tmp"    as (   with __dbt__CTE__covid_epidemiology_ab1_558 as (   select    jsonb_extract_path_text(_airbyte_data, 'key') as "key",    jsonb_extract_path_text(_airbyte_data, 'date') as "date",    jsonb_extract_path_text(_airbyte_data, 'new_tested') as new_tested,    jsonb_extract_path_text(_airbyte_data, 'new_deceased') as new_deceased,    jsonb_extract_path_text(_airbyte_data, 'total_tested') as total_tested,    jsonb_extract_path_text(_airbyte_data, 'new_confirmed') as new_confirmed,    jsonb_extract_path_text(_airbyte_data, 'new_recovered') as new_recovered,    jsonb_extract_path_text(_airbyte_data, 'total_deceased') as total_deceased,    jsonb_extract_path_text(_airbyte_data, 'total_confirmed') as total_confirmed,    jsonb_extract_path_text(_airbyte_data, 'total_recovered') as total_recovered,    _airbyte_emitted_at   from "postgres".quarantine._airbyte_raw_covid_epidemiology   ), __dbt__CTE__covid_epidemiology_ab2_558 as (   select    cast("key" as     varchar   ) as "key",    cast("date" as     varchar   ) as "date",    cast(new_tested as     float   ) as new_tested,    cast(new_deceased as     float   ) as new_deceased,
This SQL script is parsing a JSON blob stored in a single column and extracting it into separated field columns as described by the JSON Schema. It then casts each column to its adequate SQL type converted from the JSON schema type. You can find more information about normalization in Airbyte [here](https://docs.airbyte.com/understanding-airbyte/basic-normalization). If you want to customize and change how the data is normalized with your own queries, you can implement your own transformer. For example, you could write them in a different language, create them in an analytics engine like Spark, or use a transformation tool such as dbt or Dataform. You can find more information about this [here](https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-sql). Sources: - [Transformations with SQL](https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-sql) - [Understanding Airbyte Basic Normalization](https://docs.airbyte.com/understanding-airbyte/basic-normalization)