https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
m

Mason Mahaffey

04/14/2022, 9:13 PM
Hey there, was testing out Airbyte by ingesting data into DynamoDB and noticed the following schema for all records ingested: • `_airbyte_ab_id`: a uuid assigned by Airbyte to each event that is processed. • `_airbyte_emitted_at`: a timestamp representing when the event was pulled from the data source. • `_airbyte_data`: a json blob representing with the extracted data. • `sync_time`: a tim Is there any way to customize the record schema so that data is not nested within the
__airbyte__data
key?
m

Marcos Marx (Airbyte)

04/14/2022, 9:15 PM
not for dynamodb, some destination have the option to normalize/tabular the data like postgres and bigquery
👍 1
m

Mason Mahaffey

04/14/2022, 9:16 PM
Will this feature be added anytime soon?
m

Marcos Marx (Airbyte)

04/14/2022, 9:16 PM
I don’t think so, Airbyte uses dbt to apply normalization process and today there isn’t a package for dynamodb
👍 1
m

Mason Mahaffey

04/14/2022, 9:17 PM
gotcha, you'd be surprised by how big of a need that is at my company
appreciate the help 🙂
Can airbyte load from Postgres to Dyanmodb?
m

Marcos Marx (Airbyte)

04/14/2022, 9:39 PM
yes but data will be loaded as json into dynamodb
m

Mason Mahaffey

04/14/2022, 9:42 PM
not following, dynamo is json?
you mean w/ a load into postgres it would include the same markup? So loading into Dynamo from postgres wouldn't work?
m

Marcos Marx (Airbyte)

04/14/2022, 10:01 PM
will work, but the data from postgres will be transfer as json in
_airbyte_data
column
3 Views