Aditya Guru
05/19/2021, 3:08 AMuser
05/19/2021, 4:08 AMuser
05/19/2021, 4:09 AMuser
05/19/2021, 4:15 AMuser
05/19/2021, 4:18 AMuser
05/19/2021, 4:18 AMuser
05/19/2021, 4:19 AMuser
05/19/2021, 4:22 AMuser
05/19/2021, 4:22 AMuser
05/19/2021, 4:23 AM{ "type": "object" }
user
05/19/2021, 4:23 AMuser
05/19/2021, 4:39 AM{ "type": "object" }
. I think it might be okay for the Extract and Load phases but I guess the Transform phase needs to know more about the schema. Like even the Basic Normalization. Thoughts?user
05/19/2021, 4:51 AMuser
05/19/2021, 5:33 AMuser
05/19/2021, 5:46 AMuser
05/19/2021, 5:47 AMuser
05/19/2021, 5:47 AMuser
05/19/2021, 6:01 AMuser
05/19/2021, 6:18 AMdocker run -it --rm --volume airbyte_data:/data busybox
should get you into the right volumeuser
05/19/2021, 6:21 AM# navigate to the config directory
$ cd data/config/
DESTINATION_CONNECTION/ STANDARD_DESTINATION_DEFINITION/ STANDARD_SYNC/ STANDARD_WORKSPACE/
SOURCE_CONNECTION/ STANDARD_SOURCE_DEFINITION/ STANDARD_SYNC_SCHEDULE/
/data/config/STANDARD_SYNC $ ls
31f0495e-1251-499f-9a64-a1239a99609d.json
# look at the standard sync - I believe the json_schema field here is what you want to modify
/data/config/STANDARD_SYNC $ cat 31f0495e-1251-499f-9a64-a1239a99609d.json
{"prefix":"","sourceId":"617fd945-566d-4cc4-905b-2c2c31627cba","destinationId":"0a3cde72-0a83-479b-b1db-4024ebf9516f","connectionId":"31f0495e-1251-499f-9a64-a1239a99609d","name":"default","catalog":{"streams":[{"stream":{"name":"med_table","json_schema":{"type":"object","properties":{"date_prod":{"type":"string"},"code":{"type":"string"},"len":{"type":"string"},"kind":{"type":"string"},"title":{"type":"string"},"did":{"type":"string"}}},"supported_sync_modes":["full_refresh","incremental"],"default_cursor_field":[],"source_defined_primary_key":[["code"]],"namespace":"public"},"sync_mode":"full_refresh","cursor_field":[],"destination_sync_mode":"append","primary_key":[["code"]]}]},"status":"active"}
user
05/19/2021, 6:21 AMuser
05/20/2021, 3:21 AM