i am trying to build a destination connector for S...
# announcementsa
Abhinav Vasdev
04/22/2021, 9:16 AMi am trying to build a destination connector for S3 but could not find java connector setup option in when i do npm run generate
✅ 1
Abhinav Vasdev
04/22/2021, 9:18 AMdo you have any resources where i can get a step by step guide like that of python toy connector
c
Chris (deprecated profile)
04/22/2021, 9:18 AM
have you been following one of our most popular issue here?
https://github.com/airbytehq/airbyte/issues/573
a
Abhinav Vasdev
04/22/2021, 9:19 AMnope i will look into this . thanks!
c
Chris (deprecated profile)
04/22/2021, 9:20 AM
do you have any resources where i can get a step by step guide like that of python toy connectorIt’s currently being written, you can follow here: https://github.com/airbytehq/airbyte/pull/3021 Feel free to comment to clarify it even further!!
Chris (deprecated profile)
04/22/2021, 9:22 AM
For the S3 destination, we’ve started implementing some parts as part of optimising the Redshift Destination (using a staging step to S3 before loading to Redshift)
This is being refactored and extended for more Data Warehouses here:
https://github.com/airbytehq/airbyte/issues/1958
Chris (deprecated profile)
04/22/2021, 9:22 AM
So I would presume a pure S3 destination should be coming up soon too.
You’re welcome to follow the related issues/comment/contribute there too!
Chris (deprecated profile)
04/22/2021, 9:28 AM
@Jared Rhizor (Airbyte) should be the one that would have most insight on progress here (US California timezone though)
v
Vaibhav Pal Singh
04/24/2021, 3:13 PMReally Interested in a pure s3 destination. I noticed it's not even on the roadmap yet. What exactly is holding us back from implementing a pure s3 target as well if S3 as a staging step is already being implemented for Redhift?
I believe S3 can be thought of as an append/overwrite only destination.
👍 1
j
Jared Rhizor (Airbyte)
04/25/2021, 7:47 PM
There isn’t much blocking us from releasing a simple append-only CSV/JSONL S3 destination, but there is more involved for some other formats. We’re adding staging steps for other destinations and then working on S3 syncing strategies (delta lake for example) in the near future. It’s just a matter of bandwidth/prioritization.
2 Views