Hi, I just started using Airbyte. I setup AWS-Red...
# help-connector-development
s
Hi, I just started using Airbyte. I setup AWS-Redshift destination (with S3 bucket config as well) and pull data from source to this destination.. i see that some data was dumped to S3 (as an intermediary step) but no data flowed from S3 to Redshift. [12:36 PM] My question is "How to debug it"? where are the logs? I am using Airbyte OSS on local sytem
k
To debug this, you’ll mainly want to: 1. Check the sync job logs in the UI In Airbyte OSS, every sync has full logs attached. To find them: - Go to your connection in the UI. - Open the Job History tab. - For the sync you ran, click the three grey dots on the right and choose “View logs”. This opens the full-screen log viewer for that attempt. [[Browsing logs](https://docs.airbyte.com/platform/operator-guides/browsing-output-logs)] In that viewer you can: - Search for
Error
,
Exception
, or
Fail
to see if the Redshift COPY step failed. - If there were multiple attempts, repeat for each attempt or use “Download logs” from the same menu to get a combined log file. [[Browsing logs](https://docs.airbyte.com/platform/operator-guides/browsing-output-logs)] 2. Download logs for deeper inspection or sharing From the same Job History menu (three dots): - Select “Download logs” to save the full logs locally (all attempts for that job are unioned into one file). [[Browsing logs](https://docs.airbyte.com/platform/operator-guides/browsing-output-logs)] 3. Confirm Redshift destination & S3 staging config Since Redshift uses S3 as a staging area, verify in the destination settings that: [[Redshift destination](https://docs.airbyte.com/integrations/destinations/redshift)] - S3 Bucket Name / Region / Access Key / Secret Key are correct and have read/write permissions on the bucket. - The Redshift host, port, database, schema, username, password are correct. - (Optional) If you set a Bucket Path, check that the staging files are appearing under that path in S3. If S3 has data but Redshift is empty, the logs should show whether the Redshift
COPY
step failed (permissions, schema issues, etc.). 4. (Optional) Configure central log storage If you want logs also written to S3 or GCS (beyond the UI): - Set environment variables like
S3_BUCKET
,
S3_REGION
,
S3_AWS_KEY
,
S3_AWS_SECRET
(or
GCS_LOG_BUCKET
) in your Airbyte deployment. [[Configuring logging](https://docs.airbyte.com/platform/operator-guides/configuring-airbyte#logging)] This doesn’t change the job logs in the UI, but gives you another place to browse logs. If you’d like, you can paste the relevant error section from the View logs screen (especially around the Redshift destination step), and I can help interpret what’s going wrong.