I'm using the File source now, and trying to get a...
# replication-troubleshootingg
god830
01/18/2023, 6:56 PMI'm using the File source now, and trying to get a gzip file. Is the Reader Options correct?
🆘 1
s
Sunny Hashmi (Airbyte)
01/18/2023, 9:28 PMHey, can you share the full log?
From what I can see, it looks like Airbyte is unable to parse the source file as it is not in CSV format
g
god830
01/18/2023, 9:29 PMWhere can I get the log if a connection failed? Can I pm you the URL I'm using? it contains the API key so I don't want to share it public
god830
01/18/2023, 9:39 PMThis is the full error
s
Sunny Hashmi (Airbyte)
01/18/2023, 9:41 PMIt's ok, please do not share your API key! Thanks very much for attaching the full log 🙂
Since the connection is authenticated, the error may mean that the source is sending Airbyte an authentication error which is not in csv format. Could you check that your api key is valid? Are you able to extract the file from this URL outside of airbyte, i.e. can you
wget <url>g
god830
01/18/2023, 9:42 PMI can download the file by using the URL on browser
s
Sunny Hashmi (Airbyte)
01/18/2023, 9:48 PMIf it is confirmed a gzip compressed CSV, then this should work. if so can you submit a bug report? link it here and i'll try to reproduce it
g
god830
01/18/2023, 9:53 PMHi Sunny I just PMed you the URL, can you take a quick look at it please? If it doesn't work for you, I will do a bug report
s
Sunny Hashmi (Airbyte)
01/18/2023, 10:01 PMHey, coming back to the thread
g
god830
01/18/2023, 10:01 PMI'm kinda confused, why I got the file, but you saw the error message
s
Sunny Hashmi (Airbyte)
01/18/2023, 10:02 PMTo summarize, it looks like an html webpage is returning along with the gzip file. Although the api is giving you the file you requested, this is probably what is messing up the file connector
Sunny Hashmi (Airbyte)
01/18/2023, 10:04 PMI also see both a 403 and 400 error in the browser tools. I think the 403 doesn't matter since it is on favicon.ico (browser webpage icon) but the 400 Bad Request is linked to this error message
g
god830
01/18/2023, 10:04 PMemmm...
god830
01/18/2023, 10:04 PMdid you use the full url?
god830
01/18/2023, 10:05 PMbecause the url got shrinked
god830
01/18/2023, 10:05 PMI get the error if I used the shrinked url
s
Sunny Hashmi (Airbyte)
01/18/2023, 10:06 PMI used the URL you sent, however you can check this yourself in browser dev tools > Console tab
Sunny Hashmi (Airbyte)
01/18/2023, 10:12 PMAh ok, tried the full URL and yeah it is just a csv.gz -- sorry about that!
I'm not sure how to validate this file but i'll try reproducing the issue with my own csv.gz and see if the problem is in the connector. I'll let you know the results.
g
god830
01/18/2023, 10:12 PMThanks a lot!!!
s
Sunny Hashmi (Airbyte)
01/19/2023, 1:12 AMHey @god830 I was able to reproduce the error with a gzipped csv using the https provider. Interestingly this error did not occur when extracting from Local storage, didn't try the other providers. I created a bug report here, please leave a comment and thumbs up to help with prioritization: https://github.com/airbytehq/airbyte/issues/21573
g
god830
01/19/2023, 1:31 AMThanks a lot!
11 Views