Hey team, I create a connection from Netsuite to B...
# replication-ask-ai
d
Hey team, I create a connection from Netsuite to BigQuery. But I got a error at Sync step. It's very weird. Do you have any ides for me to more investigate?
Copy code
`Unparseable date: "2015-01-21".
Copy code
2023-10-26 07:57:25 source > Stream `customer`: cannot read using date format `%m/%d/%Y
2023-10-26 07:57:25 source > Stream `customer`: retry using next date format `%Y-%m-%d
2023-10-26 07:57:26 source > Stream `customer`: cannot read using date format `%Y-%m-%d
2023-10-26 07:57:26 source > DATE FORMAT exception. Cannot read using known formats ['%m/%d/%Y', '%Y-%m-%d']
2023-10-26 07:57:26 source > Stream `customer`: INVALID_PARAMETER error occured, full error message: Invalid search query. Detailed unprocessed description follows. Search error occurred: Parse of date/time "2015-01-21" failed with date format "dd/MM/yy" in time zone America/Los_Angeles
Caused by:
	java.text.ParseException: Unparseable date: "2015-01-21".
2023-10-26 07:57:26 source > Encountered an exception while reading stream customer
Traceback (most recent call last):
k
Based on the error message you provided, it seems like the date format in your Netsuite source data is not compatible with the expected date formats in Airbyte. The error message indicates that Airbyte tried to parse the date "2015-01-21" using the formats
%m/%d/%Y
and
%Y-%m-%d
, but both attempts failed. One possible solution could be to transform the date format in your Netsuite source data to match one of the formats that Airbyte can parse. However, this solution might not be feasible if you don't have control over the format of the source data. Another possible solution could be to modify the Airbyte connector to support the date format in your source data. This would likely involve modifying the connector's code to add the new date format to the list of formats it tries when parsing dates. However, this solution would require some programming knowledge and familiarity with the Airbyte connector's code. Unfortunately, the information extracts provided do not contain specific instructions on how to implement these solutions. I would recommend checking the [Airbyte documentation](https://docs.airbyte.com) or asking for help in the [Airbyte forum](https://discuss.airbyte.io) for more detailed guidance.
d
2015-01-21 is %Y-%m-%d, but we got this issue. It's very weird
👀 1
a
same issue here... cannot start to use airbyte because of this ! (java.text.ParseException: Unparseable date: "2020-01-31".)
🙏 1
d
It is very weird, because I saw a ticket about this issue that is merged.
Hi @Ammar Khwaira, I found the solution here. It worked for me. You can try it.
Copy code
Hi there, I can confirm this behaviour, I had a setting DD/MM/YYYY then changed it to MM/DD/YYYY then ran the pipeline and it was completed successfully
agreed that the setting may be a config setting from the UI
Cheers!
Discussion to refer: https://github.com/airbytehq/airbyte/pull/17470
a
hi @Duong Le i got lost in the github comments, where exactly to set the date format in Airbyte UI ? or did they mean changing the date format in Netsuite setup? this i cannot do for sure
d
@Ammar Khwaira I try understand logic code, but it has many layer, so too difficult to debug. However, I got over it after I change DD/MM/YYYY -> MM/DD/YYYY on Netsuite UI.
You can try before you look for a new solution.
a
Thanks @Duong Le for sharing this knowledge. Appreciated. I'll try it
unfortunately , i've changed the date format to mm/dd/yyyy and it did not work, i even tried to put it as yyyy-mm-dd but did not work neither
hi @Duong Le it managed to fetch data from Customer table after multiple tries with date format, we have around 140K rows in the table. Airbyte ran for 40 minutes and only got around 8K rows ! did you face the same slow performance?
d
not yet. I haven't tried it for production yet. But, I think it support the config to do it. You can research or question to others in group.
a
Thank you
d
No problem. I will need the support from you when I deploy on Production. 😄
m
any update on this one? for sure it's not an option to change the format in netsuite, any workarounds?
a
@Marco Ciaravella since the date format only changes for the netsuite user you are using for airbyte elt, there is no harm in doing so as it will not affect any other users or scripts/saved searches. The issue for me was the connector is so slow in fetching data in which i failed to pull any of our tables , even the small ones (less than 500 K rows)
👍 1