https://linen.dev logo
Join Slack
Powered by
Hi <@U04SE163WC8>, do you know of any examples of ...
# replication-ask-ai
s
Hi @kapa.ai, do you know of any examples of connectors that convert integers representing epoch time to a timestamp?
k
Yes, there is an example of a connector that handles UNIX timestamps, which are integers representing epoch time. The Delighted connector is a good example of this. Here's the relevant configuration for the DeclarativeStream from the Delighted connector:
Copy code
yaml incremental_sync:   type: DatetimeBasedCursor   cursor_field: "created"   datetime_format: "%s"   cursor_granularity: "PT1S"   start_datetime:     datetime: "{{ config['start_date'] }}"     datetime_format: "%Y-%m-%d %H:%M:%S"   end_datetime:     datetime: "{{ today_utc() }}"     datetime_format: "%Y-%m-%d"   step: "P1W"   start_time_option:     field_name: "created_after"     inject_into: "request_parameter"   end_time_option:     field_name: "created_before"     inject_into: "request_parameter"
In this configuration, the 
datetime_format: "%s"
line is used to interpret the UNIX timestamp. You can find more details about this in the [forum post](https://discuss.airbyte.io/t/3921) where this was discussed.
s
Hmm @kapa.ai, that's not exactly what I was looking for. Essentially, I would like to convert integer values from my source in to datetime values during a sync. Is there a way to do that?
k
A message from kapa.ai
Based on the provided extracts, there isn't a direct example of a connector that converts integer values representing epoch time to datetime values during a sync. However, the data type mapping for the Zuora and Amazon Seller Partner sources show that 
integer
and 
timestamp
types are mapped to 
number
and 
string
types respectively in Airbyte. In the case of the Zuora source: - 
integer
is mapped to 
number
- 
timestamp
is mapped to 
number
(which is a number representation of the UNIX timestamp) For the Amazon Seller Partner source: - 
int
is mapped to 
number
- 
datetime
is mapped to 
datetime
While these mappings don't directly convert integers to datetime, they do provide a way to handle timestamps and integers in your data. If you need to convert these 
number
types (UNIX timestamps) to datetime in your destination, you would typically handle this transformation in your data warehouse or data lake after the data has been synced. For more information, you can refer to the following documentation: - [Zuora Source](https://docs.airbyte.com/integrations/sources/zuora) - [Amazon Seller Partner Source](https://docs.airbyte.com/integrations/sources/amazon-seller-partner)
s
@kapa.ai Is there no example of converting an EPOCH string to a one of the airbyte timestamp formats?
k
A message from kapa.ai
I'm sorry, but based on the provided extracts, there isn't a specific example or guide on converting an EPOCH string to one of the Airbyte timestamp formats during a sync. In general, data transformation like this is typically handled in the destination data warehouse or data lake after the data has been synced by Airbyte. This is because Airbyte's primary function is to ensure accurate and efficient data replication from source to destination, and it doesn't natively support data transformation during the sync process. For more information on how to handle data transformation in your specific data warehouse or data lake, you might want to refer to their respective documentation or user guides. If you have further questions or need more assistance, you can also reach out to the [Airbyte community](https://github.com/airbytehq/airbyte/discussions) for help.
2 Views
Open in Slack
PreviousNext
Powered by