https://linen.dev logo
Join Slack
Powered by
Hi AirByte team, I am setting up Builder, there is...
# help-connector-development
c
Hi AirByte team, I am setting up Builder, there is 1 record returned with the following structure: [{ "item": "deal", "id": 123450, "data": { "id": 123450, "update_time": "2023-06-01 012008", "user_id": 18488038, "person_id": 514665, "org_id": 200989} }, { "item": "deal", "id": 54981, "data": { "id": 54981, "update_time": "2023-06-01 052008", "user_id": 18488038, "person_id": 514665, "org_id": 200989} }] Now I want to get the maximum value of update_time in the returned record and assign it to the Cursor Field in Incremental Sync, what should I do?
k
A message from kapa.ai
To set up incremental sync with a nested cursor field like 
update_time
, you can use the "Add Field" and "Remove Field" transformations in the Airbyte Builder UI. Here's what you should do: 1. Use an "Add Field" transformation to copy the nested 
update_time
field to the top-level of the record. 2. Use a "Remove Field" transformation to remove the original 
update_time
field from the 
data
object. This will effectively move the field to the top-level of the record. After these transformations, your records should look like this:
Copy code
[{     "item": "deal",     "id": 123450,     "update_time": "2023-06-01 01:20:08",     "data": {       "id": 123450,       "user_id": 18488038,       "person_id": 514665,       "org_id": 200989} }, {     "item": "deal",     "id": 54981,     "update_time": "2023-06-01 05:20:08",     "data": {       "id": 54981,       "user_id": 18488038,       "person_id": 514665,       "org_id": 200989} }]
Now, you can set up incremental sync using the top-level 
update_time
field as the cursor field. For more information on setting up incremental sync, refer to the [Incremental Sync - Append](https://docs.airbyte.com/understanding-airbyte/connections/incremental-append/) and [Incremental Sync - Deduped History](https://docs.airbyte.com/understanding-airbyte/connections/incremental-deduped-history) pages. Keep in mind that the provided code snippets are examples and not directly from the information extracts. Make sure to adapt them to your specific use case and the Airbyte Builder UI.
j
• Use the record selector to make this two records instead (https://docs.airbyte.com/connector-development/connector-builder-ui/record-processing#record-selection) • Use an "Add field" transformation to copy the update_time field from the nested "data" object to the top level of the record, then you can use it as the cursor field in the incremental sync configuration (https://docs.airbyte.com/connector-development/connector-builder-ui/record-processing#adding-fields)
💯 1
c
thanks Joe. I found a way thanks to your help 😄
f
I would like to ask a similar question, I have a response from Iterate API, the response is nested and I want to get just the results from the response. So I suppose I should add record selector in a format 
body , results
but the UI does not allow me to insert the 
,
. Even if I use just 
body
it's not extracting the body from the response. Thank you for your help.
Copy code
{
  "status": 200,
  "body": {
    "results": [
      {
        "id": "123",
        "channel_id": "123",
        "company_id": "123",
        "archive": false,
        "braze_integration": {
          "enabled": true
        },
        "copy": {
          "answer_saved": {
            "current": "Answer saved:",
            "default": "Answer saved:"
          },
          "close_button_label": {
            "current": "Close",
            "default": "Close"
          },
          "date_invalid_label": {
            "current": "Please enter a date in the format YYYY-MM-DD",
            "default": "Please enter a date in the format YYYY-MM-DD"
          },
          "datetime_invalid_label": {
            "current": "Please enter a date and time, in the format YYYY-MM-DDTHH:MM",
            "default": "Please enter a date and time, in the format YYYY-MM-DDTHH:MM"
          },
j
You can omit the 
body
parts, that's implied
Just 
results
👍 1
f
great, thank you.
Open in Slack
PreviousNext
Powered by