@kapa.ai I am getting a

"Cursor field is not selected properly for stream: Contact. Please include the cursor field in selected fields for this stream."

error when using

incremental_append

on with a

"cursorField": [ "LastModifiedDate"],

and also tried to add it to

selectedFields

as well but that didn't work, anyone has an idea of what the fix could be?

The error message

"Cursor field is not selected properly for stream: Contact. Please include the cursor field in selected fields for this stream."

typically means that Airbyte requires the cursor field (in your case,

LastModifiedDate

) to be explicitly included in the list of

selectedFields

for the stream when using incremental sync. From the knowledge sources, there is a similar issue reported when using S3 as a source, where users encountered errors unless the cursor field was included in

selectedFields

. However, even after including the cursor field, the error persisted, suggesting there may be a bug or limitation in how Airbyte handles

selectedFields

in combination with cursor fields for certain connectors or configurations. The workaround attempted was to ensure the cursor field is both set as the

cursorField

and included in

selectedFields

, but this did not resolve the issue for the user in the S3 source scenario. The issue was escalated as a potential bug in Airbyte ([selectFields on Connector using S3 as source is failing](https://github.com/airbytehq/airbyte/issues/48053)). What you should try: - Double-check that the field name in

selectedFields

matches exactly (case-sensitive) the field used in

cursorField

. - Ensure that the field exists in the stream schema and is not nested or misspelled. - If you have already done the above and the error persists, it is possible you are encountering a similar bug or limitation as described in the S3 source issue. Next steps: - If you are certain your configuration is correct and the error still occurs, this may be a bug or unsupported scenario in Airbyte. You may want to open an issue on the Airbyte GitHub or check for existing issues related to your connector and the use of

selectedFields

with incremental sync and cursor fields. Unfortunately, the knowledge sources do not provide a definitive fix beyond these steps, and the issue may require attention from the Airbyte development team if the standard configuration does not work as expected.

@kapa.ai Right but

LastModifiedDate

is present in the request payload:

{
  "name": "import_job_id",
  "sourceId": "dh78b8e0-314z-4dey-bc14-fe1365c7b5e0",
  "destinationId": "1ad4ef1a-bceb-493y-a08e-z00766607db1",
  "dataResidency": "eu",
  "namespaceDefinition": "custom_format",
  "namespaceFormat": "1234_job_id",
  "prefix": "contacts_",
  "schedule": {
    "scheduleType": "manual"
  },
  "configurations": {
    "streams": [
      {
        "name": "Contact",
        "syncMode": "incremental_append",
        "cursorField": [
          "LastModifiedDate"
        ],
        "selectedFields": [
          {
            "fieldPath": [
              "Email"
            ]
          },
          {
            "fieldPath": [
              "Fax"
            ]
          },
          {
            "fieldPath": [
              "Phone"
            ]
          },
          {
            "fieldPath": [
              "Title"
            ]
          },
          {
            "fieldPath": [
              "FirstName"
            ]
          },
          {
            "fieldPath": [
              "LastName"
            ]
          },
          {
            "fieldPath": [
              "MobilePhone"
            ]
          },
          {
            "fieldPath": [
              "LastModifiedDate"
            ]
          }
        ]
      }
    ]
  }
}

A message from kapa.ai