@kapa.ai Given Airbyte connector with incremental sync. How can I tell what the incremental value that is currently stored and going to be used on the next sync?

Viewing the Current Incremental Sync State You can inspect the current cursor/state value that will be used on the next sync in a couple of ways: 1. Via the Airbyte UI (Recommended) 1. Go to Connections and click the connection you want to inspect. 2. Click the Settings tab. 3. Click the Advanced dropdown arrow. 4. The Connection State section will display the current state, including the most recent cursor values for each stream. This shows the exact state that will be passed to the connector on the next incremental sync. [Manage Connection State] 2. Via the Internal Postgres Database (Self-Hosted) If you're running Airbyte self-hosted, you can query the internal database directly:

docker exec -ti airbyte-db psql -U docker -d airbyte

Then run:

SELECT * FROM state;

This returns a table with the

connection_id

, the

state

JSON (which includes the cursor field and its latest value), and timestamps for when it was created/updated. [Incremental Data Sync Tutorial] What the State Contains The state JSON typically looks like this:

{
  "state": {
    "streams": [
      {
        "cursor": "2022-09-02T07:55:58.324989",
        "stream_name": "table_one",
        "cursor_field": [
          "updated_at"
        ],
        "stream_namespace": "public"
      }
    ]
  }
}

The

cursor

value shown is what will be used as the starting point for the next incremental sync — only records with a cursor value greater than or equal to this value will be fetched. [Incremental Data Sync Tutorial] > ⚠️ Note: You can also edit the connection state via the UI to change the starting point for the next sync, but this should be done with extreme care as incorrect edits can break your syncs. [Manage Connection State] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.