@kapa.ai what's the best sync mode in OSS?

The "best" sync mode in Airbyte's open-source software (OSS) depends on your specific use case and the capabilities of your source and destination connectors. Here are the sync modes available: 1. Incremental: This mode reads records added to the source since the last sync job. It's generally supported by all connectors whose data source allows extracting records incrementally. There are two methods for this: using a cursor or using change data capture (CDC). However, CDC is only supported by some sources. 2. Full Refresh: This mode reads everything in the source. For the destination connector, it can write data in the following ways: 1. Overwrite: This mode overwrites by replacing pre-existing data in the destination. 2. Append: This mode writes by adding data to existing tables in the destination. 3. Append Deduped: This mode writes by first adding data to existing tables in the destination to keep a history of changes. The final table is produced by de-duplicating the intermediate ones using a primary key. 4. Overwrite Deduped: This mode overwrites by replacing pre-existing data in the destination and deduplicates the final data using a primary key. A sync mode is a combination of a source and destination mode together. The UI exposes the following options, whenever both source and destination connectors are capable to support it for the corresponding stream. For more details, you can refer to the [Airbyte documentation on sync modes](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes).

@kapa.ai why some streams don't have Incremental sync?

A message from kapa.ai

Not all streams support Incremental sync due to the nature of the data source or the specific implementation of the connector. Here are a few reasons: 1. Lack of a suitable cursor field: Incremental sync relies on a cursor field (a field that can be used to identify new or updated records). If a stream doesn't have a suitable cursor field, it can't support Incremental sync. For example, if modifications to the underlying records are made without properly updating the cursor field, then the updated records won't be picked up by the Incremental sync as expected ([source](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/incremental-append#known-limitations)). 2. Lack of a primary key: For Incremental - Append Deduped sync, a primary key is required to identify and deduplicate records. If a stream doesn't have a primary key, it can't support this sync mode ([source](https://github.com/airbytehq/airbyte/issues/38659)). 3. Connector limitations: The specific implementation of a connector might not support Incremental sync for all streams. For example, in the case of the Stripe connector, the

invoice_line_items

stream doesn't support Incremental sync even though its parent stream

invoices

does ([source](https://github.com/airbytehq/airbyte/issues/30073)). 4. API limitations: The API of the data source might not support retrieving records incrementally. For example, the Snowflake-native-apps for LinkedIn Ads and Facebook Marketing only support Full Refresh sync for all streams ([source 1](https://docs.airbyte.com/snowflake-native-apps/linkedin-ads#supported-streams), [source 2](https://docs.airbyte.com/snowflake-native-apps/facebook-marketing#supported-streams)). In such cases, Full Refresh sync is typically used as a fallback. However, this can be less efficient as it requires re-fetching all data each time.

@kapa.ai does airbyte OSS has api healtcheck endpoint by default?

A message from kapa.ai

Yes, Airbyte's open-source software (OSS) does include a health check API endpoint by default. The endpoint is

GET /health

. This operation does not require authentication and it's used to check the health status of the Airbyte instance. A successful operation returns a 200 OK status. Here is the detailed information about the health check API: - Method:

GET

- Path:

/health

- Tags: health - Responses: 200 OK for successful operation - Authentication: This operation does not require authentication. You can find more details about this in the [Airbyte Public API documentation](https://reference.airbyte.com/reference/start#airbyte-public-api---health).