user
09/24/2024, 1:04 PM_airbyte_meta.errrors for large records or mistyped data. Read more in our </airbytehq/airbyte/blob/v1.0.0/using-airbyte/core-concepts/typing-deduping#_airbyte_meta-errors|typing & deduping documentation>.
• Large initial CDC syncs are gracefully handled by consuming WAL and acknowledges logs periodically to ensure disk space is freed in a timely manner.
• </airbytehq/airbyte/blob/v1.0.0/understanding-airbyte/jobs#workloads|Workloads> provide a more scalable and reliable architecture to run sync jobs by separating scheduling and orchestration from data movement tasks. This improvement unlocks more automated management of workloads by managing job spikes and enables horizonal scaling (for Cloud and Enterprise users).
Tackling the long-tail of connectors
• We launched our connector Marketplace for our community-maintained connectors. Automated testing ensures our connectors stay high quality, and most connectors have been migrated to low/no-code.
• The Connector Builder seamlessly switches betweem UI and YAML modes, supports custom components, and contains extensive testing and error handling.
• AI Assist now builds connectors from scratch for you with just a link to the API docs.
• Anyone can now contribute new connectors directly from the builder to add a new connector directly to Airbyte's Marketplace. Additionally, connectors can be forked directly from the UI for faster edits, ensuring our connectors stay up-to-date.
• Airbyte supports writing to several vector store destinations, including pgVector, Pinecone, and Snowflake Cortex.
Self-Managed Enterprise
Self-Managed Enterprise extends on Airbyte 1.0 by introducing new classes of functionality: multitenancy and access management, enterprise source connectors , sensitive data masking and enterprise support – all while data never leaves your infrastructure.
• Support for Okta and OpenID Connect (OIDC) SSO
• Availability of </airbytehq/airbyte/blob/v1.0.0/access-management/rbac|Role-based Access Control> and multitenancy
• Columns can now be hashed to protect sensitive data
Announcements
• To ensure adherence to security best practices, Airbyte is migrating all connectors to non-root versions. It is highly recommended that you upgrade your platform version to v0.63.9 or later before October 2024 to ensure your syncs continue to succeed.
• As we prepare to deprecate Docker Compose, we published a </airbytehq/airbyte/blob/v1.0.0/using-airbyte/getting-started/oss-quickstart#migrating-from-docker-compose-optional|migration guide> for those migrating from Docker Compose to abctl.
airbytehq/airbyte