This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Eli Sigal, @Kevin Han and @Adam Bloom 🔥 for the contributors that sparked the most discussion: @Gemini Keith, @Opeyemi Daniel and @Robert Put So thank you!!

airbyte-oktoberfest-with-background.png

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Benoit Fayolle, @Eli Sigal and @Giovani Freitas 🔥 for the contributors that sparked the most discussion: @Renan Rigo Calesso, @gunu and @NITESH KUMAWAT So thank you!!

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Kyryl Skobylko, @Eli Sigal and @Marcelo Pio de Castro 🔥 for the contributors that sparked the most discussion: @claudio viera, @Rocky Appiah and @Ben Pankow So thank you!!

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Marcelo Pio de Castro, @Eli Sigal and @Kyryl Skobylko 🔥 for the contributors that sparked the most discussion: @sonti srihari, @Robert Put and @Jordan Young So thank you!!

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Marcelo Pio de Castro, @Robert Put and @Kyryl Skobylko 🔥 for the contributors that sparked the most discussion: @Ryan Coordinator, @Yoan Yahemdi and @Zaza Javakhishvili So thank you!!

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Francisco Viera, @Eli Sigal and @Kyryl Skobylko 🔥 for the contributors that sparked the most discussion: @Rahul Borse, @Kevin Phan and @Vishwajeet Dabholkar So thank you!!

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Kyryl Skobylko, @Francisco Viera and @Haithem (WOOP) 🔥 for the contributors that sparked the most discussion: @Leo G, @Brian Castelli and @laila ribke So thank you!!

This week, I have a few amazing people in mind that I wanted to recognize octavia loves airbyte heart for the most helpful contributors: @Francisco Viera, @Geoff Genz and @Dusty Shapiro 🔥 for the contributors that sparked the most discussion: @Sebastian Brickel, @Berzan Yildiz and @thomas trividic So thank you!!

Deprecation of Legacy State As of Airbyte v0.62.4, we've deprecated the use of

Legacy

state for connectors, as all connectors should utilize either

Stream

or

Global

states. Your connector's syncs may be affected if it is on an older version of the CDK. We've outlined migration steps here: https://github.com/airbytehq/airbyte/discussions/39430

👋 Request for Input on Configuration API deprecation As we move towards deprecation of the configuration API, we'd like to understand what endpoints or data you are currently missing from the Airbyte API. Please share with us what you are using the config API for today that you think we should support here so that we can better understand your use case and build a roadmap for the Airbyte API. Looking forward to hearing from you!

Airbyte 0.62.2 Security Update (Link) Note: this only affects users of the Connector Builder UI Airbyte was alerted to a security vulnerability, identified as a Remote Code Execution (RCE) issue, that allows another logged in user to get access to the underlying Docker image of the connector builder after they have logged in. Sensitive information, such as credentials, could have been exposed if a user tested a new connector on a compromised instance of the connector builder. This component is used to create and test new connectors. It does not have access to any data processes. It is also important to note that an attacker must have an account on the Airbyte system and be logged in to exploit this vulnerability. Based on what we know now, there is no evidence that any secrets were shared via our platform or compromised. Upon discovering the vulnerability, Airbyte immediately acted to close the security vulnerability and released an update to our platforms. The patch has been released to OSS (v0.62.2), our Cloud environment has been patched, and we have ongoing security efforts to ensure your instance stays secure. Though this exploit is only possible if a user was logged into your instance, to ensure the security and integrity of your system, we strongly recommend that everyone upgrade to the latest version of Airbyte as soon as possible. The updated version contains a fix for this vulnerability, and upgrading will protect your systems from potential exploits. We also recommend rotating any test credentials used in the connector builder. Action Required: 1. Upgrade to the latest version: Please follow our instructions to upgrade to the latest version Airbyte Security Team

Announcement: Deprecation of Docker Compose Today, we are announcing that Docker Compose will be deprecated from future versions of Airbyte OSS after version 1.0 is released (expected in August 2024). Support for Docker Compose and running Airbyte via the

run_ab_platform.sh

script will end in August 2024. With the release of abctl, we've provided a best-in-class tool that improves the initial user experience and ensures you are set up for a production-ready instance of Airbyte from the start. To support the migration of existing Docker Compose deployments, we’ve created a migration path that we are beginning to test with external users, and looking for volunteers from the Community to help ensure our migration paths can adequately cover the ways in which users currently use Docker Compose. This beta group of testers will work directly with our Engineering team to migrate their existing Airbyte deployments to

abctl

. You can read more and sign up for the beta testing group here: https://github.com/airbytehq/airbyte/discussions/40599

octavia wave Hello Everyone <!channel>! 🤖 AI Research Our Product team is eager to hear from YOU! We’re shaping the future of our AI roadmap and need your valuable input. If you’re involved in moving data for your AI projects, we’d love to chat and find ways to make your experience even smoother! Ready to share your thoughts? Schedule a time with us directly here. ⚠️ Future Feature Deprecation • Docker Compose Deprecation (scheduled for 2 August) the team is developing a migration tool to help Docker Compose users easily transition to the new deployment method. If you’re using Docker Compose to deploy Airbyte please check the discussion and stay tune! • Config API Deprecation the team is collecting all important endpoints that are not currently enabled in the Airbyte API. If you’re using the Config API, please provide inputs here. After the deprecation, endpoints will only be implemented based on feature requests. 💻 PyAirbyte Hackathon Winners The results of the PyAirbyte Quickstarts & Feature Contribution Hackathon are here! 🥁 For the Quickstarts and Tutorials: • 🥇 1st place: avirajsingh7 • Sharing 2nd / 3rd: ombhardwajj and bala-ceg • 4th: harmaton • Sharing 5th place: btkcodedev, gupta, pangeran For the Feature Contributions: • 🥇 1st place: btkcodedev • 2nd: avirajsingh7 • Sharing 3rd place: fvgm-spec, itsxdamdam, suraj, tlazarevic, vspanxcode More information check out the blog post. For participants who tied, we will combine the prizes for the positions and divide them equally among the contributors. 🪵 Rootless Connector Images Starting with Airbyte 0.64.0 and further, Airbyte operating as root will support running connectors in non-root images with less privileges to tighten security. Before that, to run non-root connectors, you’d have to pass an environment variable to the platform to also run as a non-root user. After 1.0 launch, Airbyte will release all new connector versions (across Airbyte and Marketplace connectors) as non-root. For Airbyte Cloud users, nothing changes and they will work smoothly. For Airbyte OSS users, you will have to upgrade your cluster to 0.64. If you don’t upgrade to a newer version of Airbyte, all current versions of connectors will continue to work, but to install any updates to them, you’d need to either update all connectors, or upgrade the platform first.

octavia wave Hi <!everyone>, big news and upcoming events! 🎉 Enhanced Security and Simplified API Key Management in the Airbyte Open Source!!! The latest version of Airbyte Open Source has a Native authentication by username and password is available so that any instance of Airbyte is secure by default. Managing and obtaining API keys is now consistent across all our products. We have brought our API portal in-house and API keys can now be generated and found in the “Applications” page of our UI. Along with this, the Terraform provider and Java/Python SDKs have been updated to automatically retrieve tokens between each action, so that users don’t need to worry about fetching tokens themselves. ☎️ Community Call Tomorrow! The last time the team gathered in the office, we had the pleasure of welcoming one of our own Airbyte Champions, Justin Beasley. Justin told us the ways they have been leveraging Airbyte since 2021. If you’re interested to learn about it, register here for the Community Call happening tomorrow at 9AM PT! Do you have an interesting use case with Airbyte? We’d love to hear from you! Reach out to @[DEPRECATED] Marcos Marx and share your story! 🎈 💽 Airbyte dbt Package Hackathon The hackathon will start next week on August 8 instead of August 1 (We decided to postpone the event because the team is planning to work on a related project that will be helpful for the participants). The event goal is to build transformation using dbt to leverage existing models to be compatible with Airbyte connectors. You can check all requirements and rules on this page. For each package migrated, it will have a prize of $300 💰! If there are any questions or if you run into any roadblocks, we can help you out in #airbyte-dbt-packages channel. See you all next week! octavia loves

octavia wave 📣 Announcing Workloads: Airbyte's Next Generation Job Framework We are excited to announce a significant update to Airbyte’s infrastructure that will improve scalability and resilience - Workloads. This is automatically enabled with Airbyte Helm Chart

0.390.0

. Workloads is Airbyte's next generation Worker architecture. It is more scalable, reliable and maintainable than the previous Worker architecture. It performs particularly well in low-resource environments. Workloads is an Airbyte-internal job abstraction decoupling the number of running jobs (including those in queue), from the number of jobs that can be started. Jobs stay queued until more resources are available or canceled. This allows for better back pressure and self-healing in resource constrained environments. Careful users will notice two new Airbyte services 1) the Workload Api Server and the 2) Workload Launcher. Both services function together to enable Workloads. While we do not expect OSS users to interact with these services often, or at all, we still want to note the additional services to minimise surprises. Workloads is automatically enabled for Community users as of Airbyte Helm Chart version

0.390.0

. We recommend users upgrading from existing deployments to pause existing connections before upgrading to minimise errors. Users interested in learning more can check out this blogpost and the public docs. While we expect users to not need to interact with Workloads on a day-to-day basis, or at all, we welcome feedback/improvement on this Github Discussion. Thank you!

🌟👋 Calling all Databricks users! We need beta testers for our updated

destination-databricks

! New features include: • Per-record error handling • Typing and deduping • Refreshes • Resumable Full Fresh • All the great benefits of Destinations Version 2.0! Please note: you must be using Unity Catalog to use this connector. Learn more in the Github Discussion and comment here if you'd like to be added to our private slack channel. Airbyte OSS and Cloud users are welcome! Note: For OSS you must upgrade your Airbyte Platform version to at least 0.63.7 before upgrading to this connector version.

octavia wave Hi <!channel>! 🤖 Connector Builder AI Assist: Join the waitlist! We are excited to announce early access to Connector Builder AI Assist. With this feature, an LLM helps you build and update connectors by reading the API documentation and suggesting changes. It’s a Connector Builder copilot. See how it works here. We have to limit access as we scale up the service. Fill out this form to let us know you are interested. Preference will be given to people looking to build specific connectors and willing to contribute them back to the community. See the announcement below on how that is now even easier! The team at Airbyte is building this in collaboration with Fractional AI. We can’t wait to see what you think! ⚒️*Deploying with abctl Improvements* The team is working to make the experience using abctl the best possible to run Airbyte. The latest version includes the following improvements: • 🔑 The command

abctl local credentials

returns the email. This was a common problem for folks who migrated and didn’t remember which email they used in the legacy Docker deployment. Also the command now supports

--email

and

--password

to set email and password. • ⛳ New flag

--insecure-cookies

to anyone trying to access the instance in a non-localhost setup and didn’t configure a TLS certificate. Example: create a simple network rule to allow your private IP to access an EC2 instance. • ⛳ New flag

--low-resource

in case you don’t have the minimal requirements to run Airbyte through abctl. This can reduce performance but unlock you to use Airbyte for PoC or quick demos. We hope everyone has a smooth transition migrating from Docker Compose or deploying a fresh install using abctl. If you have any troubles, don’t hesitate to contact @[DEPRECATED] Marcos Marx 👩‍💻 You can now contribute connectors in Connector Builder! Both on Airbyte Cloud, and starting Airbyte OSS 0.63.19, you can now contribute your connectors to the Marketplace via Connector Builder! Once you’re happy with your connector, if you’d like to make it available with everyone else who is using Airbyte, you can choose to contribute it to the Marketplace! . Builder will make the pull request with your new connector, we’ll review and help get it merged. We’re aiming for under 2 weeks review and merge time, but the first connector contributions that we see are merged in just 2 days. ☎️ Next Community Call Join Joey Benamy from OncoLens to learn how they use Airbyte for seamless data integration. Discover insights on handling sensitive data, regulatory compliance, and deploying Airbyte in healthcare settings. If you’re interested to learn about it, register here for the Community Call happening on September 9th at 8:30AM PT! Do you have an interesting use case with Airbyte? We’d love to hear from you! Reach out to @[DEPRECATED] Marcos Marx and share your story! 🎈

octavia wave Hi everyone! 🚀*Airbyte 1.0 Tomorrow at 8am PT* Just a note that tomorrow we are launching Airbyte 1.0!! Our biggest launch ever 💪!! 🧑‍🎤*Reddit AMA with Airbyte Team on Wednesday 25th* We will be hosting an AMA session on Reddit (r/dataengineering) on Wednesday from 11am to 1pm PT if you would like to participate. No questions out of bound! 🏆 Hacktoberfest We have a special Hacktoberfest this year, related to what we’re announcing at the Airbyte 1.0 launch: our new Marketplace and AI Assistant. Note it down in your agenda!

octavia wave Hi everyone! <!channel> 🚀🚀*Airbyte 1.0 is out!* 🚀🚀 After 1,000+ community contributions and 150,000+ deployments, Airbyte 1.0 is now officially available! For those of us knee-deep in data pipelines, this release is a game-changer in how you can approach addressing all your current and future data movement needs. Here’s what Airbyte 1.0 includes: • 💪 Battle-tested reliability and higher performance throughput to reach 1.0 status: No more pipeline disruptions over one bad row. Every sync and connector is built to keep your data flowing smoothly. • 🪄 Connector Marketplace + AI Assist: Build or edit connectors in just a few clicks, or let our beta AI Assistant handle it for you! • 🏗️ Self-Managed Enterprise GA: Powering companies like TUI and Invesco, Airbyte 1.0 is ready for your toughest challenges. • 🤖 Empowering GenAI: With vector database support, unstructured sources, and AI integrations, we’re setting the stage for a new era of data-driven innovation. 🙏 Don’t hesitate to give your support on Reddit and HackerNews (search “Airbyte 1.0”)!! 🙏

Hey community! 👋 We’re excited to invite you to an AMA on Reddit with the Airbyte founders and engineering team! As always, your feedback is incredibly important to us, and we take it seriously. We’d love to open this space to chat with you about the future of data integration. Whether you want to discuss Airbyte’s roadmap, design, or your thoughts on data integration practices in general, we’re here for it. Let’s dive deep into all things data and explore what the future holds together! Ask away on Reddit here! https://www.reddit.com/r/dataengineering/comments/1fpb48l/ama_with_the_airbyte_founders_and_engineering_team/

Hi everyone! 🏆 2024 Hacktoberfest is Live! Prepare yourselves! This Hackathon is going to be incomparable. After the 1.0 Launch, we’re pushing the limits of the AI Assistant and UI Connector Builder. We have prepared a suggestion list with more than 150 connectors, but you’re not limited to those. Do you have an API you want to try with the Builder/AI Assistant? You’re welcome! Read the rules and guidelines to participate in the event here. Join #C027KKE4BCZ channel to ask questions or get help. 🧪*abctl Feedback Survey* We’re actively building the future roadmap and collecting feedback for abctl. Your insights will guide us in prioritizing the right features and enhancements. Share your feedback in this quick survey to help us make abctl even better!

Hello <!everyone>! This week is Halloween 🍭🎃 🌉 and the team wants to offer some delightful treats to all data sweet teeth in the community. 🏎️ Last Week to Submit Contributions to Hacktoberfest! There have been over 100 contributions so far, with more to come. There’s still time for anyone interested to participate in the event and win swag and incredible prizes. Find more information here. 🧪 State of Data & AI 2024 Survey Would you like to win a Steam Deck? No tricks, just treats, I promise! 🎃 Airbyte is running the State of Data & AI. The survey provides a vendor-agnostic, industry-wide overview of the tools you use daily as data practitioners. After the deadline, we’ll enter all survey respondents into a raffle for a chance to win one Steam Deck. Here’s the link so you don’t miss this chance => State of Data &amp; AI 2024 survey!

Hello <!everyone> octavia wave! 🏆 2024 Hacktoberfest Results: A New Record! HUGE NEWS! Our Connector Marketplace Hackathon just wrapped up with incredible results - 140+ new connectors built by our amazing community! From Segment to Vercel, Miro to WordPress, you all crushed it. Want to see who won a share of the $10K prize pool? Check out our latest blog post 🚀 [Early Access] Introducing Universal File Transfer: Raw File Copying Exciting update! We’ve launched raw file transfer capability in Airbyte v1.2+, allowing you to copy files without parsing their contents - perfect for unstructured text, non-text, and compressed files. Currently available for moving files from the SFTP Bulk and S3 sources to the S3 destination, with plans to expand to GCS, Azure Blob, Google Drive, and SharePoint. As this is in early access, we’d love your feedback on the feature, including suggestions for additional connectors and larger file support. Read more and share your feedback here. 🆘 Looking for brave open-source collaborators Airbyte integrates with orchestration tools like Airflow, Dagster, and Prefect. Recently, Airbyte unified its API across Cloud, Enterprise, and Open-Source editions by replacing the Config API with the new Airbyte API, standardizing endpoints and authentication. While the Airflow Operator is being updated, we need assistance with upgrading the Dagster and Prefect integrations. If you’re interested in contributing, please DM me for mentorship on implementing these changes. First-time contributors are welcome! 🎉*Join us at OSA Con 2024, The Open Source Analytics Conference!* The Open Source Analytics Conference 2024 is this week (Nov 19-21 Free & Online) AJ Steers from the Airbyte team will discuss how to build an AI Data Hub using Airbyte and Motherduck. Click here to add the event to your agenda! More about OSA Con here 🧪*State of Data & AI 2024 Survey* Airbyte is running the State of Data & AI. The survey provides a vendor-agnostic, industry-wide overview of the tools you use daily as data practitioners. After the deadline, we’ll enter all survey respondents into a raffle for a chance to win one Steam Deck. State of Data &amp; AI 2024 survey!

🎄 Ho ho ho, <!everyone>! Get ready for a special end-of-year treat: octavia wave Airbyte & Motherduck ducky Hackathon is here! With the launch of the new MotherDuck connector for Airbyte, we’re thrilled to continue our partnership with MotherDuck by announcing our upcoming hackathon that brings together the power of Airbyte and MotherDuck to solve the needs of delivering modern data integration, AI, and analytics solutions. Ready to join? Let’s make data flow as smoothly as Santa’s sleigh! 🛷 Details & Registration here Join #C05DVB06458 to discuss the event with fellow participants.

Happy New Year <!everyone> 🎇 ! 🌟 Calling Beta Testers for S3 datalake destination with Iceberg support! We need beta testers for our upcoming S3 datalake destination with Iceberg support! This is the evolution of our existing Apache Iceberg destination. New features include: • Airbyte 1.0 features:

destination-s3-datalake

now supports refreshes / resumable full refresh. • Iceberg support: Iceberg is the first datalake format this connector can write, with support for the Glue catalog and the Nessie catalog (new!) . • Reliability improvements: We now leverage Iceberg’s snapshot branching capabilities to checkpoint and recover from failed syncs. • Strongly-typed tables: The new connector writes data with true Iceberg types, instead of a single JSON string. • Sync mode support: Dedup mode (via merge-on-read) is now available. Learn more in the Github Discussion and reply here if you’d like to be added to our private slack channel. We’ll kick-off after the holidays are over. Airbyte OSS users are welcome! Contact us if you’re interested in testing on Airbyte Cloud. ☘️ Airbyte & Motherduck Hackathon The Airbyte + Motherduck hackathon is happening, and there is still time to participate. The event will run until 15 of January. In this event, you will showcase your creativity by building a data app. More details here or join #C05DVB06458 to discuss with other participants.

Hello <!everyone> 🏆 Airbyte Motherduck Hackathon Winners Announced! We were blown away by the creativity and quality of the projects submitted during the hackathon. The results are in, and we’re excited to share the winning projects with you! 🎉 Click here to see the winners 🗞️ Airbyte 1.4.0 Has Arrived! We’re excited to announce the release of Airbyte 1.4.0! This version comes packed with new features and important bug fixes to enhance your experience. Check out the changelog here to explore what’s new.

📢 𝗠𝗼𝘃𝗲 𝗗𝗮𝘁𝗮 𝟮𝟬𝟮𝟱 – 𝗖𝗮𝗹𝗹 𝗳𝗼𝗿 𝗣𝗿𝗲𝘀𝗲𝗻𝘁𝗲𝗿 has been extended for one more week closes on Feb 7th. Are you a data expert, practitioner, or innovator with a story to share? Move(data) 2025, celebrates the hard work and dedication of data and AI practitioners, founders, and like-minded individuals! We are looking for great speakers who will provide engaging and informative content to the audience. We're looking for insightful sessions on: 🤖 Lessons learned from building with AI 🌐 Bridging Silos: Advanced Techniques in Data Integration 🔗 Integrating RAG (Retrieval-Augmented Generation) in Data Pipelines 🗓️ Submit your proposal today and join us in shaping the future of data: https://sessionize.com/movedata-2025/ Let’s move data together! 🌟

Hello <!everyone> ! octavia wave🌟*Calling all Destination MSSQL users! We need beta testers for our new MSSQL-V2 Destination!* New features include: • Improved performance • Typing and Deduping • No more raw tables • _All the great benefits of Destinations Version 2.0!_ Learn more in the Github Discussion and comment here if you'd like to be added to our private slack channel. Airbyte OSS and Cloud users are welcome! Note: For OSS you must upgrade your Airbyte Platform version to at least 1.0.0 before using this connector version.

Hey <!everyone> octavia wave ! 🚀 move(data) 2025 is back—March 20, 2025! Get ready for 6+ hours of cutting-edge insights on *AI, automation, and the future of data engineering*—all 100% online and free to attend! 🔍 Why Join? 📢 Meet the Experts • This year’s lineup features top minds from Netflix, Warner Bros. Discovery, AWS, Google, ZoomInfo, and more, • Only 10-minute 🌩️ talks for maximum learning! • Discussions on the latest in AI, automation, and data engineering • Network with over 8,000 data professionals. 🔗 Sign Up Now! Don't miss out on this opportunity to connect and grow. Register here! If you can’t attend live, consider still registering, as we’ll share recordings to the people that signed up. 👍 Drop a '✅' below if you're in! Questions? Shoot them here and let's get the conversation started!