Hey everyone <!channel>! :octavia-hello: In Sept...
# announcements
j
Hey everyone <!channel>! 👋 In September, we released the most number of GA connectors and new sources we have ever released in a single month! airbyte rocket 17 GA connectors, 3 beta connectors, and 10 new alpha ones! airbyte rocket We also published quite a lot of content on the blog and our Youtube channel, including a new guide on the Rise of the Semantic Layer by @Simon Späti and Part 3 of @Alex Marquardt ’s deep dive into Airbyte’s sync internals (full refresh, incremental sync, CDC replication). @Charles Giardina (Airbyte) and @Chris Duong [Airbyte] will be speaking at dbt Coalesce in New Orleans - come join us or say hi in the dbt slack! Last but not least, remember to sign up for our move(data) conference (now Dec 7-8!) and get special insights and Octavia swag via our State of Data Engineering Survey! ----- A big thank you to our amazing contributors Bjorn from Choose, @macmv, @iberchid, Andrzej, @drrest, Foldi, @Ernestas, Yashkumar from Pliancy, Nugusbayev, Daemon! Changelogs in the thread as usual 🙂
keanu thanks 11
🙏 10
venting 5
attention 3
jaredgif 4
yay 9
🙏🏼 1
octavia rolling eyes 9
octavia partying 13
airbyte heart 9
🎉 22
🎉 6
💯 7
🙏🏻 1
batman pls 2
airbyte code 5
🔥 8
octavia loves 10
octavia wave 11
airbyte rocket 8
fiesta parrot 21
space float 2
New and improved features • New Sources and Promotions ◦ New Source: Adjust (#16051) - thanks Bjorn from Choose! ◦ New Source: Fauna (#15274) - thanks Neil Macneale! (@macmv) ◦ New Source: Display&Video 360 (#11828) - thanks @iberchid! ◦ New source: Primetric (#15880) - thanks Andrzej! ◦ New Source: NetSuite (#16093) - thanks @drrest! ◦ New Source:
Wrike
(#15638) - thanks Foldi! ◦ New Source: Yandex metrica (#15061) - thanks @Ernestas! ◦ New Source: Zenefits (#14809) - thanks Yashkumar from Pliancy! ◦ New Destination: R2 Cloudflare (#15296) - thanks Nugusbayev! ◦ New Destination: ElasticSearch strict encrypt (#16862) ◦ New Destination: TiDB (#15592) - thanks Daemon! ◦ Source Amazon Ads is now GA (#15358) ◦ Source Chargebee is now GA (#17142) ◦ Source Greenhouse is now GA (#17239) ◦ Source Klaviyo is now GA (#17044) ◦ Source Marketo is now GA (#17445) ◦ Source Mixpanel is now GA (#17199) ◦ Source Notion is now GA (#16813) ◦ Source Paypal Transaction is now GA (#17046) ◦ Source Pinterest is now GA (#17045) ◦ Source Recharge is now GA (#17606) ◦ Source Sentry is now GA (#17374) ◦ Source Slack is now GA (#16102, #17315) ◦ Source Snapchat Marketing is now GA (#17138) ◦ Source Twilio is now GA (#17173) ◦ Source Zendesk Talk is now GA (#17362) ◦ Source MySQL is now in Beta (#17473) ◦ Source Freshdesk is now GA (#17250) ◦ Source Iterable is now in Beta ◦ Source Sendgrid is now in Beta ◦ Source File is in Beta (#16347) ◦ Source Google Analytics 4 (GA4) is now in Alpha ◦ Source AlloyDB for Postgres is in Alpha (#16323) • New Features for Existing Connectors ◦ Database/datawarehouse JDBC source connectors (MySQL, MS SQL, Redshift, Snowflake, Oracle) now have checkpointing so that large syncs fail more gracefully ◦ Destination Kinesis: add required fields (#16952) ◦ Destination MariaDB: add custom JDBC params (#16391) ◦ Destination Clickhouse: added custom jdbc parameters field (#16444) ◦ Source Amazon Ads : Add attribution reports (#16342) ◦ Source Amazon Ads: improve
config.start_date
validation (#16191) ◦ Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes (#16453) ◦ Source BambooHR: Add custom fields validation (#16826) ◦ Source BigCommerce: add channel and store stream (#16647) ◦ Source Cart.com: implement Central API Router access method and improve backoff policy (#16612) ◦ Source Facebook Marketing: Added custom conversions stream (#15724) ◦ Source Facebook Marketing: Add
cursor_field
for custom_insights stream schema (#17551) ◦ Source Greenhouse: support incremental syncs (#16338) ◦ Source Hubspot: Update Tickets, fix missing properties and change how state is updated. (#16214) ◦ Source Iterable - added new events streams (#16067) ◦ Source: Kafka - Avro format message support (#15827) ◦ Source Mixpanel: Improved connector’s input configuration validation (#16570) ◦ Source MySQL: Implement support for per-stream state(#16007) ◦ Source MS SQL: add schemas for discovery during set up (#16002) ◦ Source MySQL: Make initial cdc timeout configurable for MySQL source (#17170) ◦ Source Plaid: minimum last updated datetime support added for Capital One items (#15231) ◦ Source Pinterest: increase
unit_test
coverage up to 90% (#16164) ◦ Source Pinterest: Add tests Pinterest OAuth flow (#16393) ◦ Source Pinterest: Added backoff strategy for rate-limit errors (#16161) ◦ Source Sentry: migrate to per-stream states (#17466) ◦ Source Zenloop: add new stream properties (#15843) • New Features in Airbyte Core ◦ Self Hosting: Improve airbyte-metrics support in the Helm chart (#16166) ◦ Remote definitions: airbyte-cron: update connector definitions from remote (#16438) ◦ Performance: support large schema discovery (#17394), Make listing all connections for workspace faster. (#17004) ◦ Low code connectors: alpha docs now available (#17534) ◦ UI: Improve behavior of password input field (#16011), Connection job list pagination (#15938), Add tooltip to source-defined cursors and primary keys (#16116), Show Validation Errors on Service Form for Missing Fields (#15625) ◦ Versioning: Add an explicit v0 version of Airbyte protocol (#16797)
🚨 Breaking changes none 🐛 Bug fixes • Helm Chart: Indentation error in templates with “extraContainers” values (#17084) • Destination: BigQuery (denormalized typed struct) - Could not evaluate union, field is expected to be one of these: NULL, STRING. (#16401) • Destination MongoDB: Remove additionalProperties:false (#16025) • Destination Postgres: fix missing database name URL Encoding (#16046) • Destination S3: fix json to avro converter when there is field name clashes (#14492) • Destination Postgres: fix missing database name URL Encoding (#16046) • Source Bing Ads: Fix: limit cache size for ServiceClient (#17505) and ReportingServiceManager instances (#17403) • Source BigQuery: added user agent header (#16902) • Source Facebook Marketing: remove “end_date” from config if empty value (#16096) • Source Facebook Marketing: limit time range to 37 months (#17027) • Source File: add data presence validation for
HTTPS
for
check_connection
(#17504) • Source Freshdesk: Added nullable to all stream schemas, added transformation into declared schema types (#17249) • Source Github: Skip repository if got HTTP 500 for WorkflowRuns stream (#17555) • Source Github: bugfix WorkflowJobs stream (#17287) • Source Google Ads: remove “end_date” from config if empty value (#16344) • Source Hubspot: remove FeedbackSubmissions stream (#16632) • Source Instagram: fix incompatible metrics error for reels. (#16428) • Source Mixpanel: disable stream “cohort_members” on discover if no access (#17415) • Source Netsuite: mention API method used to fetch the data for connector (#17082) • Source OpenWeather: Update openweather onecall api to 3.0 (#16136) • Source Postgres: remove config logging (#16138) • Source Recharge: Handle total_weight as a number (#16045) • Source Recharge: fix
Charges.total_weight
field data-type (#17080) • Source S3: choose between data types when merging master schema (#16631) • Source S3: fix reading jsonl files with nested data (#16607) • Source SalesForce: better detect API type (#16086) • Source Salesforce: exclude KnowledgeArticleVersion from bulk api (#16119) • Source Salesforce: filter out objects not supported by the Bulk API (#17453) • Source Salesforce: fix response encoding (#17314) • Source Snowflake: Handle date-time data types (#17144) • Source Twilio: shift start date for stream if it exceeds 400 days (#15972) • Source Twilio: test fix backward compatibility (#17493) • Source Twilio: Lookback_window config (#17478) • Source YouTube Analytics: fix
check_connection
403 error (#17399)
f
@Elizabeth Rodriguez jajajaja
m
🎉 hope to see the source connector for Dynamodb in the list soon 🙏🏻