Hey everyone! :octavia-wave: November was a busy ...
# announcements
j
Hey everyone! octavia wave November was a busy month!! • We concluded our Hacktoberfest contest, which you’ve heard plenty about: 103 new connectors merged!! We're now the platform with the most connectors on the market, thanks to our beloved community! • We also concluded our inaugural State of Data Engineering survey, which we’re pleased to report is the largest survey of data engineers ever (our tease of results)! • Airbyte OSS saw visible improvements to Airbyte Core (Progress Bars!), as well as internal ones (Airbyte Protocol v1)! • On the Airbyte Cloud side, we finally launched Airbyte Cloud in Europe, with a hybrid model that works great for Data Protection, and also launched our Cloud API in Alpha! • Last but not least, get your virtual ticket for our move(data) conference (THIS WEEK!) - speakers, share on Twitter! Awesome raffle prizes to be won! ----- Sync Progress Bars! @Davin Chia (Airbyte), @Jimmy Ma (Airbyte), and @Evan Tahler won our most recent Airbyte Hack Day with this one feature - giving you a progress bar for your sync and estimating how much time is left! See the PR and send us your ideas for other UX improvements! 🚨 MVP release of the connector builder: OSS Users running Airbyte 0.40.19 or higher can now visit
localhost:8000/connector-builder
and get to building them low-code, low-frills connectors! https://www.loom.com/share/acf899938ef74dec8dd61ba012bc872f This is the first release on our Connector Builder UI, and is focused on creating a fast feedback loop for connector developers writing YAML connectors. Look out for a full talk on this during our move(data) conference this week! We’ll be iterating on this MVP with feedback from people in our private early access program (closed for now). Meanwhile, find the docs for getting started here!
yay 4
batman pls 4
octavia thanks 5
airbyte code 3
🎉 5
🔥 6
octavia thinking 3
this is fine fire 3
🙏 2
airbyte 4
airbyte growth 4
octavia partying 5
venting 2
octavia wave 2
👋 13
octavia loves 7
airbyte heart 12
party parrot 5
airbyte rocket 18
Here are the patch notes from v0.40.18 to v0.40.23: New and improved featuresHeadline: MVP release of the connector builder (see above) • Sources Certifications ◦ Source Tempo: now in Beta! (#18936) ◦ Source Prestashop: now in Beta! (#18927) • New Features for Existing Connectors ◦ Destination Meilisearch: migrating to python and fix integration (#18036) • New Features in Airbyte Core ◦ Add airbyte-cron Helm chart (#18542) ◦ Progress Bar Estimate (#19814) • New Connectors (mostly from Hacktoberfest) ◦ Destination: Apache Doris (#17884) ◦ Destination: Apache Iceberg (#18836) ◦ Destination: Heap Analytics [python cdk] (#18530) ◦ Destination: Redpanda (#18884) ◦ Destination: S3-Glue (#18695) ◦ Destination: Typesense (#18349) ◦ Destination: Yugabytedb (#18039) ◦ Source: ActiveCampaign (#18335) ◦ Source: Aha (#18893) ◦ Source: Ashby (#18334) ◦ Source: Auth0 [python cdk] (#18338) ◦ Source: CallRail (#19407) ◦ Source: ClickUp (#17770) ◦ Source: Clockify [python cdk] (#17767) ◦ Source: CoinGecko Coins (#18248) ◦ Source: Coda [python cdk] (#18675) ◦ Source: Coinmarketcap [low-code CDK] (#18565) ◦ Source: Coin API (#18302) ◦ Source: ConvertKit (#18455) ◦ Source: Copper [python cdk] (#18848) ◦ Source: DataScope [low-code cdk] (#18725) ◦ Source: Dynamodb (#18750) ◦ Source: getLago [low-code cdk] (#18727) ◦ Source: Genesys [python cdk] (#17559) ◦ Source: Google Webfonts [low-code cdk] (#18496) ◦ Source: Intruder [low-code cdk] (#18668) ◦ Source: Mailjet SMS (#18345) ◦ Source: Mailjet Mail API (#18332) ◦ Source: MailerLite (#18336) ◦ Source: Mailersend [low-code CDK] (#18669) ◦ Source: Microsoft Dataverse [python cdk] (#18646) ◦ Source: Nasa APOD [python cdk] (#18394) ◦ Source: News API (#18301) ◦ Source: Lokalise [low-code SDK] (#18522) ◦ Source: Oura (#18224) ◦ Source: Omnisend (#18577) ◦ Source: Pexels API [low-code cdk] (#18854) ◦ Source: Plausible [low-code cdk] (#18657) ◦ Source: Polygon (#19504) ◦ Source: Public APIs [python CDK] (#18471) ◦ Source: Punk API [low-code cdk] (#18724) ◦ Source: Qonto [python cdk] (#17452) ◦ Source: Recreation.gov (#18881) ◦ Source: Rocket.chat [low-code cdk] (#18635) ◦ Source: RD Station Marketing [python cdk] (#18348) ◦ Source: RSS [Python CDK] (#18838) ◦ Source: SAP Fieldglass [low-code cdk] (#18656) ◦ Source: SmartEngage [low-code cdk] (#18701) ◦ Source: Sonar Cloud (#18475) ◦ Source: SFTP Bulk [python cdk] (#17691) ◦ Source: SurveyCTO (#19371) ◦ Source: TMDb [low-code cdk] (#18561) ◦ Source: TVMaze Schedule (#18333) ◦ Source: Twitter (#18863) ◦ Source: Tyntec SMS [low-code cdk] (#18883) ◦ Source: Waiteraid [low-code cdk] (#18165) ◦ Source: Workable (low-code SDK) (#18033) ◦ Source: Workramp (#18843) ◦ Source: Vantage [low-code cdk] (#18665) ◦ Source: Vitally (#18545) ◦ Source: Xero [python cdk] (#18666) ◦ Source: Zendesk Sell [python cdk] (#17888)
🚨 Breaking changes • Airbyte Protocol v1 (#19846): Define a set of well-known data types (#17486) ◦ This adds a set of data type schemas. These will be used in place of the existing JsonSchema primitive types. They provide out-of-the-box validation, and enforce specific formatting on the more interesting data types (e.g. timestamps). Non-primitive types (object, array, oneOf, etc) will still use raw JsonSchema types. • These well-known types map mostly 1:1 with the existing Airbyte data types. Notable differences: ▪︎ BinaryData is the only new type; it’s intended to be used in places where we currently produce a base64 string (e.g. source-postgres’ handling of BINARY columns). ▪︎ Time with/without timezone have been de facto in-use for some time, so we’ll make them official ◦ The big_integer and big_number types are being retired, because they weren’t used. In the future, we’ll add support for large numeric values via
airbyte_properties
, described below. • Note: Connectors in your instance can’t be upgraded via the UI in v 0.40.19-21. Use 0.40.22. 🐛 Bug fixes • Destination Bigquery: updated Check method to test for non-billable project (#19489) • Destination Google Sheets: Fix empty headers list (#18729) • Destination Elasticsearch: enforce ssl connection on cloud (#18341) • Destination Redshift: handle empty s3 bucket_path (#18434) • Source Airtable: improve discovery logic (#18491) • Source Asana: migrate to new SAT, added base HTTP errors handling (#19561) • Source Amplitude: fix empty
series
for
AverageSessionLength
stream (#18684) • Source Exchange Rates: Fix handling error during check connection (#18726) • Source Facebook Marketing: reduce request limit after specific error (#18734) • Source Facebook Marketing: increase timeout (#18536) • Source: Facebook Marketing - New default for
action_breakdowns
, improve “check” speed (#19803) • Source Freshdesk: do not call response.json() inside should_retry (#19349) • Source freshdesk - fix schema (#19496) • Source Postgres: Handle Arrays data types (#16990) • Source Prestashop: prevent users from using http url (#18599) • Source Salesforce: Use Apparent Encoding (#19294) • Source Salesforce: add get_error_display_message for ConnectionError (#18753) • Source Salesforce: add more unsupported bulk entities, fix fallback to rest (#19286) • Source Salesforce: make AccountHistory support BULK API (#19869) • Source Sendgrid: fixed expected records (#19819) • Source SurveyMonkey: added OAuth for Eu and Ca (#17964) • Source Tiktok Marketing: fix test timeout (#18433) • Source Twilio: fix generating date ranges (#19479) • Source Twilio: fixed expected records (#19828)
k
Congrats @Elliot Trabac @Haithem (WOOP) @Isaac Harris-Holt on Hacktoberfest!!! and I seriously impressive 28 connectors built among the three of you!!
party parrot 2
h
Thanks Karen!
airbyte heart 1
i
Thanks 🙂
airbyte heart 1