Hey everyone <!channel>! :octavia-hello: Last wee...
# announcements
j
Hey everyone <!channel>! 👋 Last week, we celebrated Airbyte’s second birthday, so first and foremost, happy anniversary octavia partyingoctavia partying octavia partying !! The team took some time off on Thursday and Friday to celebrate it 🙂! We are so humble by our community, to see how dedicated people are to achieve our vision of solving data integration once and for all! We wanted to share a few pictures of the team at different events in the last month! ----- So what about our progress? In July, we shipped 6 huge sources to GA: • GitHubGoogle AnalyticsInstagramTiktok MarketingBing AdsS3 With 5 more in Beta: Postgres (look for a GA update soon!), Recharge, Klaviyo, Notion, and Snapchat Marketing and 5 others in alpha thanks to our awesome community (see thread) 👇 We are also releasing many big improvements to Airbyte, including Octavia CLI updates, Per Stream State , and OpenTelemetry metrics. Check the thread to see all the patch notes, features and bug fixes, there are a lot of them! And don’t hesitate to ask questions there, we’ll be happy to answer 🙂 . Thank you to our awesome contributors! @Adam Bloom, @Mohamed Magdy, @Zawar khan, @drrest, @gunu (as always!), @Tyler Rouze, @Stella Chung, @Tomas Perez, @cenegd, @Leo Sussan @Steve Withington and many more!
airbyte growth 2
this is fine fire 2
🎉 12
yay 2
fiesta parrot 4
👋 9
airbyte 5
octavia shock 3
octavia loves 3
❤️ 6
octavia thinking 4
airbyte heart 4
airbyte rocket 19
keanu thanks 2
🔥 42
airbyte code 2
Here are the patch notes from v0.39.27-alpha to v0.39.41-alpha: *New and improved features* • 🎉Source GitHub: now GA • 🎉 Source Google Analytics: now GA (#13957) • 🎉 Source Instagram Ads: now GA (#14162) • 🎉 Source Tiktok Marketing: now GA (#14358) • 🎉 Source Bing Ads: now GA (#13679) • 🎉 Source S3: now GA • 🎉 Source Postgres now in beta (#14326) • 🎉 Source Recharge now in beta • 🎉 Source Klaviyo now in beta • 🎉 Source Notion now in beta • 🎉 Source Snapchat Marketing now in beta, with stats streams (#14577) • 🎉 New alpha source: Google Analytics Data API v1 (#12701) • 🎉 New alpha source: Glassfrog (#13868) • 🎉 New alpha source: Timely #13292 (#14335) • 🎉 New alpha source: Kyriba (#12748) • 🎉New alpha source: Elasticsearch (#14118) • 🎉New alpha source: Orbit.love (#13390) • Octavia CLI improvements (full docs) ◦ Users should be able to easily switch between Airbyte instances and deploy the same configurations on multiple instances. ◦ Users can enable normalization or custom DBT transformation from their yaml configurations. ◦ Users can set custom HTTP headers on the requests made to the Airbyte server. It allows users with instances secured with basic auth or IAP to use the CLI, and it’s a significant step to make the CLI compatible with Airbyte Cloud once we release the public API. ◦ Users can import existing remote resources to a local octavia project with
octavia import
◦ Users can get existing resources (#13254) ◦ Users can retrieve the JSON configuration from a remote resource using
octavia get
, this can be useful for some scripting / orchestration use cases. • Core Airbyte features: ◦ Per Stream State ▪︎ Per-Stream state new flow (#14634) ▪︎ Release per stream to the OSS project (#15008) ◦ Display new per-stream and global state to users when viewing connection settings (#15020) ◦ Base Normalization: handle airbyte_type from stream schema in normalization (#13591) ◦ Security: Upgrade platform to openjdk:19-slim-bullseye (#14971) ◦ Self Hosting: Refactor OSS Helm Charts (#14794) ◦ Monitoring: We’ve officially added an Open Telemetry interface to Airbyte OSS, along with a set of actionable metrics to use when running OSS Airbyte at scale! ◦ CDK: Add support for enabling debug from command line and some basic general debug logs (#14521) ◦ CDK: Add a schema_generator tool (#13518) ◦ Low Code CDK in development: take a peek! (#14402) (#14317) (#14288)(#14317)(#14004) ◦ Docs: improve doc for contributing locally (#14661) ◦ Docs: Airbyte Cloud’s Single vs. Multiple Workspaces (#14608) • New Connector features: ◦ 🎉Redshift, Databricks, Snowflake, S3 Destinations: Make S3 output filename configurable (#14494) ◦ Destination S3: update INSTANCE_PROFILE to use AWSDefaultProfileCredential (#14231) ◦ Destination Oracle: custom JDBC parameters (#13841) ◦ Source Amazon Seller Partner: add FBA storage fees report (#14625) ◦ Source Amazon Seller Partner: Add new streams (#13604) ◦ Source Azure Table Storage: Add incremental append capability (#14212) ◦ Source BingAds: expose hourly/daily/weekly/monthly options from configuration (#13801) ◦ Source Cart.com: add order_statuses and increase pagesize to 1000 (#14752) ◦ Source File: add user-agent option (#14488) ◦ Source File: Add YAML format (#14588) ◦ Source Gitlab: add GroupIssueBoards stream (#13252) ◦ Source PayPal Transaction: added
OAuth2.0
, fixed bug with normalization (#15000) ◦ Source Postgres: make initial cdc waiting time configurable (#14451) ◦ Source Okta: add User_Role_Assignments and Group_Role_Assignments stream (#14556) ◦ Source Okta: add GroupMembers stream (#14380) ◦ Source Okta: OAuth2.0 authorization method (#14710) ◦ Source Okta: add custom roles stream (#14610) ◦ Source Notion: add OAuth authorization for source-notion connector (#14706) ◦ Source Sendgrid: add /messages stream (#14256) ◦ Source Tiktok Marketing: Video metrics stream (#13650)
🚨 Breaking changes • Source Facebook Marketing: update sdk version to 14.0.0 (#15007) • Source Github: rename field
mergeable
to
is_mergeable
(#14274) • Source Google Ads: enable full refresh, disable incremental and expected_records (#14191) • Source Tiktok marketing - remove granularity config option (#13890) 🐛 Bug fixes • Destination BigQuery can handle nulls inside arrays (#14522) • Destination Clickhouse: enabled normalization tests (#15069) • Destination Clickhouse: Added socket_timeout property (#13516) • Destination Google Sheets: flushes on every state message and flush when buffer gets too large (#14751) • Destination Redshift: use s3 bucket path for s3 staging operations (#13916) • Source Chargebee: fix transactions sort-by (#14672) • Source Klaviyo: added new keys to schema (#14947) • Source Klaviyo: unit test coverage 96% (#14872) • Source Klaviyo: max_retries = 10 for
Lists
stream (#14617) • Source Kustomize: Use
resources
since
bases
is deprecated (#14037) • Source Facebook Marketing: Fix since cannot be in the future in time_range error (#14831) • Source Facebook Marketing: check “MANAGE” task permission (#14786) • Source Facebook Marketing: allow configuration of MAX_BATCH_SIZE (#14267) • Source Freshsales: tune get_view_id function (#14705) • Source Freshdesk: Update parsing of start_date param (#14049) • Source Github: improve schema for stream
pull_request_commits
added “null” (#14613) • Source Github: use GraphQL for
reviews
stream (#13989) • Source Github: add Retry for GraphQL API Resource limitations (#14376) • Source Google Analytics v4: Add handle quota error (#14949) • Source Google Analytics: Specify integer for dimension ga:dateHourMinute (#14298) • Source Google Ads: Updated API version from v9 to v11 (#14614) • Source Hubspot: do not limit reading data to 30 days for property History Stream (#15035) • Source Hubspot: fixed SAT test, commented out expected_records (#14140) • Source Hubspot: Add private app authentication (#14412) • Source Hubspot: refactor PropertyHistory stream (#14666) • Source Intercom: added
state_checkpoint_interval
for all incremental streams (#14830) • Source Intercom: fix
conversation_parts
stream schema (#14554) • Source Intercom: refactored slow streams to use
/search
endpoint (#14403) • Source Intercom: extend
Contacts
schema with new properties (#14099) • Source Monday: timeout errors (#14443) • Source MySQL: test utf8 char in json column (#14865) • Source MySQL: upgrade debezium version to 1.9.2 (#14542) • Source MSSQL: aligned regular and cdc syncs and its datatype tests (#14379) • Source Okta: unit tests were added (#14927) • Source Orbit: remove workspace_old.json (#14208) • Source Outreach: fix record parsing and cursor field access (#14386) • Source Notion: fix normalization (#14505) • Source Paypal Transaction: RESULTSET_TOO_LARGE validation (#14804) • Source Postgres: retrieve only the tables in the publication under cdc mode (#14447) • Source Recharge: fix
additionaProperties
in spec.json (#14978) • Source Recharge: increase
unit_test
cov, fix schemas (#14902) • Source S3: “decimal” type added for parquet (#14911) • Source Singer: Fix incompatibilities and typing issues (#14148) • Source Snowflake: Source/Destination doesn’t respect DATE data type (#14828) • Source TiDb: Removed additionalProperties: false from spec (#14996) • Source TikTok Marketing : Access token is null #14299 (#14461) • Source Webflow add ids to streams (#14689) • Source Zendesk Support: Convert
ticket_audits.previous_value
values to string (#15036) • Source Zendesk Support: Convert
tickets.custom_fields
values to string (#14829)
n
Happy anniversary 🎉
octavia loves 3
k
Check out more announcements and some community goodness in our newsletter 🙂 happy to forward the Aug edition if you DM me your email address
j
Is there a list of all GA connectors?
a
You can find that list here @Jordan Fox https://docs.airbyte.com/integrations/
👍 1