@Tuan Nguyen published his first Airbyte recipe! 🎉 As you know, Airbyte uses dbt to normalize the extracted data. Tuan shares how you can modify the dbt code generated by Airbyte to partition and cluster BigQuery tables. This has being asked before on the Slack! https://airbyte.io/recipes/bigquery-partition-cluster

A case study on Airbyte!! https://blog.threado.com/p/airbytecasestudy

We just published an article to explain why we started by open-sourcing ELT, and not any other type of data integration. We also give a glimpse on our vision to become the new standard for all data movement in the future. https://airbyte.io/blog/airbyte-strategy-to-commoditize-all-data-integration

report on state of community tools from orbit.love https://cdn.glitch.me/9f234bd6-bd99-421b-997d-8c15b9676ce8%2FConstellation-Report-State-of-Community-Tools-2021.pdf?v=1635444413419

Databricks proving lakehouse pattern by breaking warehouse performance record: https://databricks.com/blog/2021/11/02/databricks-sets-official-data-warehousing-performance-record.html

are there any good books on airbyte out there? or any introductory material that isn't the documentation

I just recently read the classic papers from Codd and Boyce/Chamberlin and felt very inspired by their sense of purpose. So I decided to write a tribute to these pioneers from our field. https://jpmonteiro.substack.com/p/codds-dream

Airbyte's engineering blog is out! The first article comes from @Jared Rhizor (Airbyte) https://airbyte.io/blog/extending-docker-images-on-kubernetes "When we first started orchestrating third-party containers in Kubernetes, we found out that we needed to extend container entrypoints to be able to perform syncs. Since Kubernetes does not allow inspecting Docker entrypoints, we needed a strategy to identify this entrypoint when launching the container." More articles coming soon! Excited that our engineers are sharing the code on our Github repo and also our learning along the journey :)

thanks @Dániel Molnár! https://www.dataengineering.academy/pipeline-data-engineering-academy-blog/the-pipeline-academy-awards-2021-pipies

For those who couldn't attend the last Community Call, here is the recipe sharing how to integrate Prefect, Airbyte and dbt 🙂 https://airbyte.io/recipes/elt-pipeline-prefect-airbyte-dbt

For those that missed our Demo Hours this week, here's a demo on how you can easily deploy Airbyte on K8s with the open-source project, plural.sh!

Featuring Airbyte, dbt, Superset, and OpenMetadata among other cool projects 🎉 https://towardsdatascience.com/building-an-end-to-end-open-source-modern-data-platform-c906be2f31bd

We wrote a blog post explaining how Airbyte works on Kubernetes - those that run Kube might want to give this a read to better understand what goes on behind the scenes!

A nice article about data platform: https://towardsdatascience.com/building-an-end-to-end-open-source-modern-data-platform-c906be2f31bd

How do you collect behavioral data? @Arpit (ask.astorik.com) published his first article on our blog sharing his experience using CDI, CDP and ELT tools to collect behavioral data from what he likes to call primary and secondary sources. https://airbyte.com/blog/collect-behavioral-data-guide

Pls how can I be a writer on Airbyte??

@Madison Mae shared her best practices for creating a style guide for your dbt projects. It starts by storing a copy of your raw data, creating base models with dbt and then defining which kind of transformations to apply for each stage of your data pipeline. Curious to know what others include on a style guide for data modelling? https://airbyte.com/blog/best-practices-dbt-style-guide

interesting video "Build your open source data warehouse with Restack, Airbyte, Metabase & BigQuery"

Kudos to @Dunith who shared how to build a data pipeline to feed a user-facing analytics dashboard. I found interesting how Apache Pinot is able to parse the Airbyte raw data going through Kafka and handle updated data coming from the MySQL CDC log with upserts! https://airbyte.com/tutorials/real-time-data-analytics-pipeline

I'm trying to read more about connecting to APIs, FTP servers, and the like, in Python... do y'all know any good books for that? All things connections?

Hi all - more of a listen than a read - but the open-source startup podcast just released a new episode with Airbyte CEO Michel Tricot! Check it out⚡ https://anchor.fm/ossstartuppodcast/episodes/E21-Airbyte--Open-Source-Data-Integration-e1g1n7t/a-a7k6d4g

In this week's community call, Saif Abid teaches us how to build Go Based Connectors with Bitstrapped! Here's the video in case you missed the live stream. 😎👇

Incase you are interested in Natural Language Query (NLQ) in BI. It all started with Alan Turing writing about “Computing Machinery and Intelligence” in 1950. https://www.veezoo.com/blog/natural-language-query-nlq-in-business-intelligence-history-and-comparison/

Hey! @Sherif Nada shared how we test connectors correctness and best practices with Connector Acceptance Tests (CATs)! https://airbyte.com/blog/black-box-testing-data-connectors?

good morning team Anyone managed to upload the airbyte with EKS - Fargate on Amazon can help me? I have volume related problem

Airbyte engineering blog is on fire! If you are curious about how we orchestrate ELT jobs internally, @Benoit Moriceau (Airbyte) shared how we write Temporal workflows in Java 🙂 https://airbyte.com/blog/scale-workflow-orchestration-with-temporal

This isn't necessarily a good read but more of a good watch! Our latest community call with Faros is now live on YouTube! 😄

@Michael Louis and his team shared the process they used to create an open-source dbt package to clean and compute metrics coming from the Airbyte Github connector! 👏 Do you think that dbt packages for Airbyte sources should be shared between companies, or are those too specific for each use case? https://airbyte.com/tutorials/dbt-package-to-analyze-github-data