Wrote an article on consolidating your data to S3 with Airbyte in collaboration with the Trino community octavia loves https://medium.com/@abhi-vaidyanatha/an-opinionated-guide-to-consolidating-your-data-b09386b2b9b5

Here is a link to an article I wrote about data integration! https://airbyte.com/blog/data-integration

🍿 Fivetran vs Stripe drama… https://twitter.com/frasergeorgew/status/1529219953861140480

nice ecosystem overview from greylock https://greylock.com/greymatter/the-next-cloud-data-platform/

Hey y'all! I made a tutorial on how to easily deploy Airbyte on Kubernetes octavia loves (using open-source tools)

We shared a new Data Insights Post about Data Orchestration. It mainly covers these topics: • The Shift from Data Pipelines to Data Products • Data-aware Pipelines that know about the inner life of a task • Why a declarative approach with higher-level abstractions helps • How abstractions improve reuse code between complex cloud environments • Where do we come from with the evolution of Data Pipeline Orchestration • Open-source data orchestration tools such as Airflow, Prefect, Dagster Let us know what you think. Comments and discussions are welcome :). https://airbyte.com/blog/data-orchestration-trends

really like Narrow Waists as a form of software interop and solving combinatorial explosion. This (long) article is on HN today https://www.oilshell.org/blog/2022/03/backlog-arch.html and i really kinda look at the Airbyte Protocol as a form of Narrow Waist

Hey, here is a tutorial about how to monitor Airbyte sycns with a data quality tool. re_data is a dbt package with macros to monitor row counts, data freshness and null values. @Madison Mae shared how you can integrate Airbyte with re_data. Curious to know if others here have integrated Airbyte with other data quality tools: dbt tests, great expectations? https://airbyte.com/tutorials/identify-data-quality-issues-on-data-ingestion-pipelines

interesting discussion on HN about how Big Data became just… data https://news.ycombinator.com/item?id=31848594

Not a good read, but a discussion with the heads of preset and dbt around data engineering, semantic/metric layer, functional, and a lot more

useful snowflake summit recap: https://medium.com/snowflake/shortest-snowflake-summit-2022-recap-from-a-snowflake-data-superhero-fa7b00303936

found this timeline of the Modern Data Stack from Tristan Handy: https://www.getdbt.com/blog/future-of-the-modern-data-stack/ very interesting where he frames us in terms of historical trends

sometimes an image is a good read

An excellent article about: • Step 1: Align the Data Team to a Production Process • Step 2: Align the Data Team on Tooling • Step 3: Sanity Check the Development to Production Workflow • Step 4: Let Builders Build • Bridging the Gap Between Analytics and AI in the modern data stack Ecosystem https://continual.ai/post/building-a-modern-data-team-from-analytics-to-ai

The architecture of a B2B marketing insights platform: https://blog.christoolivier.com/p/architecture-of-a-b2b-marketing-insights?sd=pf. Thanks, @Christo Olivier for putting it together airbyte rocket.

As an example of how to create a custom Airbyte source connector, I have written a tutorial that discusses the Webflow source connector implementation that I developed with the Python Connector Development Kit (CDK). Webflow is the CMS system that is used at Airbyte for hosting our website and blog articles, and at Airbyte we use this connector to drive Webflow data into BigQuery to enhance our analytics capabilities. Find the full tutorial at: https://airbyte.com/tutorials/extract-data-from-the-webflow-api

Airbyte makes it easy to export your Google Ads data to Snowflake. Learn how in this easy-to-follow tutorial: https://airbyte.com/tutorials/export-data-from-google-ads-to-snowflake

Caught up on the MongoDB World keynote and was impressed by the demo of their Relational -> MongoDB import tool: wondering if this kind of field customization is something that we want to offer for our MongoDB destination?

Learn how to load data to a Databricks Lakehouse and run simple analytics with our Tutorial Load Data into Delta Lake on Databricks Lakehouse.

In our latest tutorial, we teach you how to easily create an ELT pipeline to replicate data from a MySQL database using log-based Change Data Capture (CDC)

here is my list of podcasts that i’m monitoring for the data space - any to add? Data Podcast list - Analytics Engineering Podcast (from dbt) - Analytics Everywhere Podcast (from Preset) - Catalog & Cocktails (from Data.world) - Data Brew by Databricks - Data Cloud Podcast by Snowflake - Data Eng Podcast by Tobias Macey - Data Stack Show by Rudderstack - Drill to Detail by Mark Rittman - Open||Source||Data by Datastax - HOSS Talks FOSS by Percona Generalist tech podcasts that sometimes cover data stuff - The Changelog - The Cloudcast - Code Story - Console DevTools - InfoQ podcast - Maintainable by Robby Russell - OSS Startup podcast - Software At Scale

• Hightouch started recently the Data Tea. • Databand I enjoy as well sometimes • And there are many more starting out, but the list above is already very extensive 🙂 👍🏻

Quick community member spotlight on @Bartosz Konieczny! He wrote a cool article about Airbyte/ data ingestion. 11 min read, gotchas section at the bottom is 🔥. Airbyte is in the air - data ingestion with Airbyte

good thread on “Modern Data Stack is garbage” cynicism https://twitter.com/rdrn_/status/1545435590354927616

Hi here, I could not find a lot of content about ingesting streaming data in an ELT fashion to your DWH. So I wrote about my experience here: https://blog.netfondstech.de/streaming-ingestion/ If you have similar use cases and would like to exchange, send me a message.

oooh nice new post from team Hightouch on measuring data team impact - with data activation. quoting @Pedram Navid:

Data Activation is the method of unlocking the knowledge sorted within your data warehouse, and making it actionable by your business users in the end tools that they use every day. In doing so, Data Activation helps bring data people toward the center of the business, directly tying their work to business outcomes.

https://hightouch.com/blog/how-to-measure-the-impact-of-your-data-team/

great TLDR on Superset 2.0: https://twitter.com/apachesuperset/status/1547698778005639168?s=21&t=TYTB2wPtJOgtmYWBjWoJ5Q

goodread on what a First Data Hire should look like https://wrongbutuseful.substack.com/p/analysts-are-explorers

A new tutorial is now live! — Learn how to easily move your LinkedIn Ads marketing data into BigQuery where it can be combined with data from other sources to get a more holistic view of your business! Gain valuable insights about customer acquisition and the value of your customer conversions from advertisements.

A good listen from @Shruti Kuber how to scale Airbyte on Kubernetes