Hello everyone! To kick-start this channel I would love to know if you run data quality checks after loading data to your data warehouse? Curious to know if others run tests with dbt or Great Expectations after each data sync.

Hi there, I'm curious about which data quality tools have you used and what was your experience with them? Any favourites?

Hello. Have people used Airbyte with Datafold SaaS? I'm trying out the Datafold SaaS. Love'in it!

Curious how people validate data coming from airbyte connectors from 3rd party sources? e.g. zendesk support connector. I currently have this data coming from Segment and so I am able to compare segment vs airbyte records. I’m noticing missing records (users etc.). But if I didn’t have segment data coming through, I probably wouldn’t have known about the missing records. How do people perform QA against 3rd party data sources that doesn’t already live in the warehouse?

I have a question: While loading the data with CDC enabled from Postgres to BigQuery, if we delete the records from the source then it does not delete that record in the Big query destination, how can we fix it?

great news for observability! https://techcrunch.com/2022/05/24/monte-carlo-raises-135m-series-d-at-1-6b-price-showing-that-unicorn-rounds-are-still-a-thing/?tpcc=tcplustwitter https://www.montecarlodata.com/blog-monte-carlo-raises-135m-series-d-to-accelerate-the-rapid-growth-of-the-data-observability-category/

Trying to add better incremental tests to various sources. Can someone direct me to some (good) examples, is there there some boilerplate code or method for adding these tests?

Hello Team, Question: do you have plan to build

arm64

image for Apple M1 for airbyte/destination-local-json and airbyte/destination-csv? Thanks.

Hey, all, I am not sure if this is the right place to post this. I am using the Shopify connector and pushed all products and variants into a Postgresql database. I notice that the

product_variants

table seems to be missing the

product_id

column. Is this a bug or by design? Might Basic Normalization cause problems with this?

Hi, apologies if this is not the place to post this question. I am adding a Universal Analytics connector and filling data back to January. The synch takes some time, which is normal since I have 8 custom queries and I am pulling the data daily. However, I am getting data for every other day, which is very odd (see screen grab). I am wondering if it is related to the data being pulled daily with the date dimension. Maybe the date breakdown returns empty values for the previous query? Has anyone been able to get the historical daily data without this issue?

Hi. I have troubles with Salesforce and text encoding. I made a connection in Airbyte, Salesforce to BQ, and everything worked nicely. Unfortunately, I deleted this connection, and I had to replicate it. But now, there are problems with special characters. Besides, even if the original text is the same (for example : "Vente à venir court terme"), the recorded data can vary ("Vente à venir court terme", "Vente àvenir court terme"). Thanks

Instead of full refresh you can do incremental or try scaling vertically/horizontally

👋 Hi team, I was hoping someone had run across a similar issue to me regarding null date fields in the salesforce connector. Setup I am currently running: • Airbyte:

0.40.0-alpha

• Salesforce:

0.2.5

----> BigQuery:

1.1.14

When i look in the raw data tables, I can identify string values for the date fields. However, when those tables are translated into the relational schema, the date fields populate as null. An example would be the

CloseDate

. Has anyone else experienced this issue or found a solution for it?

Please Someone allow merge this change: https://github.com/airbytehq/airbyte/pull/15822