Hey, just wondering what the state of pipeline monitoring support is in Airbyte - I guess I can ship the logs off, and parse the output - but I would ideally want to track such things as entities modified over time. runtimes or such via for example prometheus (I've seen a commit that removed some prometheus stuff and I've found datadog code in your repo as well, but it would be nice to have a statement of what the state+plan is 🙂 - thank you!)

Hi All, in hubspot connector is it possible pull all data for owner by ignoring start date?

Hi! Is there a way to manually change the state for a incremental connector? I’d like to fetch data a bit before, without doing a full sync again To give more context, the first full sync for Hubspot took 10+ hours (7M records, don’t ask me why 😅). With this connector, to bookmark/state is set at the end of the sync. The problem is that I missed some updates during first full sync.

Hi. How can I use the "BigQuery (denormalized typed struct)" destination, are there any docs(except the general docs on BigQuery dest.)? My issue is it doesn't make a correct schema(for a field which is an object) for BigQuery I guess, from logs:

BigQueryError{reason=invalid, location=null, message=Error while reading data, error message: JSON parsing error in row starting at position 0: JSON object specified for non-record field: meta}

I thought this destination transforms all the fields to their corresponding JSON structs or primitives. My source is MongoDB.

Any channel/resource that could help me hire AirByte expert?

Hum, the contact sales button is not working? Can you PM the sales information?

does anyone have suggestions on how to get around zendesk support connector hitting the API limits? I would have thought the rate limiting behaviour in the connector would prevent this but we have multiple APIs hitting zendesk so at high periods, the airbyte connector pushes this over the edge and seems to freeze.

good morning team Anyone managed to upload the airbyte with EKS - Fargate on Amazon can help me? I have volume related problem

I am scheduled for a user interview right now but there is no Zoom link and the facilitator does not appear to be online on Slack 🤔 can anyone help us connect?

Is DBT Cloud down for anybody else?

Can't register to 14 day trial... Slack Conversation

image.png

hey guys... which IDE do you guys recommend building/code airbyte? Intellij have a lot issues on M1 :(

Just want to flex my Airbyte progress (in Dev) :)

Hello everyone, i'm having some trouble to import an exported data of other airbyte instance, is there a deadline for solve that problem? https://github.com/airbytehq/airbyte/issues/8084

Snowflake Book Free!! 📚 https://www.linkedin.com/posts/daniel-portugal_datafriends-snowflake-dataengineering-activity-6927385658449620992-lM9D

gotta give a massive shout out to @Edward Gao on this PR (and adam in this initial PR) for resolving an issue when loading large tables! i imagine this is going to unblock quite a few airbyte users (myself included)

Hi everyone, My

google ads connector

does not load any data for yesterday and today. Airbyte does not show an error, it just pretends there is no data, although in the google ads UI I see data for the past two days. • does anybody experience the same problem? • any ideas how I could resolve the issue? Grateful for any hints. P.S.: I already reset the connection and did a fresh full load, but still no data for yesterday

This message was deleted.

Don’t forget to register for the BEAM Collage event, totally Free 😄 🐝 https://www.linkedin.com/posts/daniel-portugal_season-2022-activity-6929430204679434242-IZI8

Hi , I deployed airbyte on aws graviton instances by building using the source code with M1 instructions , Airbyte started successfully and I was able tot the use the UI but got errors while setting up anny sources or destinations

2022-05-10 08:39:24 ERROR i.a.c.i.LineGobbler(voidCall):82 - WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested .

any quick suggestion how to resolve this or I have to go back and use x86 EC2 instances

Hey everybody! When syncing from salesforce to snowflake I’m getting this in the logs:

2022-05-13 14:20:14 source > Syncing stream: Attribute_Filter__Tag 
2022-05-13 14:20:14 source > Giving up for returned HTTP status: 400, body: [{"errorCode":"INVALIDENTITY","message":"Entity '01I0e000000FMSL.Tag' is not supported by the Bulk API."}]

do you know a way to fix it?

Hi all, is it possible to customize airbyte worker logs? Specifically, I am trying to extract destination, source and normalization tag from sync worker log

awesome work today @Evan Tahler (Airbyte) & @Marcos Marx (Airbyte)! ICYMI `git push` your data stack with Airbyte, Airflow and dbt

The slack invite link online says it expired

Serverless Toronto Community invites you to join us (online) to learn how to build an AI/ML-based app that guides users through identifying and submitting complex digital documents using GCP Serverless services. Even if Chatbots and Document Processing automation are not your "cup of tea", getting access to the source code of the end-to-end solution (with Flutter, Firebase, App Engine, Cloud Run) will be priceless. Plus, our guest presenter Rui Costa is the author of two O’Reilly books on Google Cloud topics (https://www.oreilly.com/pub/au/8187), so he will be able to answer many of your work questions during the Q&A portion of the meetup :) RSVP at https://www.meetup.com/Serverless-Toronto/events/285559749/ and enjoy the weekend!

hey folks, would you consider the workflow automation tasks part of analytical engineers' scope of work? I'm seeing some clear benefits that can optimize the data flow in the org and hence minimize the further ETL and reverse-ETL processing. In my daily job, I've picked up the https://n8n.io/, a low-code tool to manage trigger-based automation between several API-based apps. interesting what the community thinks of it. the number of APIs used in the avg company drives the interconnection between them. who would you think is responsible for such kinds of tasks remains an open question?

Hey ! did someone already started to build a connector for alchemy web3 ? or something similar ?

Adding Braze as a destination would be amazing