Hi everyone. I’m still testing the system out and have just came across this error during normalization: “Runtime Error in model” … “Relation name ‘xyz.abc__dbt_backup’ is longer than 63 characters” … The relation name with the __dbt_backup amended is 64 characters :(. Is this a bug, or perhaps a DBT limitation?

Hi All, Question, can i set a start time for my integration? i would like to schedule it daily at a certain time.

Hi everyone, I have a database in firebird, is it possible that Airbyte can connect through odbc?

hello all, I'm using Airbyte to sync 2 postgres databases and it's great so far, but I was wondering can this support 2-way sync i.e. changes from A go to B and changes from B go to A

Hi, I have a question regarding the usage of the Airbyte Configuration API. So, we're planning to use the processes of creating a source, setting up replication schedule, selecting tables to replicate supposedly through the API. I'm just wondering where to put the secret key or anything related upon configuration—say, I use Stripe as a source, in the request payload.

Hey there, question about Airbyte in prod: We have deployed on an EC2 (first t3.small, then t3.large) and ca. 8 Google Sheets Connectors implemented - tiny data volume. Both with the first as well as the second size, CPU usage of the box jumps to 100% around the time the first pulls are scheduled and constantly stays there. In the first instance this lasted for five hours straight - nowhere near the amount of time or frequency we had scheduled extracts - until all CPU credits were consumed and the box was essentially killed. Any hint on what is going on? Is this expected in any way?

Hi, I'm trying to create a BigQuery destination but I'm getting this error message:

2021-01-26 03:04:46 DEBUG (/tmp/workspace/66/0) DefaultCheckConnectionWorker(run):97 - Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@416cdada[status=failed,message=com.fasterxml.jackson.core.JsonParseException: Unexpected character ('-' (code 45)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value
at [Source: (ByteArrayInputStream); line: 1, column: 3]]

My

dataset ID

only contains letters whereas the

project ID

contains

-

; I suspect that it's because the

project ID

has

-

in its name? Is it not supported? Thanks. 🙂

Hi Team, Is there any option like can add users to Airbyte? Thanks, Nithin

do I need to open some ports on docker?

Are there any docs available for running the connectors directly rather than via the web panel Airbyte comes with?

Hi! I see the terms on the Airbyte website such as "integration" and "sync". Sync to my mind can mean bidirectional: change in place 1 is reflected in place 2 and vice-versa. Can Airbyte be used in a bidirectional capacity or is it strictly source > destination integration in on direction? Thanks!

Hi, I am new to Airbyte and I am getting the below error when trying to access the UI. I can see the scheduler running. Any help would be appreciated!

Hi, Is there an official Twitter account for Airbyte ?

Hey, do you have any community (assets/icons) that we can include in architecture reference diagrams or presentations. Thanks

Your software is freaking great. I'm using it to sync signed up users, segment form traces, api usage logs from Google Sheets, RDS, and Redshift into a centralized Redshift DB. Absolutely thrilled.

👋 fun times chatting data in the office hours today! I just did a

docker-compose up

on

master

and got this:

airbyte-scheduler | 2021-01-29 19:09:39 WARN i.a.s.SchedulerApp(main):186 - {workspace_app_root=/tmp/workspace/scheduler/logs} - Waiting for Server to start...
airbyte-scheduler | Exception in thread "main" java.lang.IllegalStateException: Unable to retrieve Airbyte Version, aborting...
airbyte-scheduler | 	at io.airbyte.scheduler.SchedulerApp.main(SchedulerApp.java:194)

Web server is responding but returns

Cannot reach server. The server may still be starting up.

I think someone mentioned I might need to remove from docker volumes - any steps to do that?

Really loving Airbyte so far, thanks for the great product and even better community support! I'm working on a less straightforward use case and would love some advice on if there's a good pattern for supporting it (or if it's not a good fit!). We have an application that lets users pull data, then we build visualizations on top of this data for each customer. We're currently handling the integrations via custom code (no tap or other services), which lets us easily append customer specific data (eg: our internal customer id). I'm trying to figure out if there's a good way to integrate Airbyte in a similar way that supports a self sign up flow. I think there's an alpha API which lets me programmatically create configurations, but is there an easy way for me to either: Add custom data to each integration event or assign a raw table name ahead of time so I can map it to a specific customer in our system? I was thinking I could probably modify an existing connector, but I'm wondering if I'm missing an easier way to handle this.

Odd question - do you have sample data available for each of your different sources, if yes how did you collate it?

👋 Hi everyone!

Good morning! I notice AWS ECS not listed in the Deploy docs. Is that because it is not supported or just not tested?

Hi, May you explain the following statement written in your github? "Note that you are able to create connectors using the language you want, as Airbyte connections run as Docker containers."

Is airbyte cursor information stored in the target destination somewhere or is record-keeping in the Airbyte system's DB? I ask because I'm wondering if I were to

DROP DATABASE my_sad_db

and then reactivate my incremental sync, would I get all the data, or only the ones after the last sycn.

Hello everyone! I’m having some issues with the github connector, it seems to be looping and printing the same error strings over and over again… has this appened to anyone else before? Also, is there a nice way to kill a running synchronization from the interface (or from somewhere else)?

Hello guys! I'm having issues with the Apple App Store connector. Basically I can't setup the connection because of an error

KeyError: 'private_key'

. I'm trying to start the whole stack with

docker-compose up

and I'm on most recent master,

5dfd81a1

. It seems like the issue was fixed in this PR https://github.com/airbytehq/airbyte/pull/1975, although for whatever reason the webapp is still trying to set the old

key_file

parameter instead of the new

private_key

argument. Do you have any suggestions on how to make the webapp use the most recent specification for the connector? Thanks in advance!

Hello everyone! I have a technical question: when a source is using incremental, i see the command to start it has the flag

--state input_state.json

My question is, where is the

input_state.json

generated? Is it the scheduler that creates the file before running the command or a different machine? Also, where is this data stored?

Any idea on when https://github.com/airbytehq/airbyte/issues/957 will be available?

Hey there again! About normalization in snowflake: I’m testing the Jira connector. The data gets synced correctly to the raw tables, however the normalization is not doing a great job with that data. I have found the DBT folders in the workspace but i am struggling to understand how the normalization is actually being set-up by airbyte and if it’s possible to modify it to be improved. Anyone can help me with this?

Reminder: This is office hour time!! Join now: https://us02web.zoom.us/j/84317290190?pwd=S0E5RzZRV2NrTGw1VDM5cFZoV2JCUT09

👋 Hi everyone! Trying out airbyte to sync postgres db's.... first thing I'm noticing is the "Cleaned name"... it's not using the correct table name, it's using the cleaned name, any way to fix this?

Hello world! Trying out Airbyte with a consulting client of mine. Admirer and advocate for all things that "just work" to make my data life easier 🙂