Connector: Facebook Issue: Expanding Rate Limit Hi All, I am putting in a request to facebook to expand my api usage limit, but before I can request that it requires I provide them with “an application” for them to load and test themselves to make sure I am using their data properly. As Airbyte is just a backend application that isn’t client facing, did anyone find away around this?

Hi team, sorry to post here and not in discourse. I upgraded my Snowflake destination from 0.4.22 to 0.4.24 and all of my jobs are failing even after reconfiguring my sync to use User name and password. Log failure is showing an oAuth error

Hi. When deploying via Helm, can I controll how much cpu/ram the source/destination pods use ? I have no limit or request set, but there are limits being set, and they are a bit low for my data volume. Things are running slow...

Hi there! ⚠️ Thank you for your time posting in our Slack Channel. We’re moving our support discussions over to discuss.airbyte.io; if you would be so kind as to post your questions there, you’ll be able ask for help from fellow users as well as the Airbyte team. We’ll leave everything in place for the rest of April if you need to get information or troubleshooting from existing threads, with the intention of archiving the channel in early May. We really appreciate everyone helping us to build our community, and we’re looking forward to the next phase.

Hello! I have a ec2 Linux Instance with Airbyte and I need configure a bucket s3 (same account) as source/destination. Already allow via function role full acess to the bucket between my ec2 instance and I’m having problem with the autentication returns the message that the key and secret key of the user (my user has full access) it’s is incorrect. Somebody has a example of a successful connection between airbyte and aws s3?

Hello there, may I ask what this error is about when setting up Google PubSub as a destination connector?

Hi there! ⚠️ Thank you for your time posting in our Slack Channel. We’re moving our support discussions over to discuss.airbyte.io; if you would be so kind as to post your questions there, you’ll be able ask for help from fellow users as well as the Airbyte team. We’ll leave everything in place for the rest of April if you need to get information or troubleshooting from existing threads, with the intention of archiving the channel in early May. We really appreciate everyone helping us to build our community, and we’re looking forward to the next phase.

Hi ! Just bumping a github issue. Is this issue being looked at ? I have several fields with this problem. As I commented there, I have the same issue but happening with BIGSERIAL on the Postgres source. https://github.com/airbytehq/airbyte/issues/9590

Long (Int64) is serialized as float-like (scientific) number

Hi folks! Not sure if this is the right place for security questions, but I’m a security engineer in an organization thinking about using Airbyte. Regarding telemetry that’s being sent back to Airbyte, I had a few questions • What ports are sending out telemetry and to where? If we cut off that access will this break the Airbyte instance? Is it a common customer use case to deny all access except what’s needed for current connectors, or does that break behavior? • Are we getting responses back from Airbyte that influences container behavior in any way? • What does the “Anonymize my usage data” toggle do in the screenshot? If it’s off, does it mean no telemetry is being sent, or does it mean that it’ll be sent regardless without anonymization? ◦ Is there an option to fully disable telemetry? In case Airbyte is breached I’m worried about its impact, including reconnaissance in customer systems. Related issue: https://github.com/airbytehq/airbyte/issues/9428 Thanks!

postgres

Hi, I have this exact error running on arm64. Did you resolved this? Thanks in advance.

These log buttons seemingly do nothing (maybe because there is no data?)

Hi there, are you aware that some parts of Airbyte's docs are broken?

Hi there, are you aware that some parts of Airbyte's docs are broken?

Hi everyone, I new aborad and come here to take a look

i’m experiencing this issue in the latest zendesk support version. thankfully in the issue they reference a previous version where rate limits are handled correctly

Hi all. I'm trying to configure the Amazon Seller Partner source (latest: 0.2.15) in Airbyte (0.35.53) but I'm getting this error when I try to set up the source: TypeError('cannot unpack non-iterable NoneType object') All the credentials seem to be correct. Anyone familiar with this issue?

Looks like this belongs here 😅 hey there, i tried updating my instance and seeing issues, could anyone help? 🙂

airbyte-server      | 2022-04-12 09:31:28 INFO i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):99 - Database available!
airbyte-server      | 2022-04-12 09:31:28 INFO i.a.d.Databases(createPostgresDatabaseWithRetry):57 - Database available!
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.811Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/GET_SPEC/3","wf-task-queue-type":"Activity","lifecycle":"Starting","logging-call-at":"taskQueueManager.go:238"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.811Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/GET_SPEC/3","wf-task-queue-type":"Activity","lifecycle":"Started","logging-call-at":"taskQueueManager.go:242"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.826Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"1@07b096598acc:23029e80-aa07-4c6f-aa67-81fbefda049e","wf-task-queue-type":"Workflow","lifecycle":"Starting","logging-call-at":"taskQueueManager.go:238"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.827Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"1@07b096598acc:23029e80-aa07-4c6f-aa67-81fbefda049e","wf-task-queue-type":"Workflow","lifecycle":"Started","logging-call-at":"taskQueueManager.go:242"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.828Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/1@07b096598acc:23029e80-aa07-4c6f-aa67-81fbefda049e/2","wf-task-queue-type":"Workflow","lifecycle":"Starting","logging-call-at":"taskQueueManager.go:238"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.829Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/1@07b096598acc:23029e80-aa07-4c6f-aa67-81fbefda049e/2","wf-task-queue-type":"Workflow","lifecycle":"Started","logging-call-at":"taskQueueManager.go:242"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.837Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/1@07b096598acc:23029e80-aa07-4c6f-aa67-81fbefda049e/3","wf-task-queue-type":"Workflow","lifecycle":"Starting","logging-call-at":"taskQueueManager.go:238"}
airbyte-temporal    | {"level":"info","ts":"2022-04-12T09:31:28.837Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/1@07b096598acc:23029e80-aa07-4c6f-aa67-81fbefda049e/3","wf-task-queue-type":"Workflow","lifecycle":"Started","logging-call-at":"taskQueueManager.go:242"}
airbyte-server      | 2022-04-12 09:31:28 INFO i.a.c.EnvConfigs(getEnvOrDefault):834 - Using default value for environment variable CONFIGS_DATABASE_INITIALIZATION_TIMEOUT_MS: '60000'
airbyte-server      | 2022-04-12 09:31:28 INFO o.f.c.i.l.s.Slf4jLog(info):49 - Flyway Community Edition 7.14.0 by Redgate
airbyte-server      | 2022-04-12 09:31:28 INFO o.f.c.i.l.s.Slf4jLog(info):49 - Database: jdbc:<postgresql://db:5432/airbyte> (PostgreSQL 13.6)
airbyte-server      | 2022-04-12 09:31:29 ERROR i.a.s.ServerApp(main):249 - Server failed
airbyte-server      | java.lang.NullPointerException: Cannot invoke "org.flywaydb.core.api.MigrationInfo.getVersion()" because the return value of "io.airbyte.db.instance.DatabaseMigrator.getLatestMigration()" is null
airbyte-server      |   at io.airbyte.db.instance.MinimumFlywayMigrationVersionCheck.assertMigrations(MinimumFlywayMigrationVersionCheck.java:75) ~[io.airbyte.airbyte-db-lib-0.35.65-alpha.jar:?]
airbyte-server      |   at io.airbyte.server.ServerApp.assertDatabasesReady(ServerApp.java:135) ~[io.airbyte-airbyte-server-0.35.65-alpha.jar:?]
airbyte-server      |   at io.airbyte.server.ServerApp.getServer(ServerApp.java:160) ~[io.airbyte-airbyte-server-0.35.65-alpha.jar:?]
airbyte-server      |   at io.airbyte.server.ServerApp.main(ServerApp.java:247) [io.airbyte-airbyte-server-0.35.65-alpha.jar:?]
airbyte-server exited with code 1

Hi all Has anyone written an Airbyte webhook listener on FastAPI? I have a very strange error (

WARNING:  Unsupported upgrade request.

) and I can't figure out what's wrong. If you did, please share the code with me. It is mine:

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
2022-04-12 13:26:42.464 UTC [30] WARNING:  no usable system locales were found
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....2022-04-12 13:26:43.543 UTC [36] LOG:  starting PostgreSQL 13.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
2022-04-12 13:26:43.549 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-04-12 13:26:43.555 UTC [37] LOG:  database system was shut down at 2022-04-12 13:26:43 UTC
2022-04-12 13:26:43.558 UTC [36] LOG:  database system is ready to accept connections
 done
server started
CREATE DATABASE


/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/000_init.sql
ALTER SYSTEM
CREATE DATABASE
You are now connected to database "airbyte" as user "docker".
GRANT


waiting for server to shut down....2022-04-12 13:26:44.030 UTC [36] LOG:  received fast shutdown request
2022-04-12 13:26:44.034 UTC [36] LOG:  aborting any active transactions
2022-04-12 13:26:44.037 UTC [36] LOG:  background worker "logical replication launcher" (PID 43) exited with exit code 1
2022-04-12 13:26:44.037 UTC [38] LOG:  shutting down
2022-04-12 13:26:44.053 UTC [36] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

2022-04-12 13:26:44.156 UTC [1] LOG:  starting PostgreSQL 13.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
2022-04-12 13:26:44.157 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2022-04-12 13:26:44.157 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2022-04-12 13:26:44.168 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-04-12 13:26:44.173 UTC [53] LOG:  database system was shut down at 2022-04-12 13:26:44 UTC
2022-04-12 13:26:44.178 UTC [1] LOG:  database system is ready to accept connections
2022-04-12 13:26:49.619 UTC [108] ERROR:  relation "public.workspace" does not exist at character 15
2022-04-12 13:26:49.619 UTC [108] STATEMENT:  select * from "public"."workspace"

airbyte-db can't start and all the other serives are airitng for it. Anyone knows why?

hi folks, i’m trying to deploy airbyte using the helm chart. i keep getting the following error:

Error: template: airbyte/charts/airbyte/templates/worker/deployment.yaml:60:23: executing "airbyte/charts/airbyte/templates/worker/deployment.yaml" at <include "airbyte.database.secret.name" .>: error calling include: template: airbyte/charts/airbyte/templates/_helpers.tpl:78:50: executing "airbyte.database.secret.name" at <.Subcharts.postgresql>: nil pointer evaluating interface {}.postgresql

(more details in thread)

Hi there! ⚠️ Thank you for your time posting in our Slack Channel. We’re moving our support discussions over to discuss.airbyte.io; if you would be so kind as to post your questions there, you’ll be able ask for help from fellow users as well as the Airbyte team. We’ll leave everything in place for the rest of April if you need to get information or troubleshooting from existing threads, with the intention of archiving the channel in early May. We really appreciate everyone helping us to build our community, and we’re looking forward to the next phase.

@John (Airbyte) @[DEPRECATED] Marcos Marx Looks like you've posted a profile of me and my company without permission on your website, could I have that removed please?

Hi there! 🚨 We will this channel tomorrow. It will still be available for reading past messages but you won't be able to post here anymore. We are continuing our community support on our Discourse forum. See you there 👋🏻*, in the Troubleshooting category.*

glad to see everyone, How we could set up a socketTimeout to connect (jdbc) to our database?

Hi all, I’m having trouble connecting the Facebook Pages source connector. I keep getting the same error I found in a previous slack message that says

"HTTPError('400 Client Error: Bad Request for url: <https://graph.facebook.com/v12.0/>

But Facebook is now using v13.0 so perhaps that’s the issue? I just don’t know how to get an access token for v12.0 instead.

• Is this your first time deploying Airbyte: Yes • OS Version / Instance: Windows • Memory / Disk: 32Gb / 457GB • Deployment: Docker • Airbyte Version: 0.35.65-alpha • Source name/version: Postgres (version unknown) • Destination name/version: N/A • Step: Setting new source • Description: I am trying to set up a new Source in Airbyte that connects to a Google Cloud SQL Postgres database and am getting variations of this error when I try to save the connection

Could not connect with provided configuration. Error: Cannot create PoolableConnectionFactory

I want to create a destination connection to Postgres but it is failing telling me it is unable to create a poolable Data Factory. Any good advice is appreciated. Here is the log: 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[?:?] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:549) ~[?:?] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[?:?] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at java.net.Socket.connect(Socket.java:633) ~[?:?] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.core.PGStream.createSocket(PGStream.java:231) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.core.PGStream.<init>(PGStream.java:95) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:98) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.Driver.makeConnection(Driver.java:465) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.postgresql.Driver.connect(Driver.java:264) ~[postgresql-42.2.18.jar:42.2.18] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:55) ~[commons-dbcp2-2.7.0.jar:2.7.0] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355) ~[commons-dbcp2-2.7.0.jar:2.7.0] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:115) ~[commons-dbcp2-2.7.0.jar:2.7.0] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:665) ~[commons-dbcp2-2.7.0.jar:2.7.0] 2022-04-12 223743 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - ... 13 more 2022-04-12 223744 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling...

Hi, https://github.com/airbytehq/airbyte/pull/10905 shows merged but cant see in which version it was released, are the changes there or it might take more time

Hey All, Hope you don't mind me posting this again. Anyone have any more info on this error https://discuss.airbyte.io/t/json-schema-validation-failure-compression-level/481