Hello. In the Netsuite source. When the docs says "You need to select manually each record on selection lists and give Full level access on next tabs: (Permissions, Reports, Lists, Setup, Custom Records). You strongly need to be careful and attentive on this point". Have i to select all the options in the selection lists or just that records i want?

Re-attempt adding duplicate raw data

I am using airtbyte to fetch data from hubspot, it should fetch 3 rows of raw data. But due to some scope issues, workflow gets partially completed and re-attempts again. After 3 re-attempts (and partial successful execution), I’ve 9 rows of raw data (3 duplicates of each row for each attempt). Is there any way to resolve this? It works fine if I remove the specific data for which the token doesn’t have permission. Workflow gets completed and I get 3 rows.

Is there a way to purge old logs from minio on k8s?

I’m getting the error below, any insight when running an incremental export from a postgres database using CDC via wal2json plugin.

Stack Trace: org.postgresql.util.PSQLException: ERROR: out of memory
  Detail: Cannot enlarge string buffer containing 1073741293 bytes by 659 more bytes.
  Where: slot "airbyte_slot", output plugin "wal2json", in the change callback, associated LSN 1/92C0F500
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)

Setup Airbyte: 0.40.10 Source: postgres 1.0.11 connector Destination: Snowflake 0.4.38

Hey when trying to upgrade I got this following error (0.4.13 version) for the worker pod

Message: No bean of type [io.airbyte.config.persistence.split_secrets.SecretPersistence] exists for the given qualifier: @Named('secretPersistence')

Is it specific to me or it is a known issue ?

Hey, a quick question about https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm/#database-external-secrets it expect a k8s secret with the db password? how about the user??

Hi Everyone! I am trying to create a connection using airtable as a source. I am having some issue with the pre build api because I don't receive all columns in the table. How can I access the code and update it? I am not the most expert with data, so any help will be amazing

Hello! Am new to Airbyte, I might immaturely asking question that have been answered before bare with me. am getting this error.

2022-10-07 13:34:02 - Additional Failure Information: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]'COLUMNSTORE' is not supported in this service tier of the database. See Books Online for more details on feature support in different service tiers of Windows Azure SQL Database. (40536) (SQLExecDirectW)")

And it says the syncs failed! Now the weird part is.. Data has been loaded in my data warehouse but in json format How can I solve this? How can I ingest data with changing it's data type?

Helm/K8s question: What is the difference between the externalDatabase and the database objects in the values?

I am trying to customize the existing Source connector. My request has header and or JSON body. How do I create the nested field in JSON body. json_payload["limit"] = self.items_per_page_limit. -> works good at its at parent level json_payload["query/filter/date_time_filter/updated_at/start_at"] = self.start_date -> Throws error. Similarly next line too.. json_payload["query/sort/sort_order"] = "ASC"

hey all! does anyone know an approximate timeline for API access to Airbyte Cloud? (i.e. triggering syncs + other workspace actions programmatically)

Hi all, is there already an implementation of the Plaid Investment connector out there? Otherwise, I'll look to extend the current one to add investment flows.

Hi community, we had a high throughput kafka topic, is there any guidance about how to correctly scale the kafka source connectors. Or if we have multiple connections reading same topic with same consumer group id? thanks

hello, im trying to setup airbyte from google compute engine but getting error when trying to connect to airbyte using compute ssh to port 8000, is there anyone know how to solve this? thanks. using guides from https://docs.airbyte.com/deploying-airbyte/on-gcp-compute-engine

any extra configs needed to get airbyte to work on arm? Deploying on ec2 on aws

hello guys, i am try configure a connections with postgresql and have this error

Caused by: org.jooq.exception.DataAccessException: SQL [insert into "public"."actor_catalog_fetch_event" ("id", "actor_id", "actor_catalog_id", "config_hash", "actor_version", "modified_at", "created_at") values (cast(? as uuid), cast(? as uuid), cast(? as uuid), ?, ?, cast(? as timestamp with time zone), cast(? as timestamp with time zone))]; ERROR: null value in column "actor_id" of relation "actor_catalog_fetch_event" violates not-null constraint Detail: Failing row contains (f8d15d30-8df9-42ae-a424-e2063a341212, 29b27343-639b-48f7-90bf-07f83593f804, null, null, null, 2022-10-08 182729.247629+00, 2022-10-08 182729.247629+00).

i get this same error with all sources

i have airbyte with helm

i belive that airbyte is try write a the table "actor_catalog_fetch_event" with value null and expect not null

Hello guys, I want to make a custom source connector using an existing connector for a specific edge case. can anyone tell what all steps should i follow? thanks in advance.

Hello guys, It's so good to be part of the community. I am a newbie with airbyte and i really need the community's help. I am trying to replicate data from postgres to bigquery. However, tables with "timestamp without time zone" are not replicated. I am not allowed to change the source table. Is there any workaround?

Good Evening everyone. I am having problems connecting my MongoDB Atlas database as a source at airbyte. Currently running opensource @docker locally at my machine (tried many times and always have the same error Failed to fetch mongodb schema). I am able to connect properly, but receive this error when building the pipeline. The whole database is big (around 60GB). Any ideas of what can be done?

2022-10-09 160008 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.host: is missing but it is required, $.port: is missing but it is required, $.instance: does not have a value in the enumeration [standalone] 2022-10-09 160008 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.instance: does not have a value in the enumeration [replica], $.server_addresses: is missing but it is required

Hey, I am creating a custom connector and there is an issue I am hitting. This specific source has very large number of source objects (100k), so AirbyteCatalog gets quite large. And please mind that these are standard tables, not something like various API stock tickers. The Catalog JSON is ~28MB (and this is actually smaller than final size will be, since I just added mock schemas for now, not the actual ones) If I send the complete Catalog via discover it seems to wreck the frontend. First the browser fires DiscoverSchema twice (I suppose this is because of React StrictMode? But I am not running airbyte in dev mode...). It takes few minutes for the worker to get the result, but it receives it and browser seems to get result for both requests. Finaly browser sends destination_definition_specification, and receives a response, but nothing happens in the frontend. It is stuck on "we are fetching the schema of your data source." The memory footprint of the firefox tab keeps rising (6GB), until I suppose OS kills docker to get more mem. When tried on chrome the screen just gets blank. No errors in the browser console. Running latest version of airbyte in docker on Ubuntu WSL. Do you have any advice for me what to try? Thanks

Hi, I am trying to setup Airbyte data flow from Oracle to snowflake. During my initial step of setting Source as oracle i have come across following Errors. Not sure what the error says. Need help on this.

2022-10-10 065218 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Cloud storage job log path: /workspace/460c647d-e336-44f8-9bdf-3c13d00ccc0c/0/logs.log 2022-10-10 065218 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: dev-e038ea8-cloud

these are error messages that i come across

Can anyone help