@kapa.ai i have the following data as response from the API:

"custom_fields": [
      {
        "field_id": 1,
        "entity_type": "job",
        "field_name": "Projektnummer",
        "field_type": "number"
      },
      {
        "field_id": 2,
        "entity_type": "job",
        "field_name": "Start der Kampagne",
        "field_type": "date"
      },
      {
        "field_id": 3,
        "entity_type": "job",
        "field_name": "Ende der Kampagne",
        "field_type": "date"
      },
      {
        "field_id": 4,
        "entity_type": "job",
        "field_name": "Projektstatus",
        "field_type": "dropdown"
      },
      {
        "field_id": 5,
        "entity_type": "job",
        "field_name": "Mitwirkungspflicht",
        "field_type": "text"
      },
      {
        "field_id": 6,
        "entity_type": "job",
        "field_name": "Angebot ID",
        "field_type": "text",
        "value": "custobj_emejkm0iZBbG0tL4ZyN1OhmbV5n8OlVeVmnmILOSv67"
      }],

But i want to add angebot ID value appended in the response

Using MySQL to Postgres migration on OSS, MySQL has 20+ streams. Is it possible to maintain field order of MySQL. E.g stream1 has 1st column as primary key and it should be 1st column on postgres after sync?

@kapa.ai While loading into snowflake getting error, java.lang.NullPointerException: Cannot invoke "com.fasterxml.jackson.databind.JsonNode.asInt()" because the return value of "com.fasterxml.jackson.databind.JsonNode.get(String)" is null

@kapa.ai What are some good default resource values to set for the CPU and Memory of the various resources?

@kapa.ai I have set up a source and destination, everything seems to be working fine, my syncs are completing without an issue, however no records are moved. I am connecting a source: MongoDB with a destination: Redshift

@kapa.ai I've setup the Google Secret Manager but when Airbyte tries to create the secret it's getting the error "Secret.replication should not be provided"

@kapa.ai How can I configure an incremental sync on a customer connector in the web UI Builder when there is no cursor field in the data? I would like to configure a start date, set a step value of P1D, and pass each date value in a query parameter called start_date and end_date. It will make an API call for each start_date and end_window until now(), and then the state will be stored so when the connection runs the next day, it only calls since the stream_state state that is yesterday.

@kapa.ai I'm trying to write to an S3 destination and I'm getting a

java.io.IOException: Broken pipe

error. Any common causes for this?

@kapa.ai How does appsflyer source connector pull data from the appsflyer source ? I want to schedule it such that we always get previous day's entire data . But when I run it at 18:30 UTC which is 00:00 IST our business times , I'm getting event time's date as 2025-10-22 but the extracted at min and max are 2025-10-21 190747.036 +0000 and 2025-10-22 191234.658 +0000 which means some was pulled on 21st and rest on 22nd . Can we fix this to get data only for an entire day ? What should be the schedule for such a requirement ?

@kapa.ai i am trying to build connections between postgres to postgres , when the sync is completed i get 0 bytes of data sync

@kapa.ai a replication job on airbyte is terminating with

Terminating due to java.lang.OutOfMemoryError: Java heap space

how can I provide more memory to replication job?

what do Minio is used for?

@kapa.ai can you explain what each component in airbyte is used for? and for production and scaling what shall i keep in mind?

@kapa.ai I've the following configuration in values.yaml file

minio:
  enabled: true
  secretName: ""

  image:
    # -- Minio image used by Minio helm chart
    repository: minio/minio
    # -- Minio tag image
    tag: RELEASE.2023-11-20T22-40-07Z

  rootUser: ""
  rootPassword: ""

  storage:
    volumeClaimValue: 500Mi

  # -- Node labels for pod assignment, see <https://kubernetes.io/docs/user-guide/node-selection/>
  ##
  nodeSelector: {}

how do i change the storageclass to gp3-xfs for the pvc.

@kapa.ai how can i change the volume size of pvc for postgresql db component?

@kapa.ai java.util.concurrent.CompletionException: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms (total=1, active=1, idle=0, waiting=3)

@kapa.ai Where does airbyte store my secrets for connector? I want to know how secured it is, and can someone from airbyte team can look at it

How should backoff be implemented in the low-code connector in the manifest.yml file?

@kapa.ai my server and worker pods are restarting because of the load although i have autoscaling in place in my cluster.

@kapa.ai I'm looking for a guide to deploy Airbyte on AWS EKS, do you have a terraform or cloudformation script?

@kapa.ai Configuration check failed Workload failed, source: airbyte_platform Internal message: Unable to persist the job Output, check the document store credentials. Failure origin: airbyte_platform Que significa este log?

@kapa.ai how can i add more users? by default in auth parameter I've given password and mail for admin, but for other users how can i add them on the ui i don't see much options at all. and what control level i can have over other users?

@kapa.ai if I uninstall airbyte using

abctl local uninstall

and then reinstall it, do i still get to keep all of my config files (connections, sources, destinations)?

@kapa.ai after upgrading airbyte from 1.5.1 (using helm version v1) to version 2.0.0 (using helm version 2.0.8), job pods are not starting up. is there any migration step that i might have missed. running version 2.0.0 with a new external database worked so i dont believe it was related to any configuration

This message was deleted.

@kapa.ai, The Stream:

product_variants

, Non-handable error occured: [{'field': ['query'], 'message': "Invalid bulk query: Field 'inventoryManagement' doesn't exist on type 'ProductVariant'"}, {'field': ['query'], 'message': "Invalid bulk query: Field 'weight' doesn't exist on type 'ProductVariant'"}, {'field': ['query'], 'message': "Invalid bulk query: Field 'weightUnit' doesn't exist on type 'ProductVariant'"}, {'field': ['query'], 'message': "Invalid bulk query: Field 'requiresShipping' doesn't exist on type 'ProductVariant'"}, {'field': ['query'], 'message': "Invalid bulk query: Field 'weight' doesn't exist on type 'ProductVariant'"}, {'field': ['query'], 'message': "Invalid bulk query: Field 'fulfillmentService' doesn't exist on type 'ProductVariant'"}] what is this error in my shopify connection

@kapa.ai with airbyte version 2.0.0, airbyte-server throwing error "2025-10-24 115437,144 [io-executor-thread-9] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception io.airbyte.data.services.ServiceAccountNotFound: Service account not found: 646521bc-145c-480d-8b82-714968ee1429"

@kapa.ai Issue: When creating a new source, Airbyte (v2.0.1) is able to create secrets in AWS Secrets Manager but cannot read them back. Error:

WARN    i.a.c.s.p.AwsSecretsManagerClient$DefaultImpls(getSecret):203 - Secret airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_e97b6402-2192-4c72-af0d-6b34f5f61398_v1 not found

values.yaml:

secretsManager:
    enabled: true
    type: AWS_SECRET_MANAGER
    secretName: airbyte-config-secrets
    awsSecretManager:
      region: eu-west-2
      authenticationType: instanceProfile

@kapa.ai Internal message: Failed to read the output of a successful workload decd338e-5647-4c0b-adf4-da0e75f5a750_9f3c3b8e-6063-4c05-a609-b86439958ab5_0_check Failure origin: airbyte_platform Failure type: system_error got this error