https://linen.dev logo
Join Slack
Powered by
# ask-community-for-troubleshooting
  • c
    Hey guy I have a bing connection to bigquery and I'm trying to cancel the sync but it never cancels it. There is some sort of force sync to cancel? Servers give me a 502 response back from the cancel request
    s
    • 2
    • 1
  • r
    Hi Team, I am trying to build postgres source connector, but getting below error (attached screenshot). Can someone help me out with this?
    n
    • 2
    • 5
  • q
    I have an issue, The Devops team just reach out to me about airbyte causing production database to be slowing down. Here is my setup. 1. Airbyte have read only access to the prod dB. 2. One slot/publication was allocated to the pipelines 3. Sync time interl set to 5 minutes 4. Sync mode is CDC The production dB have over 80 tables. Less than ten tables have record more than 100k, though two tables exceed 1m records. How do effectively manage airbyte so as not to impact the production dB.
    s
    • 2
    • 3
  • l
    Hi all, I´m sure I red it somewhere, but can´t find it. I need to do a slight modification to the existing bing ads source connector. add a keyword field to the keyword_performance_report. How can I do it? It´s quite urgent 😬
    🙏 1
    m
    • 2
    • 5
  • l
    Same here, in the google ads source connector.. I need to add one field to a report Can someone help me/ guide me?
    🙏 1
    m
    • 2
    • 1
  • m
    Is it possible to get ga4 events in postgresql? So far I can't really use the tables that i get - do i have to make some special custom reports?
    m
    u
    • 3
    • 3
  • t
    Hello team, I'm debug the normalization for a new destination - called databend, but when I execute 'NORMALIZATION_TEST_TARGET=databend python -m pytest -s -vv airbyte-integrations/bases/base-normalization/integration_tests' I got an error : "Unable to find image 'airbyte/destination-databend:dev' locally". So Need I build the docker image first? And how to build it?
    s
    • 2
    • 2
  • t
    I tried looking into the docs (and README.md in this dir) to see what I may be missing. Apparently some docker image needs to be built beforehand, but that was not clear from the README.md, nor how to build that image.
  • s
    Anyone encountered this with Bigquery Denormalized destination? I tried syncing a table from MongoDB with quite a deep nested structure and have encountered this error:
    Copy code
    For more details see Big Query Error collection: BigQueryError{reason=invalid, location=null, message=Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details.},
 BigQueryError{reason=invalid, location=null, message=Error while reading data, error message: JSON processing encountered too many errors, giving up. Rows: 1; errors: 1; max bad: 0; error percent: 0},
 BigQueryError{reason=invalid, location=null, message=Error while reading data, error message: JSON parsing error in row starting at position 0: JSON object specified for non-record field: input.application.data.applicants}:
    When I look at the schema of the tmp table that was created in Bigquery by Airbyte, it has mapped input.application.data.applicants as String + Repeated. However inspecting the data structure in JSON format it looks like it should be a repeated record, with further nested properties inside it. Is this kind of structure (items enclosed by 
    [{ }]
    ) supported currently?
    m
    • 2
    • 9
  • m
    Hey I'm having some trouble using the api, it's giving this error: "This deployment of Airbyte is protected by HTTP Basic Authentication" how am I supposed to provide the authentication in the api call?
    s
    j
    • 3
    • 9
  • p
    Hello Airbyte team, I have a requirement where I want to sync/copy all data from one RDS (Postgres) to another RDS (Postgres). Now the issue is, my destination RDS is KMS encrypted hence I am not able to add destination RDS. Kindly help me regarding this. Is this supported ? I found this information in forum. https://discuss.airbyte.io/t/failed-to-connect-to-rds-postgres-db-when-using-storage-encryption/1712
    m
    • 2
    • 5
  • r
    What would be the comparable service in AWS and what are the advantages that Airbyte has over it?
    s
    • 2
    • 1
  • f
    question: I am implementing splunk connector via it's api. I see that my api call is returning data correctly in a web-browser. The link built by adding a <sid> into the url is correct in airbyte code and if I take that link and post it in web-browser I see all the fields and data associated with it. However in airbyte I keep on getting Response <204>. First call made to splunk api is POST call, which returns <sid> and second call is a GET that returns response of 204. Any idea why splunk api returns 204 to the call made via airbyte connector only?
    • 1
    • 1
  • u
    Hi, Team. I am working on ETL from AWS Aurora mysql to AWS Aurora Postgresql. However, it takes 6 hours to store 10 million records. It’s too slow. Is this a normal speed?
    • 1
    • 1
  • k
    Hello, I am building and running on Kubernetes (not via Kustomize). The Kustomization spec shows below services, are all required or some are optional ? In other words , What are the minimum set these services that need to run in my cloud compute? Any pointers or doc references would be great! Thanks
    • 1
    • 1
  • g
    Hi All, Doing some testing on my local machine (first time user) . tried to set up a simple connection from Google Ads to Redshift (for testing purposes) and now i get this screen on connections, sources and destinations making it unusable anyone seen anything similar?
    n
    • 2
    • 7
  • c
    hello - I’m trying to setup a connection between Salesforce and Snowflake, but I am getting an error. Is there a permission or configuration I need to do to allow this?
    Copy code
    Error: common.error

Discovering schema failed
Max retries exceeded with url
Stream Account cannot be processed by REST or BULK API.
    m
    • 2
    • 27
  • b
    Hey team, I have a scenario where I'd like to do an incremental pull from some very large tables so I don't have to fully refresh data that's not changing. I believe there is a useful modified date on those tables, so that's good for the incremental, but there's often not a decent reliable unique key. Is there any way to manage this in Airbyte? My hack thought is to have a process that deletes the prior 30 days of data (based on a transaction date in the data) from the target table and then let Airbyte do an incremental refresh using that date as the modified date, which would theoretically replace any transactions that were just deleted. I haven't tested this, and I am not sure how Airbyte handles the high water mark (query at runtime vs. stored metadata), so I don't know if this would even work. Any thoughts for this kind of "sliding window full replace" activity (or just a generally better way to do this)?
    • 1
    • 2
  • r
    Hi- Is there an on-prem enterprise edition with Airbyte?
    a
    • 2
    • 1
  • m
    👋 Hello, team! I’m trying to set a redshift destination and keep getting this error
    Copy code
    The connection tests failed.
non-json response
    • Airbyte version (open source): 0.40.23 • Deployed on EC2 • Connector Version: 0.3.51 Any ideas what can be the reason?
    m
    n
    • 3
    • 4
  • d
    Hello Team, I am trying to deploy Airbyte on a K8s using helm (all defaults, no custom values), and I am getting an error 
    Error: failed pre-install: timed out waiting for the condition
    . If I list all resources created in the namespace, I can see that the pod 
    pod/airbyte-airbyte-bootloader
    is running, and 2 other pods (
    pod/airbyte-db-0
    and 
    pod/airbyte-minio-0
    ) are pending which seems to be an issue with `PersistentVolumeClaims`:
    Copy code
    Type     Reason             Age                   From                Message
  ----     ------             ----                  ----                -------
  Normal   NotTriggerScaleUp  3m32s (x21 over 23m)  cluster-autoscaler  pod didn't trigger scale-up: 2 pod has unbound immediate PersistentVolumeClaims
  Warning  FailedScheduling   37s (x25 over 24m)    default-scheduler   0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
    Anybody had a similar problem? Any ideas how to fix it?
    j
    e
    • 3
    • 3
  • p
    Hi Guys, i am facing some issue in extracting the data from an API
    s
    • 2
    • 1
  • p
    Copy code
    {"type": "LOG", "log": {"level": "FATAL", "message": "Modelname.path() got an unexpected keyword argument 'stream_state'\nTraceback (most recent call last):\n
  • p
    I am not passing anything called stream state
  • p
    my path is blank
  • l
    Hi airbyte team Do we have an ETA for the schema evolution feature release ? I can see it in the Q4 roadmap but still wondering it it will be released before the end of the year. Thanks for the hardwork
    m
    s
    a
    • 4
    • 4
  • u
    HI all, i got below error when im configuring S3 destination (helm version:0.42.0). Not sure what's the problem, i have successfully configure the remote log in S3 with workspace path.
    k
    s
    • 3
    • 3
  • p
    Hi, I am trying to access api documentation. But the web page is throwing an error
    • 1
    • 3
  • m
    Hello, Did someone know how to reset a connection state without deleting existing data in your table ?
    m
    • 2
    • 2
  • p
    Hi guys, Do Airbyte support connecting mongodb via SRV? Also, do you support “denesting” fields in mongodb?
    • 1
    • 1
1...103104105...245Latest