https://linen.dev logo
Join Slack
Powered by
# ask-community-for-troubleshooting
  • r
    since moving to version v0.42.0 i cant seem to load the docs of new sources or destinations
    j
    • 2
    • 2
  • r
    commands run to recreate
  • r
    Copy code
    wget <https://raw.githubusercontent.com/airbytehq/airbyte/v0.42.0/.env>
wget <https://raw.githubusercontent.com/airbytehq/airbyte/v0.42.0/docker-compose.yaml>
wget <https://raw.githubusercontent.com/airbytehq/airbyte/v0.42.0/flags.yml>
docker compose up
  • y
    Hi, everyone. Wanna ask something, we have found data discrepancies today in 
    incremental dedup
    using Airbyte by comparing (and filtering) CDC log (using other application) and batch data (ingested via Airbyte) stored in Data Warehouse (some new data is not ingested by its reference timestamp). So far, I suspected that there is some overlapping process when Airbyte has stated that the incremental process is done, the new data is persisted into database with reference timestamp should be included into running ingestion. Is there any case like this or is it possible? Specs: Airbyte Version : 0.40.28 Datasource : Postgres version 9/10 Source Connector : airbyte/source-postgres 0.4.38
    • 1
    • 1
  • y
    For Salesforce connector, when I use the full-refresh for the first sync, I ended up getting bunch of duplicate keys. I thought only incremental would result in that because of the cursor limitations. Why and how could these duplicate happen for full refresh mode ?
    s
    • 2
    • 2
  • k
    Hi everyone, I m trying to connect Airbyte with a CSV file stored in OneDrive. I m not sure how to get the correct URL of the CSV file (I tried the right click > "Copy Link" )
    🙏 1
    y
    • 2
    • 9
  • j
    Why doesn't incremental append work with id as a cursor field? The records in that table are never modified, but I still get duplicate rows in the destination.
    v
    • 2
    • 3
  • v
    Hi everyone, I faced an issue when try to sync data from oracle to postgres. The sync job pending at normalization step, and seems never end. (a table is around 2MB) A strange thing is that I expect the normalization step will generate only one destination table, but it creates two (account and account_scd). Do you guy face this situation before?
    Copy code
    Airbyte version: v0.39.32-alpha
OS Version / Instance: Deployment: GCP Compute Engine
Step where error happened: Sync job
    j
    • 2
    • 4
  • c
    Could not discover schema for source exception while creating connection octavia generate connection --source sources/salesforcetest/configuration.yaml --destination destinations/csv/configuration.yaml salesforce-to-csv
  • r
    Hi guys, Is it realist to have an airbyte on a VM with (only) 4Gb of ram (azure b2s) ? airbyte share the vm with a low activity postgre db. airbyte replicate the postgre db to a snowflake once a day.
  • l
    Hi all, I´m really struggling to set the facebook pages source connector. I checked the token on the debugger and it´s valid with the required scopes. But I get all the time a non-json response. Any ideas??
  • s
    Hello all, I am new to Airbyte and trying to build a connector using the low-code connector and with modifying the manifest.yaml and other necessary fields. Using the authenticator as BearerAuthentication. from an API. Did almost as per the documentation provided to build a connector using low-code. I have checked the connection, read the data from API and build a docker image and while testing the i am facing the issue. I am attaching the issue screenshot please someone look into it and help me. Did anyone face same issue ? Any solutions please.
    n
    • 2
    • 1
  • w
    bumping this issue again
  • y
    👋 Hello everyone! As you know, Google Analytic-Universal Analytics is getting going away in July. To make sure that its successor, Google Analytics v4, is ready for all of your use cases, we want to collect your feedback on how to improve the connector before then. We would love to hear from you in this form here! Thank you for your help! octaviaparty 🙏🏾
    n
    • 2
    • 3
  • f
    Hi ! Can someone help me with an Airbyte API issue ? This is specifically about the 
    supportsNormalization
    response property for the 
    api/v1/destination_definition_specifications/get
    endpoint . According to the 
    openapi.yaml
    file, the response for this endpoint should follow this schema
    Copy code
    DestinationDefinitionSpecificationRead:
      properties:
        connectionSpecification:
          $ref: "#/components/schemas/DestinationDefinitionSpecification"
        destinationDefinitionId:
          $ref: "#/components/schemas/DestinationDefinitionId"
        documentationUrl:
          type: string
        jobInfo:
          $ref: "#/components/schemas/SynchronousJobRead"
        supportedDestinationSyncModes:
          items:
            $ref: "#/components/schemas/DestinationSyncMode"
          type: array
        supportsDbt:
          type: boolean
        supportsNormalization:
          type: boolean
      required:
        - destinationDefinitionId
        - jobInfo
      type: object
    but when i call the API for the 
    BigQuery
    destination id (22f6c74f-5699-40ff-833c-4a879ea40133), i am getting a response which has a missing 
    supportsNormalization
    field . Based on docs, this destination does support normalization but why is the endpoing returning a response with this property missing ? I am adding the response that i got from the API in the comments
    n
    j
    • 3
    • 6
  • a
    Hi friends - I have a question regarding the new Weaviate alpha integration / connector (a thread)
    • 1
    • 3
  • s
    calendly has joined this channel by invitation from Airbyte.
  • r
    hi, I’m building a prototype app, trying to embed the slack connector so that each user gets their own slack connection. how would I get access to the Cloud API? and actually I may eventually need to fork the slack connector to add some bespoke features, so would it make sense to start with OSS + config API?
    n
    • 2
    • 3
  • a
    Minor issue - it seems like the File Source examples may not be working? Im trying to use 
    <gs://airbyte-vault/financial.csv>
    which is linked as a sample from https://docs.airbyte.com/integrations/sources/file/#examples and it 404s in the file source setup, unless im misunderstanding the docs and those are 'pretend' configs to illustrate some usage of file source options?
  • a
    Question: For a CSV data type which has missing elements - I see a schema which has values like string, string, number, number - when a destination connector receives a number or string which is from an empty element in the data source, is that passed through as NULL, or as "" for string values, and 0 for number values?
    • 1
    • 3
  • j
    Hi team, it looks like there is no document to set up postgres connection to AWS RDS. There are some nuances in the commands. For example when you set up CDC you need to run 
    GRANT rds_replication TO <user_name>;
    instead the one listed in document 
    ALTER USER <user_name> REPLICATION;
    s
    • 2
    • 1
  • a
    I'm testing airbyte and a mysql table that I want to move to redshift or s3 has a text column that is too big or troublesome. I want to exclude this column as it is unecessary. Is there a work around? in connection to redshift it creates a String that is too large and in s3 a csv file that is malformed.
    n
    • 2
    • 1
  • j
    Hi team I am getting the following error message because I have table column named order which is a sql reserved word. Is it possible to use double quotes to wrap the column names in the table creation statement for it to work?
    Copy code
    2023-03-23 22:31:48 [ERROR] - Stream [AssignmentQuestion] Error storing 431 source objects in [5ynfrw115cdzbio29kt6s.cbeif4p6gtjk75vw6n12l8] destination (1 attempts left): Error creating [postgres_assignmentquestion] table with statement [CREATE TABLE PROD_DB.postgres_assignmentquestion (_source_id text,_timestamp timestamp(6),assignmentid text,createdat timestamp(6),eventn_ctx_event_id text,gradingtype text,id text,instructiontext text,isactive boolean,nummaxmcqselections double precision,order double precision,responsetype text,responsevisibility text,src text,title text,updatedat timestamp(6))]: 001003 (42000): SQL compilation error:
syntax error line 1 at position 256 unexpected 'order'.
    s
    • 2
    • 1
  • d
    Is there a preferred channel to ask whether some part of cloud.airbyte.com is down / not working? I've tried 3 different browsers on 3 different devices and when I try to drill into the details of a particular Connection I get this dreaded message. Note that all other pages that I've tried to navigate to are working as expected. Not working for Chrome on Windows (x2 devices), Safari on iOS, or Firefox on Windows. Same problem in all combinations
    Copy code
    Oops! Something went wrong…
Unknown error occurred
    👀 1
    a
    e
    n
    • 4
    • 4
  • j
    Is there any UI logout function? https://discuss.airbyte.io/t/secure-ui-login/938 I'm using the UI to config, but kind of want to know the best process to secure when done.
  • k
    Hello, I am new to airbyte and deployed it in local using Docker. I am trying to pull data from postgres and insert into redshift table, but I am facing below error: ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No timezone information: 2022-04-04T213122.610000 How can I resolve this error?
    w
    n
    • 3
    • 2
  • d
    Hello Team: Do we have any documentation or refernce links to pull or sync data from Databricks to Postgres. Thanks in advance
  • c
    below error is thrown when we octavia apply , what could be wrong in configaration file if raw_configuration["definition_type"] == "source": TypeError: 'NoneType' object is not subscriptable
    w
    • 2
    • 1
  • g
    Hi all.......in airbyte , is there a way to apply business logic as custom transformation after connecting to the source , and applying the logic to sync to the destination database using python language
    j
    • 2
    • 8
  • l
    Hi team, after upgrading our S3 Source Connector to 2.0.0, I notice that the 
    _ab_additional_properties
    field is now empty. Is this a intended design or bug? This design doesn’t quite make sense to me, since according to the docs in 2.0.0 onwards, the schema inference became more strict (based on the earliest file only). So, if the new fields is not kept in 
    _ab_additional_properties
    , it will become impossible to reconcile newer files with latest schema vs the older files - even if we refresh schema in Airbyte.
    u
    • 2
    • 1
1...168169170...245Latest