So does anyone have it?

Hello everyone 👋 I wanted to use Airbyte for creating product integrations for my users, but I’m not sure if this is the right fit When playing with Airbyte to test it, correct me if I’m wrong, but it seems like you can only get data from tools you personally have access to What my users want is to connect their tool accounts on our webapp, so that we can sync their data to our tool Example: user want to connect their Pipedrive CRM to our tool to sync data between them. They connect their Pipedrive account through our webapp, and we use Airbyte to sync their data to our DB for this user Is it possible to setup something like that using Airbyte ? Or is it too complicated and not a good fit ?

Hello All! I am trying to implement

Google Play

connector using

HttpStream

class. I need to get app feedbacks by multiple langugages and countries. Should I make new

HttpStream

for each pair - language and country or I can make multiple HTTP-request by single

HttpStrem

?`

@Hi I am trying buils and connector to call an API with querystring

public override bool CheckConnection(AirbyteLogger logger, JsonElement config, out Exception exc)

{

_Key = config.GetProperty("Key").GetString();

_UID = config.GetProperty("UID").GetString();

exc = null;

try

{

return

UrlBase.AppendPathSegment($"sensorspot").SetQueryParams(new { uid = _UID, key = _Key }).GetAsync()

.Result.GetJsonAsync().Result.success;

}

catch (Exception e)

{

exc = e;

}

return false;

}

Hi I am trying to develop a connector fro calling and API with parameters

public override bool CheckConnection(AirbyteLogger logger, JsonElement config, out Exception exc)
        {
           
            _Key = config.GetProperty("Key").GetString();
            _UID = config.GetProperty("UID").GetString();
            exc = null;
            try
            {

                //return UrlBase.AppendPathSegment($"sensorspot").SetQueryParam("uid",_UID).SetQueryParam("key",_Key).GetAsync()
                //    .Result.GetJsonAsync().Result.success;

                //return UrlBase.AppendPathSegment("sensorspot")
                //.GetAsync()
                //.Result.GetJsonAsync().Result.success;
                return
               UrlBase.AppendPathSegment($"sensorspot").SetQueryParams(new { uid = _UID, key = _Key }).GetAsync()
               .Result.GetJsonAsync().Result.success;




            }
            catch (Exception e)
            {
                exc = e;
            }

            return false;
        }

Hi I am trying to develop a connector to call an API with parameters with this as code, when I try to create a source it gives "'System.Dynamic.ExpandoObject' does not contain a definition for 'success'" error. If I write

UrlBase.AppendPathSegment($"sensorspot?uid=_UID, key = _Key ").GetAsync().Result.GetJsonAsync().Result.success;

then it returns

<https://api.abcsfag.com/sensorspot>*%3F*uid=8C289812CFA4&key=RJUL)

as URL replacing ? as %3F .Attaching logs for reference. Any help would be highly appreciated. 🙂

Good morning! I'm trying test my connector and just running the existing tests using the documented command,

python -m pytest -s unit_tests

from the venv. I'm running into an error with the generated patch_base_class in all the generated tests:

E       fixture 'mocker' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, patch_base_class, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

Any tips to resolve?

My http connection requires custom error handling. I have set up the

should_retry

method accordingly. Based on the logs

should_retry

is evaluating to

True

as expected given line 200 of the logs contains the

DefaultBackoffException

class, but the default retry behavior of retrying 5 times with exponential backoff does not seem to be kicking in. Has the default behavior changed, or is there something additional I need to do to enable it?

For those working on HubSpot connectors I was curious what the common CDWs the data was getting moved to?

turns out that the API i'm working with has a per minute rate limit; how do i configure that on airbyte connectors

What would be a good way to debug this error in the

Set up the source

screen in a k8s airbyte environment on GCP? The exact same image is working if I set the source up on the local docker-compose environment. However if I try to set-up the source with the same image in the cloud environment (kubernetes) I'm getting this error. Can't find any more error messages, except that the kubernetes pod is shutting down almost immediately with exit code 1.

Do you know when you will have Netsuite as a connector?

Hello everyone, I am about to develop a connector for a Rest api that should fetch the data and then store them an sql database. Some endpoints require a parameter which is contained in another endpoint response. In details: • call and endpoint to retrieve all the ids • use those ids to call another endpoint What is the best way to tackle this kind of problem? Should I fetch the parent endpoint each time? Or should I get the ids from the sql table?

I have so many connectors on production those were created for test purpose. Is there any way to delete them?

https://data.lacity.org/resource/2nrs-mtv8.json This is the endpoint of API want to send the returned JSON response from this APl to snowflake (edited) have used the REST API connector https://hub.docker.com/r/airbyte/source-http-request which should be good enough for a simple request like this

This is the settings for source connector

But it's not working, please help

This is the log file

Hello everyone, is anyone able to use Google Analytics connector? It still requires View ID and since Google Analytics switched to v4, there are no Views anymore. So I cannot make this connector run.

Hi 👋 I’m thinking of extending the BambooHR source connector to support the custom report endpoint. The main complexity I see is that the endpoint takes a POST request with a list of fields, and returns only those fields. Has something like this already been implemented in another connector? One solution would be to add this list of field as a property in

spec.json

, and use that in the request. Maybe there’s a better way 🤷

Hi Team, How can we connect to Oracle Database using Oracle destination Connector having specified service name instead of SID? Can you please suggest?

Hi all,

I am building a connector for a source that does not support refresh tokens, so when the current token expires you have to log in with basic auth again. I have overwritten the `should_retry` method to reset `self._session.auth` on 401 errors and return `True`. The initial login is working and based on the logs the request is getting retried, but it seems like the new login is not being applied. Any ideas where I am going wrong?

i know its been a while but we're still encountering issues with Hubspot even after updating to the latest release. I've got access to the logs now and I've spotted a KeyError that might be worth looking at

Hi guys, getting an error Woocommerce connector connect to Kafka

at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more Caused by: java.lang.IllegalStateException at com.google.common.base.Preconditions.checkState(Preconditions.java:486) at io.airbyte.workers.protocols.airbyte.DefaultAirbyteDestination.notifyEndOfStream(DefaultAirbyteDestination.java:96) at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$5(DefaultReplicationWorker.java:307) ... 4 more ,retryable=<null>,timestamp=1647939936767], io.airbyte.config.FailureReason@79ef25f6[failureOrigin=destination,failureType=<null>,internalMessage=io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1,externalMessage=Something went wrong within the destination connector,metadata=io.airbyte.config.Metadata@6965f824[additionalProperties={attemptNumber=2, jobId=19}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at io.airbyte.workers.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$6(DefaultReplicationWorker.java:354) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more ,retryable=<null>,timestamp=1647939930470]]] 2022-03-22 090536 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating...

Hi guys, getting an error Woocommerce connector connect to Kafka at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more Caused by: java.lang.IllegalStateException at com.google.common.base.Preconditions.checkState(Preconditions.java:486) at io.airbyte.workers.protocols.airbyte.DefaultAirbyteDestination.notifyEndOfStream(DefaultAirbyteDestination.java:96) at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$5(DefaultReplicationWorker.java:307) ... 4 more ,retryable=<null>,timestamp=1647939936767], io.airbyte.config.FailureReason@79ef25f6[failureOrigin=destination,failureType=<null>,internalMessage=io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1,externalMessage=Something went wrong within the destination connector,metadata=io.airbyte.config.Metadata@6965f824[additionalProperties={attemptNumber=2, jobId=19}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at io.airbyte.workers.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$6(DefaultReplicationWorker.java:354) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more ,retryable=<null>,timestamp=1647939930470]]]

Copy code

2022-03-22 09:05:36 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating...

Hi Guys getting an error when try to connect Woocommerce connector to kafka

at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more Caused by: java.lang.IllegalStateException at com.google.common.base.Preconditions.checkState(Preconditions.java:486) at io.airbyte.workers.protocols.airbyte.DefaultAirbyteDestination.notifyEndOfStream(DefaultAirbyteDestination.java:96) at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$5(DefaultReplicationWorker.java:307) ... 4 more ,retryable=<null>,timestamp=1647939936767], io.airbyte.config.FailureReason@79ef25f6[failureOrigin=destination,failureType=<null>,internalMessage=io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1,externalMessage=Something went wrong within the destination connector,metadata=io.airbyte.config.Metadata@6965f824[additionalProperties={attemptNumber=2, jobId=19}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at io.airbyte.workers.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$6(DefaultReplicationWorker.java:354) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more ,retryable=<null>,timestamp=1647939930470]]] 2022-03-22 090536 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating...