Hey everyone I ve been trying to make a connection using Air

Question

Hey everyone I ve been trying to make a connection using Airbyte API as listed here gt <https airbyte public api docs s3 us east 2 amazonaws com rapidoc api docs html post v1 connections create> However if I mention the field operationIds in the input schema then it errors out saying that operationIds is not a valid field name so I proceeded ahead without the field operationIds and when I posted the data I got status code 200 for this When I went back to the Airbyte UI the sync operation had started by itself and it was failing citing the reason that Unexpected character code 39 was expecting double quote to start field name if there were an error then how come I got the response code 200 I m attaching the logs below Can anyone please suggest as to what I m missing here

Rahul Padhy · Answer

I ve followed the exact schema as given in the link above for passing the JSON input please do tell if there s anything incorrect or changed from there

Subodh (Airbyte) · Answer

hey Rahul can you share the curl request for the API call with your input Also which version of Airbyte are you using

Rahul Padhy · Answer

Curl request as in The input JSON to the POST method

Subodh (Airbyte) · Answer

yes correct

Rahul Padhy · Answer

Ok

Chris (deprecated profile) · Answer

It would be safer to have ` operationsIds ` instead of empty strings

Rahul Padhy · Answer

0 22 2 alpha gt my current airbyte version

Rahul Padhy · Answer

< Chris deprecated profile > It isn t even taking the field operationIds just errors out saying that its an invalid field

Chris (deprecated profile) · Answer

it s normally optional so you can completely omit it otherwise we re still working on this part of the API

Rahul Padhy · Answer

Okay yes I ve omitted it as of now sweat smile

Chris (deprecated profile) · Answer

instead of using empty strings in the field you should leave them empty arrays instead

Chris (deprecated profile) · Answer

like `cursorField` `primaryKey` etc

Rahul Padhy · Answer

Okay as of now just using operationIds gives out an error saying that its an invalid field

Chris (deprecated profile) · Answer

that field does not exist for 0 22 2

Rahul Padhy · Answer

Okay anyways I m omitting that

Chris (deprecated profile) · Answer

you can check here whereas in 0 23 0 it s there

Rahul Padhy · Answer

I ve seen the fields that are to be there for that table and I ve made sure that I ve included all the fields with the correct inputs and types for those fields still I m stuck as to why in the logs JSON field name error is coming up

Subodh (Airbyte) · Answer

Can you retry excluding the `operationsIds` and using empty array without empty string for fields like `cursorField`

Rahul Padhy · Answer

But for 0 22 2 my version of Airbyte the field operationIds wasn t even included referring to the yaml file link sent by < Chris deprecated profile > just above

Subodh (Airbyte) · Answer

Am a bit confused here <https airbytehq slack com archives C021JANJ6TY p1622110968116000 thread ts=1622110220 112700 amp cid=C021JANJ6TY> in your request it seems like you did put `operationsIds`as part of the request Am curious about the result when you make the POST request without `operationsIds` and do not use empty string in array for other fileds

Rahul Padhy · Answer

Aah sorry sending you the right one

Rahul Padhy · Answer

This is the response that I get after doing the POST request without using operationIds

Rahul Padhy · Answer

`Status code 200` ` ` ` connectionId 8c456147 405c 4079 a292 be1d84a6b77d ` ` name default ` ` prefix ` ` sourceId 61952097 5cea 46bd 96f8 90fc4cd2290e ` ` destinationId c89a4973 2bfd 4273 a72d 58754549cae2 ` ` syncCatalog ` ` streams ` ` ` ` stream ` ` name api oppo hist ` ` jsonSchema ` ` $schema <http json schema org draft 07 schema > ` ` type object ` ` properties ` ` ACV ` ` type number ` ` ` ` CS Revenue ` ` type number ` ` ` ` CVR SQL to Customer ` ` type number ` ` ` ` CVR from MQL to SQL ` ` type number ` ` ` ` Churn ARR ` ` type number ` ` ` ` Churned Customers ` ` type number ` ` ` ` Closed Date ` ` type string ` ` ` ` Id ` ` type number ` ` ` ` Invoiced Revenue ` ` type number ` ` ` ` MQL BDR ` ` type number ` ` ` ` MQL Marketing ` ` type number ` ` ` ` MQL Marketing Adwords ` ` type number ` ` ` ` MQL Marketing Digital ` ` type number ` ` ` ` MQL Marketing Events Roundtable amp Webinars ` ` type number ` ` ` ` MQL Marketing X ` ` type number ` ` ` ` MQL Partnerships ` ` type number ` ` ` ` New ARR ` ` type number ` ` ` ` Number of MQL ` ` type number ` ` ` ` Number of New Customers ` ` type number ` ` ` ` Number of SQL ` ` type number ` ` ` ` Old ARR to go Live ` ` type number ` ` ` ` Region ` ` type string ` ` ` ` Renewal ` ` type number ` ` ` ` Sales Driven Expansion Revenue ` ` type number ` ` ` ` Total ARR ` ` type number ` ` ` ` Total Customer ` ` type number ` ` ` ` Usage expansion ` ` type number ` ` ` ` sdc row ` ` type number ` ` ` ` sdc sheet id ` ` type number ` ` ` ` sdc spreadsheet id ` ` type string ` ` ` ` sdc batched at ` ` type string ` ` ` ` sdc extracted at ` ` type string ` ` ` ` sdc received at ` ` type string ` ` ` ` sdc sequence ` ` type number ` ` ` ` sdc table version ` ` type number ` ` ` ` ` ` ` ` supportedSyncModes ` ` full refresh ` ` ` ` sourceDefinedCursor false ` ` defaultCursorField ` ` ` ` ` ` sourceDefinedPrimaryKey ` ` ` ` ` ` ` ` ` ` namespace ` ` ` ` config ` ` syncMode full refresh ` ` cursorField ` ` ` ` ` ` destinationSyncMode overwrite ` ` primaryKey ` ` ` ` ` ` ` ` ` ` aliasName api oppo hist ` ` selected true` ` ` ` ` ` ` ` ` ` schedule ` ` units 24 ` ` timeUnit hours ` ` ` ` status active ` ` `

Subodh (Airbyte) · Answer

did you use empty string inside array ` ` for fields like `primaryKey` in your post request

Rahul Padhy · Answer

Yes for primaryKey and cursorField

Subodh (Airbyte) · Answer

can retry without using empty string just use empty array ` `

Rahul Padhy · Answer

Okay sure give me a sec empty array for both primaryKey and cursorField

Rahul Padhy · Answer

sourceDefinedPrimaryKey

Rahul Padhy · Answer

defaultCursorKey

Rahul Padhy · Answer

For all these 4 fields do I stop using empty string inside array

Subodh (Airbyte) · Answer

yes correct where ever you are using empty string inside array just use an empty array instead

Rahul Padhy · Answer

Okay just 1 more doubt here Should I keep the empty arrays as 2 D arrays or make them as 1 D empty array

Chris (deprecated profile) · Answer

empty array of ` ` is probably preferable

Rahul Padhy · Answer

Ok

Subodh (Airbyte) · Answer

it depends on the field for `primaryKey` you use 2 D arrays for `cursorField` you use 1 D Refer to the example in the doc <https airbyte public api docs s3 us east 2 amazonaws com rapidoc api docs html post v1 connections create>

Rahul Padhy · Answer

`Status code 200` ` ` ` connectionId dabb363f 1d6f 4ee7 90d2 f97d1d15762f ` ` name default ` ` prefix ` ` sourceId 61952097 5cea 46bd 96f8 90fc4cd2290e ` ` destinationId c89a4973 2bfd 4273 a72d 58754549cae2 ` ` syncCatalog ` ` streams ` ` ` ` stream ` ` name api oppo hist ` ` jsonSchema ` ` $schema <http json schema org draft 07 schema > ` ` type object ` ` properties ` ` ACV ` ` type number ` ` ` ` CS Revenue ` ` type number ` ` ` ` CVR SQL to Customer ` ` type number ` ` ` ` CVR from MQL to SQL ` ` type number ` ` ` ` Churn ARR ` ` type number ` ` ` ` Churned Customers ` ` type number ` ` ` ` Closed Date ` ` type string ` ` ` ` Id ` ` type number ` ` ` ` Invoiced Revenue ` ` type number ` ` ` ` MQL BDR ` ` type number ` ` ` ` MQL Marketing ` ` type number ` ` ` ` MQL Marketing Adwords ` ` type number ` ` ` ` MQL Marketing Digital ` ` type number ` ` ` ` MQL Marketing Events Roundtable amp Webinars ` ` type number ` ` ` ` MQL Marketing X ` ` type number ` ` ` ` MQL Partnerships ` ` type number ` ` ` ` New ARR ` ` type number ` ` ` ` Number of MQL ` ` type number ` ` ` ` Number of New Customers ` ` type number ` ` ` ` Number of SQL ` ` type number ` ` ` ` Old ARR to go Live ` ` type number ` ` ` ` Region ` ` type string ` ` ` ` Renewal ` ` type number ` ` ` ` Sales Driven Expansion Revenue ` ` type number ` ` ` ` Total ARR ` ` type number ` ` ` ` Total Customer ` ` type number ` ` ` ` Usage expansion ` ` type number ` ` ` ` sdc row ` ` type number ` ` ` ` sdc sheet id ` ` type number ` ` ` ` sdc spreadsheet id ` ` type string ` ` ` ` sdc batched at ` ` type string ` ` ` ` sdc extracted at ` ` type string ` ` ` ` sdc received at ` ` type string ` ` ` ` sdc sequence ` ` type number ` ` ` ` sdc table version ` ` type number ` ` ` ` ` ` ` ` supportedSyncModes ` ` full refresh ` ` ` ` sourceDefinedCursor false ` ` defaultCursorField ` ` sourceDefinedPrimaryKey ` ` ` ` ` ` namespace ` ` ` ` config ` ` syncMode full refresh ` ` cursorField ` ` destinationSyncMode overwrite ` ` primaryKey ` ` ` ` ` ` aliasName api oppo hist ` ` selected true` ` ` ` ` ` ` ` ` ` schedule ` ` units 24 ` ` timeUnit hours ` ` ` ` status active ` ` `

Rahul Padhy · Answer

Does the aliasName field need to contain some specific value

Chris (deprecated profile) · Answer

it s not used

Rahul Padhy · Answer

Oh okay but still the same error exists I mean even after using some of the recommendations suggested by < Subodh Airbyte > above

Subodh (Airbyte) · Answer

Hey < Rahul Padhy> I think we would need to debug this together Am not sure why this might be happening just by looking at the logs

Rahul Padhy · Answer

Yeah sure I m there now

Subodh (Airbyte) · Answer

I am in another call meeting right now and it will be on for another 1 and half hour

Rahul Padhy · Answer

Do tell me 1 thing though in here gt <https airbyte public api docs s3 us east 2 amazonaws com rapidoc api docs html post v1 connections create> what should be the stream name

Rahul Padhy · Answer

Okay no problem

Subodh (Airbyte) · Answer

We can debug this tomorrow if you want Regarding the stream name its the table name that we are syncing

Rahul Padhy · Answer

Table name in the destination right Its fine we can debug tomorrw slightly smiling face

Rahul Padhy · Answer

Hi Subodh I m there if and when you get time we can debug slightly smiling face

Subodh (Airbyte) · Answer

Hey < Rahul Padhy> apologies I could not get back to you today I am happy to schedule a call on Monday to help you out with this

Rahul Padhy · Answer

Ok

Subodh (Airbyte) · Answer

I and Rahul had a call and the issue was mainly with the way we were making API requests We fixed the API request and everything worked fine

Rahul Padhy · Answer

Yeah thanks a lot < Subodh Airbyte >

Sunil Kumar B · Answer

Hi Rahul Can you post the json which worked for you

takan · Answer

Hi < Rahul Prasad> or < Subodh Airbyte > could you post the working json in case other guys having the similar issue

Subodh (Airbyte) · Answer

< takan> < Sunil Kumar B> hey guys sorry for the late response the JSON is specific to each connector based on your configuration this document might be of help