Hi. I need help in adding a new stream in Quickboo...
# help-connector-developmentm
Mahesh Thirunavukarasu
06/20/2023, 12:31 PMHi. I need help in adding a new stream in Quickbooks connector. I am trying to add a quickbooks cdc api to capture the deleted invoice records. But the api throws error in the cdk connector. However, it works in the connector builder ui, but the authentication of quickbooks oauth2 type is not supported there. Appreciate any help on this topic.
k
kapa.ai
06/20/2023, 12:31 PMA message from kapa.ai
kapa.ai
06/20/2023, 12:32 PMHmm, I don't know enough to give you a confident answer yet. However, you can refer to the following documentation for more information on various topics related to connector development: - [Authentication](https://docs.airbyte.com/connector-development/connector-builder-ui/authentication/) - [Record processing](https://docs.airbyte.com/connector-development/connector-builder-ui/record-processing/) - [Pagination](https://docs.airbyte.com/connector-development/connector-builder-ui/pagination/) - [Error handling](https://docs.airbyte.com/connector-development/connector-builder-ui/error-handling/) - [Incremental sync](https://docs.airbyte.com/connector-development/connector-builder-ui/incremental-sync/) - [Partitioning](https://docs.airbyte.com/connector-development/connector-builder-ui/partitioning/) For more specific guidance on your issue, you may want to check the [Airbyte forum](https://discuss.airbyte.io) or join the [Airbyte Slack community](https://airbytehq.slack.com) to ask for help from other users and the Airbyte team.
j
Joe Reuter (Airbyte)
06/20/2023, 1:05 PMGood news first - we are very close to supporting single-use refresh token in the connector builder: https://github.com/airbytehq/airbyte-platform-internal/pull/7040
Joe Reuter (Airbyte)
06/20/2023, 1:05 PMAbout making it work right now, what error are you seeing? And can you share your source code somewhere (e.g. a github repo)?
m
Mahesh Thirunavukarasu
06/20/2023, 3:15 PMHello @Joe Reuter (Airbyte)
Please find my manifest.yaml here. I am trying to sync two streams. I modified the current Quickbooks alpha connectors and removed few streams
https://github.com/believe-Mahesh/airbyte-qbo/blob/bing/source-quickbooks-try-custom/source_quickbooks/manifest.yaml
j
Joe Reuter (Airbyte)
06/20/2023, 4:10 PMCan you copy/pate the exact error?
m
Mahesh Thirunavukarasu
06/20/2023, 4:56 PM@Joe Reuter (Airbyte),
Please find the log details below. This is a failed sync
Copy code
yield from self._read_stream(
File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 196, in _read_stream
for record in record_iterator:
File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 268, in _read_incremental
for message_counter, record_data_or_message in enumerate(records, start=1):
File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/declarative_stream.py", line 101, in read_records
yield self._apply_transformations(record, self.config, stream_slice)
File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/declarative_stream.py", line 121, in _apply_transformations
raise ValueError(
ValueError: Unexpected record type. Expected typing.Union[typing.Mapping[str, typing.Any], airbyte_protocol.models.airbyte_protocol.AirbyteMessage]. Got <class 'list'>. This is probably due to a bug in the CDK.
,retryable=<null>,timestamp=1687280019192,additionalProperties={}], io.airbyte.config.FailureReason@16d2f495[failureOrigin=source,failureType=<null>,internalMessage=Source process exited with non-zero exit code 1,externalMessage=Something went wrong within the source connector,metadata=io.airbyte.config.Metadata@2287558c[additionalProperties={attemptNumber=0, jobId=7082, connector_command=read}],stacktrace=io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1
at io.airbyte.workers.general.DefaultReplicationWorker.lambda$readFromSrcAndWriteToDstRunnable$8(DefaultReplicationWorker.java:483)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1589)
,retryable=<null>,timestamp=1687280019464,additionalProperties={}]],commitStateAsap=true,additionalProperties={}]
2023-06-20 16:53:39 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):162 - Sync summary length: 16039
2023-06-20 16:53:39 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...
2023-06-20 16:53:46 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to Get a connection by connection Id
2023-06-20 16:53:47 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to get the most recent source actor catalog
2023-06-20 16:53:47 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to Get a connection by connection IdMahesh Thirunavukarasu
06/20/2023, 4:57 PMAlso, Please let me know if syncing cdc data like the way i am doing is the right way to go. If not, Please suggest the alternatives.