Is there any way to set default value for stream? ...
# help-connector-developmentk
Kacper Adler
02/06/2023, 1:50 PMIs there any way to set default value for stream? I'm receiving
None[]m
Marcos Marx (Airbyte)
02/06/2023, 5:08 PMHello @Kacper Adler, you can apply limited transformations when reading data from sources. I’m curious did you try to use the “default” property in json-schema? https://opis.io/json-schema/1.x/default-value.html if this doesn’t work you can edit the read function to transform on-the-fly your record
k
Kacper Adler
02/06/2023, 5:13 PMthanks @Marcos Marx (Airbyte) I have a little more complicated thing to fix. in aha_source in the airbyte project it's trying to get: features, goals, ideas, products, and users, where goals and features are optional and depend on a payment plan. Integration fails due to lack of these streams, I need to fix it 😕
s
Sherif Nada
02/06/2023, 6:41 PMare you buildling in python or lowcode?
k
Kacper Adler
02/06/2023, 7:04 PMI'm trying to fix already done CDK module it's in airbyte merged
Kacper Adler
02/06/2023, 7:12 PMIf I add to ignore failed fetches it fails with below:
NoneType has no len()
Copy code
"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "object of type 'NoneType' has no len()", "stack_trace": "Traceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in <module>\n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 131, in launch\n for message in source_entrypoint.run(parsed_args):\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 122, in run\n for message in generator:\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 134, in read\n raise e\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 120, in read\n yield from self._read_stream(\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 189, in _read_stream\n for record in record_iterator:\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 307, in _read_full_refresh\n for record_data_or_message in record_data_or_messages:\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/declarative_stream.py\", line 115, in read_records\n for record in self.retriever.read_records(sync_mode, cursor_field, stream_slice, stream_state):\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py\", line 376, in read_records\n for record in records_generator:\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py\", line 432, in _read_pages\n next_page_token = self.next_page_token(response)\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py\", line 359, in next_page_token\n return self.paginator.next_page_token(response, self._last_records)\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/requesters/paginators/default_paginator.py\", line 102, in next_page_token\n self._token = self.pagination_strategy.next_page_token(response, last_records)\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/requesters/paginators/strategies/page_increment.py\", line 31, in next_page_token\n if len(last_records) < self.page_size:\nTypeError: object of type 'NoneType' has no len()\n", "failure_type": "system_error"}}}Kacper Adler
02/07/2023, 8:54 AMLooks like my issue is here: https://github.com/airbytehq/airbyte/issues/16639 and it's marked as finished by
erohmensing