Hi everyone, I’m a newbie to Airbyte (building a simple use-case since we’re looking to move away from Stitch) and I’m running into the following issue: I’ve defined a new connector and was able to successfully build out the Docker image, create the new connector locally, and establish a connection. When I try to ingest data for my “segment” stream with

Basic Normalization

, I get the following error (I know that the connector works because when I try to ingest with the normalization setting set to

Raw data (JSON)

, I am able to successfully retrieve data):

2022-09-21 04:30:31 normalization >   File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 36, in run
2022-09-21 04:30:31 normalization >     self.process_catalog()
2022-09-21 04:30:31 normalization >   File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 64, in process_catalog
2022-09-21 04:30:31 normalization >     processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema)
2022-09-21 04:30:31 normalization >   File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 55, in process
2022-09-21 04:30:31 normalization >     stream_processors = self.build_stream_processor(
2022-09-21 04:30:31 normalization >   File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 138, in build_stream_processor
2022-09-21 04:30:31 normalization >     properties = get_field(get_field(stream_config, "json_schema", message), "properties", message)
2022-09-21 04:30:31 normalization >   File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 230, in get_field
2022-09-21 04:30:31 normalization >     raise KeyError(message)
2022-09-21 04:30:31 normalization > KeyError: "'json_schema'.'properties' are not defined for stream segments"

Hey looks like an issue with json schema, can you verify if properties is defined for all the streams Kindly create an issue in our forum (https://discuss.airbyte.io) so that team can help you there.

@Harshith (Airbyte) Thanks for the response. I only have this

segments

stream defined because it’s the only one I’m replicating and it does have the properties key defined.

Hey can you confirm the schema file of segments in the schema folder

schemas/segments.json

{
  "streams": [
    {
      "name": "segments",
      "supported_sync_modes": [
        "full_refresh"
      ],
      "source_defined_cursor": false,
      "json_schema": {
        "type": "object",
        "properties": {
          "id": {
            "type": [
              "null",
              "string"
            ]
          },
          "name": {
            "type": [
              "null",
              "string"
            ]
          },
          "creator": {
            "type": [
              "null",
              "string"
            ]
          },
          "created": {
            "type": [
              "null",
              "string"
            ]
          },
          "url": {
            "type": [
              "null",
              "string"
            ]
          }
        }
      }
    }
  ]
}

Gotcha you can check a sample https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-google-ads/source_google_ads/schemas/accounts.json here on the schema files