Hello, I have hard time to integrate my Low Code c...
# help-connector-development
t
Hello, I have hard time to integrate my Low Code connector https://airbytehq.slack.com/archives/C021JANJ6TY/p1674483110076999 Now, everything is working fine locally (with python main.py read ... command) I test also with my Dockerfile just in case but I cannot connect it in the UI. I think it is the dicovery part which is failing.
image.png
How would you debug this ? I don't see anything in log
Do I need to transform null values to string ? I try to connect to posgres DB
For exemple :
Copy code
{
  "type": "RECORD",
  "record": {
    "stream": "invoices",
    "data": {
      "id": 5588501,
      "number": "F20221231-10010",
      "order_number": null,
...

}
a
can you try the discover command locally? (
python main.py discover --config secrets/config.json
)
t
Sure this is the output
Copy code
{
  "type": "CATALOG",
  "catalog": {
    "streams": [
      {
        "name": "invoices",
        "json_schema": {},
        "supported_sync_modes": [
          "full_refresh",
          "incremental"
        ],
        "source_defined_cursor": true,
        "default_cursor_field": [
          "date"
        ],
        "source_defined_primary_key": [
          [
            "id"
          ]
        ]
      },
      {
        "name": "payments",
        "json_schema": {},
        "supported_sync_modes": [
          "full_refresh",
          "incremental"
        ],
        "source_defined_cursor": true,
        "default_cursor_field": [
          "date"
        ],
        "source_defined_primary_key": [
          [
            "id"
          ]
        ]
      }
    ]
  }
}
a
thanks! Can you share your connector manifest too? It seems like the schemas are not defined
t
Copy code
version: "0.1.0"

definitions:
  selector:
    extractor:
      field_pointer: [] # "id", "number" ]
  requester:
    url_base: "<https://axonaut.com/api/v2>"
    http_method: "GET"
    authenticator:
      type: ApiKeyAuthenticator
      header: "userApiKey"
      api_token: "{{ config['userApiKey'] }}"    
    request_options_provider:
      request_parameters:
        date_after: "{{ config['date_after'] }}"    
    error_handler:
      response_filters:
        - http_codes: [ 404 ]
          # "{{ 'Resource not found' in d['error']['message'] }}"
          action: IGNORE          
  stream_slicer:
    type: "DatetimeStreamSlicer"
    start_datetime:
      datetime: "{{ config['date_after'] }}"
      datetime_format: "%d/%m/%Y"
    end_datetime:
      datetime: "{{ now_utc() }}"
      datetime_format: "%Y-%m-%d %H:%M:%S.%f+00:00"
    step: "P1M"
    datetime_format: "%d/%m/%Y"
    cursor_field: "{{ options['stream_cursor_field'] }}"   
    cursor_granularity: "P1M"
  retriever:
    record_selector:
      $ref: "*ref(definitions.selector)"
    paginator:
      type: NoPagination
    requester:
      $ref: "*ref(definitions.requester)"
    stream_slicer:
      $ref: "*ref(definitions.stream_slicer)"      
  base_stream:
    retriever:
      $ref: "*ref(definitions.retriever)"
  invoices_stream:
    $ref: "*ref(definitions.base_stream)"
    $options:
      name: "invoices"
      primary_key: "id"
      path: "/invoices"
      request: ""
      stream_cursor_field: "date"
      transformations:
        - type: RemoveFields
          field_pointers:
            - [ "mandatory_mentions" ]
            - [ "payment_terms" ]
  
  payments_stream:
    $ref: "*ref(definitions.base_stream)"
    $options:
      name: "payments"
      primary_key: "id"
      path: "/payments"
      stream_cursor_field: "date"
streams:
  - "*ref(definitions.invoices_stream)"
  - "*ref(definitions.payments_stream)"

check:
  stream_names:
    - "invoices"
    - "payments"

spec:
  documentation_url: <https://docs.airbyte.com/integrations/sources/axonaut>
  connection_specification:
    title: Axonaut Spec
    type: object
    required:
      - userApiKey
    additionalProperties: true
    properties:
      # 'TODO: This schema defines the configuration required for the source. This usually involves metadata such as database and/or authentication information.':
      userApiKey:
        type: string
        description: API access key used to retrieve data from the Exchange Rates API.
        airbyte_secret: true
      # date_after:
      #   type: string
      #   description: Start getting data from that date.
      #   pattern: ^(0[1-9]|[12][0-9]|3[01])[- /.](0[1-9]|1[012])[- /.](19|20)\d\d$
      #   examples:
      #     - "%d/%m/%Y"
OH !
I think you spot the problem...
I will look if the schema is well defined !
@Alexandre Girard (Airbyte) thanks ! It works now. I assumed too much the simplicity 🙂 since the schema is well present in openapi spec. Have a nice weekend
🙏🏼 1
🙏 1