Hi Team, I am getting this error while working on ...
# ask-community-for-troubleshooting
b
Hi Team, I am getting this error while working on the new connector? Can anyone help me with this one?
airbyte_cdk.sources.declarative.parsers.undefined_reference_exception.UndefinedReferenceException: Undefined reference definitions.base_requester.url_base from ('definitions', 'retriever', 'url_base')
h
Can you share the code? i think the
url_base
property is missing in
base_requester
b
Copy code
version: "0.1.0"

definitions:
  selector:
    extractor:
      field_pointer: ["data"]
  requester:
    url_base: "<https://api.twitter.com/2/tweets>"
    http_method: "GET"
    authenticator:
      type: BearerAuthenticator
      api_token: "{{ config['api_key'] }}"
    request_options_provider:
      request_parameters:
        user_id  : "{{ config['user_id'] }}"
        tweet_id : "{{ config['tweet_id'] }}"
        query    : "{{ config['query'] }}"      
  
  retriever:
    record_selector:
      $ref: "*ref(definitions.selector)"
    paginator:
    type: DefaultPaginator
    pagination_strategy:
      type: CursorPagination
      cursor_value: "{{ response['meta']['next_token'] }}"
      page_size: 100 # Not used, but check fails without it
    page_token_option:
      field_name: "next_token"
      inject_into: "request_parameter"
    url_base: "*ref(definitions.base_requester.url_base)"
    page_size_option: # Not used, but check fails without it
      field_name: ""
      inject_into: "request_parameter"
    requester:
      $ref: "*ref(definitions.requester)"
  base_stream:
    retriever:
      $ref: "*ref(definitions.retriever)"
  # liked_tweets_stream:
  #   $ref: "*ref(definitions.base_stream)"
  #   $options:
  #     name: "user_liked_tweets"
  #     path: "<https://api.twitter.com/2/users/{{> config['user_id'] }}/liked_tweets"
  retweed_by_tweets_stream:
    $ref: "*ref(definitions.base_stream)"
    $options:
      name: "retweed_by_tweets"
      path: "/{{ config['tweet_id'] }}/retweeted_by"
  recent_search_tweets_stream:
    $ref: "*ref(definitions.base_stream)"
    $options:
      name: "recent_search_tweets"
      path: "/search/recent?query={{ config['tweet_id'] }}"

streams:
  # - "*ref(definitions.user_liked_tweets_stream)"
  - "*ref(definitions.retweeted_by_tweets_stream)"
  - "*ref(definitions.recent_search_tweets_stream)"

check:
  stream_names:
    # - "user_liked_tweets"
    - "retweed_by_tweets"
    - "recent_search_tweets"
Thanks @Haithem (WOOP) for taking a quick look
h
Copy code
url_base: "*ref(definitions.base_requester.url_base)" - > url_base: "*ref(definitions.requester.url_base)"
You don't have a
base_requester
property in your file.
u
Thanks for jumping in, Haithem! Let me know if you need more input on this, Balaji!
1
b
thanks for your reply, team. I am new to airbyte and I am sorry I don't follow. I think I need 2 variables right base_requester or requester
h
That depends on your needs, if you don't need another requester, you can use the default one:
Copy code
requester:
    url_base: "<https://api.twitter.com/2/tweets>"
    http_method: "GET"
    authenticator:
      type: BearerAuthenticator
      api_token: "{{ config['api_key'] }}"
    request_options_provider:
      request_parameters:
        user_id  : "{{ config['user_id'] }}"
        tweet_id : "{{ config['tweet_id'] }}"
        query    : "{{ config['query'] }}"
Another suggestion, you don't need to put all your streams in
check
, just ONE, to verify that Airbyte is able to communicate with the source.
b
Thanks @Haithem (WOOP) I see the below error and I am using the same post man query here, not sure why it is failing
{"type": "LOG", "log": {"level": "ERROR", "message": "Check failed"}}
{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "FAILED", "message": "'Unable to connect to stream retweeted_by_tweets - Request <PreparedRequest [GET]> failed with response <Response [400]>'"}}
h
Can you re-run the check command with
--debug
and check if the url and headers are correct? Also did you put the _api_key_ in the
secret/config.json
?
A good way to start the low-code cdk journey.
b
thanks Haitheem for all the help
i was help to figure with your help
u
That's wonderful to hear!
thanku 1