Balaji Seetharaman
11/02/2022, 1:36 PMairbyte_cdk.sources.declarative.parsers.undefined_reference_exception.UndefinedReferenceException: Undefined reference definitions.base_requester.url_base from ('definitions', 'retriever', 'url_base')
Haithem (WOOP)
11/02/2022, 1:38 PMurl_base
property is missing in base_requester
Balaji Seetharaman
11/02/2022, 1:40 PMversion: "0.1.0"
definitions:
selector:
extractor:
field_pointer: ["data"]
requester:
url_base: "<https://api.twitter.com/2/tweets>"
http_method: "GET"
authenticator:
type: BearerAuthenticator
api_token: "{{ config['api_key'] }}"
request_options_provider:
request_parameters:
user_id : "{{ config['user_id'] }}"
tweet_id : "{{ config['tweet_id'] }}"
query : "{{ config['query'] }}"
retriever:
record_selector:
$ref: "*ref(definitions.selector)"
paginator:
type: DefaultPaginator
pagination_strategy:
type: CursorPagination
cursor_value: "{{ response['meta']['next_token'] }}"
page_size: 100 # Not used, but check fails without it
page_token_option:
field_name: "next_token"
inject_into: "request_parameter"
url_base: "*ref(definitions.base_requester.url_base)"
page_size_option: # Not used, but check fails without it
field_name: ""
inject_into: "request_parameter"
requester:
$ref: "*ref(definitions.requester)"
base_stream:
retriever:
$ref: "*ref(definitions.retriever)"
# liked_tweets_stream:
# $ref: "*ref(definitions.base_stream)"
# $options:
# name: "user_liked_tweets"
# path: "<https://api.twitter.com/2/users/{{> config['user_id'] }}/liked_tweets"
retweed_by_tweets_stream:
$ref: "*ref(definitions.base_stream)"
$options:
name: "retweed_by_tweets"
path: "/{{ config['tweet_id'] }}/retweeted_by"
recent_search_tweets_stream:
$ref: "*ref(definitions.base_stream)"
$options:
name: "recent_search_tweets"
path: "/search/recent?query={{ config['tweet_id'] }}"
streams:
# - "*ref(definitions.user_liked_tweets_stream)"
- "*ref(definitions.retweeted_by_tweets_stream)"
- "*ref(definitions.recent_search_tweets_stream)"
check:
stream_names:
# - "user_liked_tweets"
- "retweed_by_tweets"
- "recent_search_tweets"
Balaji Seetharaman
11/02/2022, 1:41 PMHaithem (WOOP)
11/02/2022, 1:44 PMurl_base: "*ref(definitions.base_requester.url_base)" - > url_base: "*ref(definitions.requester.url_base)"
Haithem (WOOP)
11/02/2022, 1:45 PMbase_requester
property in your file.user
11/02/2022, 1:57 PMBalaji Seetharaman
11/02/2022, 1:58 PMHaithem (WOOP)
11/02/2022, 2:00 PMrequester:
url_base: "<https://api.twitter.com/2/tweets>"
http_method: "GET"
authenticator:
type: BearerAuthenticator
api_token: "{{ config['api_key'] }}"
request_options_provider:
request_parameters:
user_id : "{{ config['user_id'] }}"
tweet_id : "{{ config['tweet_id'] }}"
query : "{{ config['query'] }}"
Haithem (WOOP)
11/02/2022, 2:17 PMcheck
, just ONE, to verify that Airbyte is able to communicate with the source.Balaji Seetharaman
11/02/2022, 2:24 PM{"type": "LOG", "log": {"level": "ERROR", "message": "Check failed"}}
{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "FAILED", "message": "'Unable to connect to stream retweeted_by_tweets - Request <PreparedRequest [GET]> failed with response <Response [400]>'"}}
Haithem (WOOP)
11/02/2022, 2:26 PM--debug
and check if the url and headers are correct? Also did you put the _api_key_ in the secret/config.json
?Haithem (WOOP)
11/02/2022, 2:29 PMHaithem (WOOP)
11/02/2022, 2:29 PMBalaji Seetharaman
11/02/2022, 2:37 PMBalaji Seetharaman
11/02/2022, 2:37 PMuser
11/02/2022, 3:13 PM