Hi I m trying to build a connection between appstore and big Airbyte #replication-troubleshooting

Hi! I'm trying to build a connection between appst...

Regitze Sdun

08/22/2022, 9:39 AM

Hi! I'm trying to build a connection between appstore and bigquery. However, when I press sync schemas I don't get anything back from airbyte, and thus can't choose which tables I want to sync. Anyone know how to fix this?

Harshith (Airbyte)

08/22/2022, 9:43 AM

Hey looks like we try to download a report and then add that to streams https://github.com/miroapp/tap-appstore/blob/2c448137f6dead04148653ad721d5f23186bb0ee/tap_appstore/__init__.py#L122 Could you check if you have required permissions?

Harshith (Airbyte)

08/22/2022, 9:43 AM

Also if you need more assistance you can create a post on discuss.airbyte.io

Regitze Sdun

08/22/2022, 12:10 PM

which permissions are required? I chose the admin role for the key hence I would expect that to cover all permissions.

Regitze Sdun

08/22/2022, 12:29 PM

I tried the

_attempt_download_report

and it runs however I get empty data back

Copy code

{'streams': [{'stream': 'subscription_report',
   'tap_stream_id': 'subscription_report',
   'schema': {'type': ['null', 'object'],
    'properties': {'_line_id': {'type': ['null', 'integer']},
     '_time_extracted': {'type': ['null', 'string'], 'format': 'date-time'},
     '_api_report_date': {'type': ['null', 'string'], 'format': 'date'},
     'app_name': {'type': ['null', 'string']},
     'app_apple_id': {'type': ['null', 'string']},
     'subscription_name': {'type': ['null', 'string']},
     'subscription_apple_id': {'type': ['null', 'string']},
     'subscription_group_id': {'type': ['null', 'string']},
     'standard_subscription_duration': {'type': ['null', 'string']},
     'promotional_offer_name': {'type': ['null', 'string']},
     'promotional_offer_id': {'type': ['null', 'string']},
     'customer_price': {'type': ['null', 'string']},
     'customer_currency': {'type': ['null', 'string']},
     'developer_proceeds': {'type': ['null', 'string']},
     'proceeds_currency': {'type': ['null', 'string']},
     'preserved_pricing': {'type': ['null', 'string']},
     'proceeds_reason': {'type': ['null', 'string']},
     'client': {'type': ['null', 'string']},
     'device': {'type': ['null', 'string']},
     'state': {'type': ['null', 'string']},
     'country': {'type': ['null', 'string']},
     'active_standard_price_subscriptions': {'type': ['null', 'integer']},
     'active_free_trial_introductory_offer_subscriptions': {'type': ['null',
       'integer']},
     'active_pay_up_front_introductory_offer_subscriptions': {'type': ['null',
       'integer']},
     'active_pay_as_you_go_introductory_offer_subscriptions': {'type': ['null',
       'integer']},
     'free_trial_promotional_offer_subscriptions': {'type': ['null',
       'integer']},
     'pay_up_front_promotional_offer_subscriptions': {'type': ['null',
       'integer']},
     'pay_as_you_go_promotional_offer_subscriptions': {'type': ['null',
       'integer']},
     'marketing_opt_ins': {'type': ['null', 'integer']},
     'billing_retry': {'type': ['null', 'integer']},
     'grace_period': {'type': ['null', 'integer']}}},
   'key_properties': []},
  {'stream': 'subscriber_report',
   'tap_stream_id': 'subscriber_report',
   'schema': {'type': ['null', 'object'],
    'properties': {'_line_id': {'type': ['null', 'integer']},
     '_time_extracted': {'type': ['null', 'string'], 'format': 'date-time'},
     '_api_report_date': {'type': ['null', 'string'], 'format': 'date'},
     'event_date': {'type': ['null', 'string'], 'format': 'date'},
     'app_name': {'type': ['null', 'string']},
     'app_apple_id': {'type': ['null', 'string']},
     'subscription_name': {'type': ['null', 'string']},
     'subscription_apple_id': {'type': ['null', 'string']},
     'subscription_group_id': {'type': ['null', 'string']},
     'standard_subscription_duration': {'type': ['null', 'string']},
     'promotional_offer_name': {'type': ['null', 'string']},
     'promotional_offer_id': {'type': ['null', 'string']},
     'subscription_offer_type': {'type': ['null', 'string']},
     'subscription_offer_duration': {'type': ['null', 'string']},
     'marketing_opt_in_duration': {'type': ['null', 'string']},
     'customer_price': {'type': ['null', 'string']},
     'customer_currency': {'type': ['null', 'string']},
     'developer_proceeds': {'type': ['null', 'string']},
     'proceeds_currency': {'type': ['null', 'string']},
     'preserved_pricing': {'type': ['null', 'string']},
     'proceeds_reason': {'type': ['null', 'string']},
     'client': {'type': ['null', 'string']},
     'device': {'type': ['null', 'string']},
     'country': {'type': ['null', 'string']},
     'subscriber_id': {'type': ['null', 'string']},
     'subscriber_id_reset': {'type': ['null', 'string']},
     'refund': {'type': ['null', 'string']},
     'purchase_date': {'type': ['null', 'string'], 'format': 'date'},
     'units': {'type': ['null', 'integer']}}},
   'key_properties': []},
  {'stream': 'sales_report',
   'tap_stream_id': 'sales_report',
   'schema': {'type': ['null', 'object'],
    'properties': {'_line_id': {'type': ['null', 'integer']},
     '_time_extracted': {'type': ['null', 'string'], 'format': 'date-time'},
     '_api_report_date': {'type': ['null', 'string'], 'format': 'date'},
     'provider': {'type': ['null', 'string']},
     'provider_country': {'type': ['null', 'string']},
     'sku': {'type': ['null', 'string']},
     'developer': {'type': ['null', 'string']},
     'title': {'type': ['null', 'string']},
     'version': {'type': ['null', 'string']},
     'product_type_identifier': {'type': ['null', 'string']},
     'units': {'type': ['null', 'integer']},
     'developer_proceeds': {'type': ['null', 'string']},
     'begin_date': {'type': ['null', 'string'], 'format': 'date'},
     'end_date': {'type': ['null', 'string'], 'format': 'date'},
     'customer_currency': {'type': ['null', 'string']},
     'country_code': {'type': ['null', 'string']},
     'currency_of_proceeds': {'type': ['null', 'string']},
     'apple_identifier': {'type': ['null', 'string']},
     'customer_price': {'type': ['null', 'string']},
     'promo_code': {'type': ['null', 'string']},
     'parent_identifier': {'type': ['null', 'string']},
     'subscription': {'type': ['null', 'string']},
     'period': {'type': ['null', 'string']},
     'category': {'type': ['null', 'string']},
     'cmb': {'type': ['null', 'string']},
     'device': {'type': ['null', 'string']},
     'supported_platforms': {'type': ['null', 'string']},
     'proceeds_reason': {'type': ['null', 'string']},
     'preserved_pricing': {'type': ['null', 'string']},
     'client': {'type': ['null', 'string']},
     'order_type': {'type': ['null', 'string']}}},
   'key_properties': []},
  {'stream': 'subscription_event_report',
   'tap_stream_id': 'subscription_event_report',
   'schema': {'type': ['null', 'object'],
    'properties': {'_line_id': {'type': ['null', 'integer']},
     '_time_extracted': {'type': ['null', 'string'], 'format': 'date-time'},
     '_api_report_date': {'type': ['null', 'string'], 'format': 'date'},
     'event_date': {'type': ['null', 'string'], 'format': 'date'},
     'event': {'type': ['null', 'string']},
     'app_name': {'type': ['null', 'string']},
     'app_apple_id': {'type': ['null', 'string']},
     'subscription_name': {'type': ['null', 'string']},
     'subscription_apple_id': {'type': ['null', 'string']},
     'subscription_group_id': {'type': ['null', 'string']},
     'standard_subscription_duration': {'type': ['null', 'string']},
     'promotional_offer_name': {'type': ['null', 'string']},
     'promotional_offer_id': {'type': ['null', 'string']},
     'subscription_offer_type': {'type': ['null', 'string']},
     'subscription_offer_duration': {'type': ['null', 'string']},
     'marketing_opt_in': {'type': ['null', 'string']},
     'marketing_opt_in_duration': {'type': ['null', 'string']},
     'preserved_pricing': {'type': ['null', 'string']},
     'proceeds_reason': {'type': ['null', 'string']},
     'consecutive_paid_periods': {'type': ['null', 'integer']},
     'original_start_date': {'type': ['null', 'string'], 'format': 'date'},
     'client': {'type': ['null', 'string']},
     'device': {'type': ['null', 'string']},
     'state': {'type': ['null', 'string']},
     'country': {'type': ['null', 'string']},
     'previous_subscription_name': {'type': ['null', 'string']},
     'previous_subscription_apple_id': {'type': ['null', 'string']},
     'days_before_canceling': {'type': ['null', 'string']},
     'cancellation_reason': {'type': ['null', 'string']},
     'days_canceled': {'type': ['null', 'integer']},
     'quantity': {'type': ['null', 'integer']}}},
   'key_properties': []}]}

Regitze Sdun

08/22/2022, 12:40 PM

Created an issue now! https://discuss.airbyte.io/t/connecting-appstore-and-bigquery/2332

Regitze Sdun

08/23/2022, 8:45 AM

Turns out that I had set a start date too early before there was any data and that was why it failed. I think it would be nice to have an error message for this. I have resolved the issue. Now my connection manages to sync • subscription_event_report • subscription_report however I get an error for the sales_report

Regitze Sdun

08/23/2022, 9:12 AM

https://discuss.airbyte.io/t/normalization-failed-during-appstore-dbt-run-for-the-sales-report/2340

👀 1

2 Views

Open in Slack

Previous Next