Hi All wave I m trying to sync data from Shopify just the or Airbyte #replication-troubleshooting

Hi All :wave:, I'm trying to sync data from Shopif...

Eric Lamphere

10/17/2022, 2:51 PM

Hi All 👋, I'm trying to sync data from Shopify (just the orders table for now) to S3 but I'm getting the following 404 error:

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://templestclair.myshopify.com/admin/api/2021-07/shopify_payments/balance/transactions.json?limit=250&order=id+asc&since_id=0>

It seems like there's been some discussion around this on GH. Does anyone know if this issue has been fixed? Note: I'm connecting with the OAuth method

✍️ 1

user

10/17/2022, 2:51 PM

@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2871 to ensure timely resolution!

Harshith (Airbyte)

10/17/2022, 2:55 PM

Hey did you mean this https://github.com/airbytehq/airbyte/issues/15242. If so yeah we closed the ticket as per https://github.com/airbytehq/airbyte/pull/17777

Eric Lamphere

10/17/2022, 3:13 PM

Yeah that's the one I was looking at. I pulled master and ran

docker-compose up --build

but the error seems to be persisting. Are there any updates I need to make to the source/connection?

Eric Lamphere

10/17/2022, 3:17 PM

Here are the error logs:

Copy code

2022-10-17 14:50:11 [44msource[0m > Starting syncing SourceShopify
2022-10-17 14:50:12 [44msource[0m > Syncing stream: abandoned_checkouts 
2022-10-17 14:50:14 [43mdestination[0m > Starting a new buffer for stream abandoned_checkouts (current state: 0 bytes in 0 buffers)
2022-10-17 14:50:14 [44msource[0m > Read 11 records from abandoned_checkouts stream
2022-10-17 14:50:14 [44msource[0m > Finished syncing abandoned_checkouts
2022-10-17 14:50:14 [44msource[0m > SourceShopify runtimes:
Syncing stream abandoned_checkouts 0:00:02.389329
2022-10-17 14:50:14 [44msource[0m > Syncing stream: balance_transactions 
2022-10-17 14:50:14 [44msource[0m > Encountered an exception while reading stream balance_transactions
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 114, in read
    yield from self._read_stream(
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 159, in _read_stream
    for record in record_iterator:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 248, in _read_full_refresh
    for record in records:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 373, in read_records
    response = self._send_request(request, request_kwargs)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 337, in _send_request
    return backoff_handler(user_backoff_handler)(request, request_kwargs)
  File "/usr/local/lib/python3.9/site-packages/backoff/_sync.py", line 110, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/backoff/_sync.py", line 110, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 299, in _send
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://templestclair.myshopify.com/admin/api/2021-07/shopify_payments/balance/transactions.json?limit=250&order=id+asc&since_id=0>
2022-10-17 14:50:14 [44msource[0m > Finished syncing balance_transactions
2022-10-17 14:50:14 [44msource[0m > SourceShopify runtimes:
Syncing stream abandoned_checkouts 0:00:02.389329
Syncing stream balance_transactions 0:00:00.354938
2022-10-17 14:50:14 [44msource[0m > 404 Client Error: Not Found for url: <https://templestclair.myshopify.com/admin/api/2021-07/shopify_payments/balance/transactions.json?limit=250&order=id+asc&since_id=0>
Traceback (most recent call last):
  File "/airbyte/integration_code/main.py", line 13, in <module>
    launch(source, sys.argv[1:])
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 127, in launch
    for message in source_entrypoint.run(parsed_args):
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 118, in run
    for message in generator:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 123, in read
    raise e
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 114, in read
    yield from self._read_stream(
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 159, in _read_stream
    for record in record_iterator:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 248, in _read_full_refresh
    for record in records:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 373, in read_records
    response = self._send_request(request, request_kwargs)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 337, in _send_request
    return backoff_handler(user_backoff_handler)(request, request_kwargs)
  File "/usr/local/lib/python3.9/site-packages/backoff/_sync.py", line 110, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/backoff/_sync.py", line 110, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 299, in _send
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://templestclair.myshopify.com/admin/api/2021-07/shopify_payments/balance/transactions.json?limit=250&order=id+asc&since_id=0>
2022-10-17 14:50:14 [32mINFO[m i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):355 - Source has no more messages, closing connection.
2022-10-17 14:50:14 [43mdestination[0m > Airbyte message consumer: succeeded.

Oleksandr Bazarnov [GL]

10/17/2022, 4:20 PM

Please share the actual connector version, you was working with?

Eric Lamphere

10/17/2022, 4:49 PM

I see, I had to update the shopify connector version in settings. Sorry about that. Getting a different error now but I'll try to debug first

💪 1

Oleksandr Bazarnov [GL]

10/17/2022, 4:51 PM

Share the logs if needed

🙌 1

Eric Lamphere

10/17/2022, 5:29 PM

Seems to be an issue just with the

balance_transactions

stream. I was able to sync everything else. I'll send the error logs, just a sec

Eric Lamphere

10/17/2022, 5:31 PM

9fcfb364_b7bf_4a96_a501_434e341289ae_logs_13_txt.txt

👍 1

Oleksandr Bazarnov [GL]

10/17/2022, 6:21 PM

Should be fixed by this one: https://github.com/airbytehq/airbyte/pull/17962

Oleksandr Bazarnov [GL]

10/17/2022, 6:22 PM

It’s in review now

Eric Lamphere

10/17/2022, 6:32 PM

Great, thank you!

Eric Lamphere

10/18/2022, 1:29 PM

Hey @Oleksandr Bazarnov [GL], It looks like that pull request you sent above was merged. I upgraded the shopify source to

v0.1.39

but I'm still getting the same error I sent above and I'm not seeing the most recent version in the shopify docs changelog. Is there still something that needs to get deployed for those changes to propagate through?

👍 1

Eric Lamphere

10/18/2022, 1:34 PM

Here are the logs:

9fcfb364_b7bf_4a96_a501_434e341289ae_logs_19_txt.txt

👀 1

Oleksandr Bazarnov [GL]

10/18/2022, 2:19 PM

Got it, simple fix didn’t help( will do my best to catch and fix asap

keanu thanks 1

🙌 1

3 Views

Open in Slack

Previous Next