My http connection requires custom error handling. I have set up the

should_retry

method accordingly. Based on the logs

should_retry

is evaluating to

True

as expected given line 200 of the logs contains the

DefaultBackoffException

class, but the default retry behavior of retrying 5 times with exponential backoff does not seem to be kicking in. Has the default behavior changed, or is there something additional I need to do to enable it?

Hey @Will Watkinson do you mind sharing your stream class code?

Could you please try to set

max_retries = 5

in the class attribute to check if anything changes? I dug a bit in the CDK code and I don't think the usual logic changed...

yeah I dug through the code too and it seemed like max_retries should be defaulting to 5 as specified in the

HttpStream

class

Nice catch, thanks for digging in @Will Watkinson. I guess you solved this problem as we started another CDK conversation.

I resolved this temporarily by overwriting the 401 status code with a status code that would be retried. Also opened and issue and put up a PR to resolve it in the CDK. I couldn't sign the CLA, though