https://linen.dev logo
Join Slack
Powered by
im getting a 401 error when trying to setup OAuth ...
# help-connector-development
d
im getting a 401 error when trying to setup OAuth for a new stream, but I know the client_id and client_secret are good because they work in postman. id like to see what airbyte is actually sending as a request to the refresh endpoint, similar to how it will show the request when you're building a stream. is there a way to see that data? currently all i see is the error message itself.
šŸ‘ 1
j
You currently can't see those requests, it's a known issue: https://github.com/airbytehq/airbyte/issues/21014
šŸ‘ 1
Could you share your full manifest?
d
sure thing:
Copy code
version: 0.39.1
type: DeclarativeSource
check:
  type: CheckStream
  stream_names:
    - customers
streams:
  - type: DeclarativeStream
    name: customers
    primary_key: []
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: <http://json-schema.org/draft-07/schema#>
        additionalProperties: true
        properties: {}
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: <https://tandym-api.us.canopyservicing.com/>
        path: customers
        http_method: GET
        request_parameters: {}
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          client_secret: '{{ config[''client_secret''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: <https://tandym-api.us.canopyservicing.com/auth/token>
          grant_type: client_credentials
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path: []
      paginator:
        type: NoPagination
spec:
  connection_specification:
    $schema: <http://json-schema.org/draft-07/schema#>
    type: object
    required:
      - client_id
      - client_secret
    properties:
      client_id:
        type: string
        title: Client ID
        airbyte_secret: true
      client_secret:
        type: string
        title: Client secret
        airbyte_secret: true
    additionalProperties: true
  documentation_url: <https://example.org>
  type: Spec
the auth endpoint here requires client_id and client_secret in the body of the request. i thought i might need to modify the YAML for 
refresh_request_body
but didn't have any luck there either.
j
Adding client_id and client_secret to the request body is what the oauth authenticator is doing by default, that should work fine. Please double-check you are entering client id and secret into the right inputs in the testing values form without quotes and spaces (just the values)
The only other potential problem I can see is that the connector is adding another property to the request body:
Copy code
{
  "client_id": "{{client_id}}",
  "client_secret": "{{client_secret}}",
  "grant_type": "client_credentials"
}
That shouldn't cause any issues, but maybe it's tripping up this API. You can check whether this is the case by adding this additional property to the postman call as well and check whether it still works. I don't think this is the problem though
d
values are definitely getting entered correctly
ill double-check with postman that the extra data doesn't mess with the call, but i dont think that's it either
yeah, confirmed .. adding 
grant_type
to the API request in postman had no negative impact
j
That's really strange, I can't tell what's different. Unfortunately I don't have access to a canopy account so I can't test this myself. There is documentation talking about a public sandbox, but it's doesn't work for me: https://docs.canopyservicing.com/doc/upcoming-release#topic-getting-started
Do you know whether there is some other sandbox environment I could test with by any chance?
Not sure whether I'm looking in the right place, the "latest" documentation has a different path for authentication (
oauth/token
) https://docs.canopyservicing.com/doc/latest#authentication-o-auth2
Besides that, there is one other thing some APIs require - add a custom header 
User-Agent
and give it a value like 
airbyte-connector
or so:
It's guessing but I've seen other APIs causing problems if the user agent isn't set
d
let me give this a try
hmm, i think i may have figured something out .. it doesn't seem to require 
User-Agent
, but in Postman the auth request fails when I don't include 
Content-Type
. i know how to add request headers to the stream itself, but how do i add a request header to the oauth call? their public facing docs are hilariously bad, that oauth endpoint that you saw doesn't point to anything šŸ˜†
tried as such, but no luck
j
The headers are not applied to the authentication request. This might be the problem. @Alexandre Girard (Airbyte) what do you think about doing a quick-fix here and setting the content type header for this request? It should anyway be set
a
yup. that's a good idea
I'll submit a PR
šŸ™Œ 2
k
Ah yes please! I got a feeling my Oauth was failing on the same thing! Nice to see that Iā€™m not alone and that this is indeed the case.
d
@Alexandre Girard (Airbyte) and @Joe Reuter (Airbyte) .. just confirming that the recent fix to add 
content-type
to the header and typing it as json did indeed fix this issue. thanks!
šŸ™Œ 1
k
I want to try as well. @David Anderson, which version of Airbyte did you use?
d
most recent release .. 50.3
šŸ‘ 1
k
Nice, got it as well!
Open in Slack
PreviousNext
Powered by