<@U04S8PLJWMT> in the custom source builder, I wan...
# replication-ask-ai
i
@kapa.ai in the custom source builder, I want to set up a JWT authentication with the
exp
payload property, how do I do that
k
The knowledge sources confirm that JWT auth is supported in both the Connector Builder UI and config-based YAML, but they do not explicitly show an example with the
exp
claim. However, they state that you can add arbitrary properties to the JWT payload via “JWT Payload” and “Additional JWT Payload,” so you can include
exp
there. [JWT auth (Builder); JWT Authenticator schema] In the Connector Builder UI: 1. Choose Authentication method = JWT. 2. Under JWT Payload, add the standard claims your API requires (e.g.
iss
,
sub
,
aud
). 3. To add
exp
, put it either: ◦ Directly in JWT Payload as a property named
exp
, or ◦ In Additional JWT Payload as a property named
exp
. Because the docs say “Additional JWT Payload – Extra properties to add to the JWT payload,” any key you define there (including
exp
) will be included in the signed JWT. [JWT auth (Builder)] In config-based YAML, the equivalent would look like:
Copy code
authenticator:
  type: JwtAuthenticator
  secret_key: "{{ config['secret_key'] }}"
  algorithm: RS256
  header_prefix: Bearer
  jwt_payload:
    iss: "{{ config['iss'] }}"
    sub: "{{ config['sub'] }}"
    aud: "{{ config['aud'] }}"
  additional_jwt_payload:
    exp: "{{ config['exp'] }}"
Here
exp
is added via
additional_jwt_payload
, which the schema explicitly allows for arbitrary extra payload properties. [JWT Authenticator schema] One limitation: the docs do not describe any built‑in helper to auto-calculate
exp
from `token_duration`; you must provide the value you want encoded (for example via config or a precomputed timestamp). 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
i
@kapa.ai I get this error:
Copy code
airbyte_cdk.sources.concurrent_source.stream_thread_exception.StreamThreadException: Exception while syncing stream concepts: Failed to sign token: 'iss', 'sub', 'aud', 'iat', 'exp', 'nbf' are reserved properties and should not be set as part of 'additional_jwt_payload'
k
The error is telling you that standard JWT claims like
exp
must be set in the main
jwt_payload
, not in
additional_jwt_payload
. From the schema,
jwt_payload
is where standard claims such as
iss
,
sub
, and
aud
live, and
additional_jwt_payload
is only for extra, non-reserved properties. [JWT Authenticator schema] So to fix your error: • Move
exp
(and any of
iss
,
sub
,
aud
,
iat
,
nbf
if you’re setting them) out of Additional JWT Payload and into JWT Payload. In Connector Builder UI: • Under JWT Payload, add a field named
exp
with the value you need. • Make sure
exp
is not present under Additional JWT Payload. In YAML (config-based) it should look like:
Copy code
authenticator:
  type: JwtAuthenticator
  secret_key: "{{ config['secret_key'] }}"
  algorithm: RS256
  header_prefix: Bearer
  jwt_payload:
    iss: "{{ config['iss'] }}"
    sub: "{{ config['sub'] }}"
    aud: "{{ config['aud'] }}"
    exp: "{{ config['exp'] }}"   # put exp here
  additional_jwt_payload:
    custom_claim: "some value"   # only non-reserved keys here
The docs don’t describe any automatic computation of
exp
, so you need to provide the actual value (e.g., a timestamp) yourself. [JWT Authenticator schema] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
i
@kapa.ai I get the following error: Client error : 400 Bad Request {"exceptionStack":"Traceback (most recent call last):\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/concurrent_declarative_source.py\", line 318, in _validate_source\n validate(self._source_config, self._declarative_component_schema)\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/jsonschema/validators.py\", line 1332, in validate\n raise error\njsonschema.exceptions.ValidationError: Additional properties are not allowed ('exp' was unexpected)\n\nFailed validating 'additionalProperties' in schema[4]['properties']['jwt_payload']:\n {'type': 'object',\n 'title': 'JWT Payload',\n 'description': 'JWT Payload used when signing JSON web token.',\n 'additionalProperties': False,\n 'properties': {'iss': {'type': 'string',\n 'title': 'Issuer',\n 'description': 'The user/principal that issued '\n 'the JWT. Commonly a value '\n 'unique to the user.',\n 'examples': [\"{{ config['iss'] }}\"]},\n 'sub': {'type': 'string',\n 'title': 'Subject',\n 'description': 'The subject of the JWT. '\n 'Commonly defined by the API.'},\n 'aud': {'type': 'string',\n 'title': 'Audience',\n 'description': 'The recipient that the JWT is '\n 'intended for. Commonly defined '\n 'by the API.',\n 'examples': ['appstoreconnect-v1']}}}\n\nOn instance['jwt_payload']:\n {'iss': 'Lm8uTLdqvTLFvjdXj5L5LZqIXwcODKhN', 'exp': 1772463498}\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/airbyte_cdk/connector_builder/main.py\", line 106, in <module>\n print(handle_request(sys.argv[1:]))\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/airbyte_cdk/connector_builder/main.py\", line 96, in handle_request\n source = create_source(config=config, limits=limits, catalog=catalog, state=state)\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/airbyte_cdk/connector_builder/connector_builder_handler.py\", line 78, in create_source\n return ConcurrentDeclarativeSource(\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/concurrent_declarative_source.py\", line 204, in __init__\n self._validate_source()\n File \"/home/airbyte/.pyenv/versions/3.10.18/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/concurrent_declarative_source.py\", line 320, in _validate_source\n raise ValidationError(\njsonschema.exceptions.ValidationError: Validation against json schema defined in declarative_component_schema.yaml schema failed\n","exceptionClassName":"io.airbyte.protocol.models.v0.AirbyteTraceMessage","message":"Error handling request: Validation against json schema defined in declarative_component_schema.yaml schema failed"}
k
You’re running into a schema validation error because, in the current JwtAuthenticator schema,
jwt_payload
only allows three specific properties:
iss
,
sub
, and
aud
, and explicitly sets
additionalProperties: false
. [JWT Auth schema] From the error: >
Failed validating 'additionalProperties' in schema['jwt_payload'] ... 'additionalProperties': False ... On instance['jwt_payload']: {'iss': '...', 'exp': 1772463498}
This means: • You cannot add
exp
directly under
jwt_payload
in the Connector Builder / declarative YAML, because the schema forbids any keys other than
iss
,
sub
, and
aud
there. [JWT Auth schema] • Reserved claims like
exp
,
iat
,
nbf
also cannot go into
additional_jwt_payload
(as you saw from the earlier “reserved properties” error). So, based on the current documented schema and behavior, there is no supported way to set
exp
yourself in a declarative / Connector Builder JWT configuration: •
jwt_payload
is restricted to
iss
,
sub
,
aud
only. •
additional_jwt_payload
cannot contain reserved claims such as
exp
,
iat
,
nbf
, etc. • The docs do not mention any configuration option to auto-generate
exp
from
token_duration
or similar; they only describe
token_duration
as “The amount of time in seconds a JWT token can be valid after being issued.” [JWT Authenticator ref] Given the knowledge sources, the options within the declarative / Builder path are: • Use only the supported standard claims (
iss
,
sub
,
aud
) in
jwt_payload
. • Use
additional_jwt_payload
only for non-reserved custom claims. • Rely on whatever internal behavior Airbyte has around
token_duration
(the docs do not state that it sets
exp
, so this is unclear). If your target API strictly requires an explicit
exp
claim and will not accept a token without it, the knowledge sources do not describe a declarative configuration that can satisfy this. In that case, you may need to: • Implement a custom (code-based) connector where you construct and sign the JWT yourself, including
exp
, or • Adjust the API configuration if it can accept tokens without
exp
. To summarize: the error is expected given the current JwtAuthenticator schema, and there is no documented, supported way in the Connector Builder / declarative JwtAuthenticator to add
exp
to the payload. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.