https://linen.dev logo
Join Slack
Powered by
Hi everyone, I'm building my first custom connect...
# help-connector-development
b
Hi everyone, I'm building my first custom connector on Airbyte for the Crisp Chat API using the manifest.yaml file but i'm blocked and haven't found the solution yet, thus asking for your help 🙏 The API i'm requesting has a path for the stream looking like : 
<base_url>/conversations/<page_number>?<request_parameters>
I'm looking for a way to have an incremental paginator for the page_number which is not a request parameter but in the path, and which would increment itself as long as there are still results in the json response Thank you !
k
A message from kapa.ai
New message text here
j
are you using the connector builder?
b
No, just by "filling" the manifest.yaml file manually
j
got it
So you can do this by using jinja interpolation in the 
path
property of the requester referencing the 
next_page_token
variable: https://docs.airbyte.com/connector-development/config-based/understanding-the-yaml-file/reference/#/variables/next_page_token
Along with the page increment pagination strategy
b
thank you very much!
Sorry to disturb, I still have a question for this connector I have a request parameter as a datetime (format %Y-%m-%dT%H:%M:%S.%fZ) used by incremental sync But the cursor field used is saved as a milliseconds timestamp Is there a way using the manifest.yaml to cast the cursor_value so that the state is saved as a datetime formatted like above ?
j
Right now in the incoming and outgoing format has to be identical unfortunately. @Maxime Carbonneau-Leclerc I remember this came up before, did you create an issue for this? If not I can create one, it feels like one of the things we should just support
m
We indeed assume that all the datetime values sent/returned by the API have the same format. @Joe Reuter (Airbyte) we did not create a GitHub issue for this so feel free to do so!
b
Hi! Do you know if such an issue has been created and will be treated "soon" ? Or should i build my own custom connector for now ? (it would not be a problem, just to have an idea) Thank you for your work ! 🙂
m
It seems like it’s already out based on https://github.com/airbytehq/airbyte/issues/27153! I can see it in the Connector Builder UI on cloud so you should be good. Let me know if there is something amist
Open in Slack
PreviousNext
Powered by