Hello! I’m new to Airbyte. I’m using the

HTTP Request

source but I’d like to have dynamic query parameters on it. Is that possible? For example, my url has

start_date

and

end_date

. I’m executing it once per day with

start_date

= d - 1 and

end_date

= current day. How can I have that in Airbyte?

Hey @Rodrigo Morais, the recommended approach for syncing data via an API would be to build a dedicated connector for it. Our CDK offers a great template for doing this and we'll be happy to help support! Which API are you getting data from out of curiosity 🙂 ?

Hello George, Thanks for answering. They are many APIs. My company has a product where our clients can inform the APIs and we collect data for them. We don’t know which APIs and what will be the format of them. We decide to use Airbyte behind of the scene. For sure the APIs will have query parameters and we need that flexible. We choose Airbyte because the connectors already exist.

Unfortenaly the HTTP is very simple and doesn't support any additional params in the query 😞

I've got an open issue you can vote for to add this! 😃

Unfortunately, I can’t wait for someone to develop that. Our product needs that right now. I’ll develop a connector with query parameters. Should I develop a new one or extend the

HTTP Request

?

If is something easy and you can control on your side, probably extending the HTTP Request connector can be more straightforward

For us the only dynamic query parameter are dates. We have other query parameters as strings, arrays, and numbers but they are not dynamic. This is the idea that I’ll follow: Query parameters: • Date ◦ Day ▪︎ Current ▪︎ Current day - n ▪︎ Current day + n ◦ Month ▪︎ Current ▪︎ Current month - n ▪︎ Current month + n ◦ Year ▪︎ Current ▪︎ Current year - n ▪︎ Current year + n • Period ◦ Start date ▪︎ Day • Current • Current day - n • Current day + n ▪︎ Month • Current • Current month - n • Current month + n ▪︎ Year • Current • Current year - n • Current year + n ◦ End date ▪︎ Start date + Day, Month, Year Type: [Date, Period] Format: [Day, Month, Year] Value: [Current, Current - n, Current + n] Examples:

<http://abc.gov/data?date={{date_1}}>
{"type": "date", "format": "day", "variable": date_1, "value": "current - 1"}
<http://abc.gov/data?from={{start_date}}&to={{end_date}}>
{"type": "period", "format": "day", "start_date": {"variable": "start_date", "value": "current - 1"}, "end_date": {"variable": "end_date", "value": "start_date + 1"}}

My idea is to have another parameter as

headers

. Every time that Airbyte executes the stream it has to get the correct dates. Does that is possible?

I think is possible, you probably can add this parameters in the spec.json