https://linen.dev logo
Join Slack
Powered by
Hi, given the recent announcement <https://airbyt...
# airbyte-api
c
Hi, given the recent announcement https://airbyte.com/blog/airbytes-official-api-and-terraform-provider-now-in-open-source . I'd like to start testing this in my OSS deployment (was checking Octavia before). In what URL can I access the 
api
. For example if I want to call the 
sources
API endpoint, what is the URL? given local example where my current test webserver Airbyte is on http://localhost:8092/workspaces/9b64ff53-d970-4e32-85f3-cfcd9491fd96/connections . thanks,
t
the terraform provider talks to the new airbyte api server, which you can make sure is set up by following the steps in the read me here: https://github.com/airbytehq/airbyte-platform/tree/main/airbyte-api-server. Mainly to note that if you’re running with kubernetes, the default is for the airbyte-api-server to not be enabled (we’re planning on changing the default to enabled soon). For hitting the api directly: You should be able to hit 
<http://localhost:8006/v1/><endpoint>
. You may have to set up port forwarding if you’re running with a kubernetes instance. If running with docker-compose, you’ll need to add a basic auth header with your username/password since requests go through the airbyte-proxy. For terraform use cases: Your provider settings should look like this
Copy code
provider "airbyte" {
  server_url = "<http://localhost:8006/v1/>"

  // the below are only required if you're running through docker-compose using the airbyte-proxy and the actual password/username are the ones you've set (or the defaults)
  password = "password"
  username = "airbyte"
}
c
that is great Terence thanks a lot.. I am indeed using Kubernetes.. will give that a try and come back if anything!
a
Thanks a lot, was looking for this answer for weeks haha
c
this works beautifully 🙂 thanks
t
Thanks for the feedback! I’ll put this information up on the api reference documentation link so it’ll be easier to find
s
@Carlo Scarioni Did the port forwarding work? How does your 
server_url
look like ?
I am running into the error - 
You need to enable JavaScript to run this app
c
it did work yes .. my port forwarding was:
Copy code
kubectl port-forward svc/airbyte-local-airbyte-api-server-svc  8093:80 -n airbyte
then in my terraform provider I just had :
Copy code
provider "airbyte" {
  server_url = "<http://localhost:8093/v1/>"
  password = "password"
  username = "airbyte"
}
all worked fine (created a source)
s
Hello I am new to terraform and airbyte. Is it possible for someone to share a sample main.tf file with me (with sensitive info removed ofc) ? It'd be of great help as I'm running into a lot of errors just to even create a source/destination.
c
your main.tf can be as simple as :
Copy code
terraform {
  required_providers {
    airbyte = {
      source  = "airbytehq/airbyte"
      version = "0.3.2"
    }
  }
}

provider "airbyte" {
  server_url = "<http://localhost:8093/v1/>"
  password = "password"
  username = "airbyte"
}
resource "airbyte_source_stripe" "my_source_stripe2" {
  configuration = {
    source_type = "stripe"
    account_id = "acct_123"
    client_secret = "sklive_abc"
    start_date = "2023-07-01T00:00:00Z"
    lookback_window_days = 0
    slice_range = 365
  }
  name = "Stripe 2"
  workspace_id = 'your-workspace-id'
}
s
yes i did. I tried the pokeAPI source, and I got 
object not found
error. my main.tf is as follows:
Copy code
terraform {
  required_providers {
    airbyte = {
      source = "airbytehq/airbyte"
      version = "0.3.3"
    }
  }
}

provider "airbyte" {

    server_url = "<http://localhost:8000/api/v1/sources>"
}

resource "airbyte_source_pokeapi" "my_source_pokeapi" {
  name         = "bulbasaur"
  workspace_id = "ac3574df-2318-4dfd-be64-0f5bef320270"
  configuration = {
    pokemon_name = "bulbasaur"
    source_type  = "pokeapi"
  }
}
no credentials because I have disabled them in .env file
response:
Copy code
Error: unexpected response from API. Got an unexpected response code 404
│ 
│   with airbyte_source_pokeapi.my_source_pokeapi,
│   on <http://main.tf|main.tf> line 15, in resource "airbyte_source_pokeapi" "my_source_pokeapi":
│   15: resource "airbyte_source_pokeapi" "my_source_pokeapi" {
│ 
│ **Request**:
│ POST /api/v1/sources HTTP/1.1
│ Host: localhost:8000
│ Accept: application/json
│ Authorization: Basic Og==
│ Content-Type: application/json
│ User-Agent: speakeasy-sdk/terraform 0.3.3 2.86.10 1.0.0
│ 
│ {"configuration":{"pokemon_name":"bulbasaur","sourceType":"pokeapi"},"name":"bulbasaur","workspaceId":"ac3574df-2318-4dfd-be64-0f5bef320270"}
│ 
│ **Response**:
│ HTTP/1.1 404 Not Found
│ Content-Length: 17
│ Connection: keep-alive
│ Content-Type: application/json
│ Date: Thu, 24 Aug 2023 11:22:47 GMT
│ Server: nginx/1.25.2
│ 
│ Object not found.
c
your URL doesnt look right. you need to deploy the api-server. go to the beginning of this thread and have a read at @Terence Cho (Airbyte) comment
s
Ok will check. Thanks 👍
t
@Jeevan @yuvraaj This thread will be helpful for our deployment.
s
I managed to solve this. Like Carlo mentioned, I was using an incorrect port for the airbyte-api-server.
t
message has been deleted
@Carlo Scarioni @Terence Cho (Airbyte) If we are deploying this Airbyte OSS via Docker compose, is there a documentation available? We are going to deploy this via Google Cloud using the docker-compose? "You should be able to hit 
<http://localhost:8006/v1/><endpoint>
. You may have to set up port forwarding if you’re running with a kubernetes instance. If running with docker-compose, you’ll need to add a basic auth header with your username/password since requests go through the airbyte-proxy."
Does this help for deployment ? Docker Compose deployment files- https://github.com/airbytehq/airbyte-platform/tree/main/airbyte-api-server
g
What is the link if we deploy the official helm chart 0.48.2 via Argo CD and airbyte-api-server.enabled True with webapp.api.url /api/v1/
We tried *host*/api/v1/sources but it says that Object Not Found
c
hey @Göktuğ Aşcı as mentioned 
/api/v1/sources
doesn't look like the right path. that doesnt look like an API path. That looks like you trying to access the old config api in the normal UI web app
g
@Carlo Scarioni thank you for the response. There is only one ingress config in the helm chart. How will we expose the api endpoint?
c
Terence explained in the first reply to this messages. basically you need to enable
Copy code
airbyte-api-server:
         enabled: "true"
g
Thank you for the answer but when I enable this parameter then what url will it use? airbyte.floy.com/api/v1/ or airbyte.floy.com/v1/ ? Will there be a workspace suffix in the endpoints? I couldn't list the sources?
a
did you try first by port-forwarding the api server service and then doing your requests on localhost ? @Göktuğ Aşcı
g
We use the official helm chart version 0.48.2 Should we configure two ingress paths one for the web app and another for the api?
a
I would say yes as it is different services
but first try using port-forwarding maybe
then you know if it’s working at least haha
g
Haha, thanks for the tip 🙂
Open in Slack
PreviousNext
Powered by