extreme noob question re: calling the airbyte API ...
# replication-troubleshooting
d
extreme noob question re: calling the airbyte API with curl Hello everyone! Very new to airbyte and have been playing around with it today. I'm trying to call the API to trigger a sync manually but I can't work out the basics of a curl command to the API. Whatever I try from the CLI, I get the same HTML 401 page including the text "_This deployment of Airbyte is protected by HTTP Basic Authentication_" etc. From https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#overview I can see that with the default credentials, the auth header should be
'Authorization': "Basic YWlyYnl0ZTpwYXNzd29yZA=="
, so I've tried various combinations along the lines of:
curl --request POST localhost:8000/api/v1/connections/sync --header "Authorization: 'Basic YWlyYnl0ZTpwYXNzd29yZA=='" --data "{connectionId: abc123-def456-99999-99999}"
Can anyone point out the obvious flaw here? TIA!
I assume there's a bit of a mismatch in the docs too, because https://docs.airbyte.com/api-documentation/ says
Airbyte Open Source doesn't require an API Token for Authentication. All endpoints are acccessible without the token.
but the page it links to https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html says
When authenticating to the Configuration API, you must use Basic Authentication
etc. So do I need to use a token or not? Confused noob is confused.
m
Airbyte introduced basic nginx authorization in their latest versions. Use something like below. N/B: the authorization header is a base64 encoded
username:password
Copy code
curl --location --request POST '<http://localhost:8000/api/v1/connections/list>' \
--header 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=' \
--header 'Content-Type: application/json' \
--data-raw '{
    "workspaceId": "095869d2-7fee-46bf-936e-300e6e827011"
}'
P:S, You can use postman too with basic auth
n
Dan, let me know if Macharia's comment helps!
d
It did - thank you @Macharia Muguku and @Nataly Merezhuk (Airbyte)! Now I have another question but i'll post that separately... 😄
keanu thanks 1
c
@kapa.ai I want to get information such as job running time, number of records, status in a connection, which API of AirByte should I use?
k
To get information such as job running time, number of records, and status in a connection, you can use the Airbyte API. Specifically, you can use the
/v1/jobs/list
endpoint to fetch all the jobs ran for a specific connection. This was mentioned in a [forum discussion](https://discuss.airbyte.io/t/2235) on the Airbyte community forum. Here is an example of how you can monitor a job using the Airbyte API, as shown in this [tutorial](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api):
Copy code
csharp while(jobStatus == "running") {  //Gets status of the job  client = new RestClient("<https://api.airbyte.com/v1/jobs/>" + jobId);//We add the jobId  request = new RestRequest();  request.AddHeader("accept", "application/json");  request.AddHeader("content-type", "application/json");  request.AddHeader("Authorization", "Bearer {YourApiKey}");  dynamic statusResponse = client.ExecuteGet(request);   //Deserialize response to get the status of the job  apiResponse = JsonConvert.DeserializeObject(statusResponse.Content);  jobStatus = apiResponse.status;   //Waits 10 seconds..  Thread.Sleep(10000);   //Prints info in console depending on the status  if (jobStatus == "running")  {  Console.WriteLine("Job running...");  }  else  {  Console.WriteLine("Job Finished...");  Console.WriteLine(" ");  Console.WriteLine("Sync details:");  Console.WriteLine(" JobId:" + apiResponse.jobId);  Console.WriteLine(" Status:" + jobStatus);  Console.WriteLine(" Rows/Bytes Synced:" + apiResponse.rowsSynced + "/" + apiResponse.bytesSynced);  Console.WriteLine(" Sync Start Time:" + apiResponse.startTime);  Console.WriteLine(" Sync Last Updated:" + apiResponse.lastUpdatedAt);  Console.WriteLine(" Sync Duration:" + apiResponse.duration);  } }
Please replace
{YourApiKey}
with your actual API key. This code checks the status of the job every 10 seconds and prints the job details once the job is finished. You can find more details about the Airbyte API in the [API documentation](https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#servers).