Get Job status and details using Airbyte API with ...
# replication-ask-ai
n
Get Job status and details using Airbyte API with OSS AIrbyte deployed on GCP VM with a public IP
k
To get the job status and details using the Airbyte API, you can use the following code snippet. This code checks the status of the job every 10 seconds and prints the job details once the job is finished.
Copy code
csharp //Checks every 10 seconds the status of the job while(jobStatus == "running") {  //Gets status of the job  client = new RestClient("<https://api.airbyte.com/v1/jobs/>" + jobId);//We add the jobId  request = new RestRequest();  request.AddHeader("accept", "application/json");  request.AddHeader("content-type", "application/json");  request.AddHeader("Authorization", "Bearer {YourApiKey}");  dynamic statusResponse = client.ExecuteGet(request);   //Deserialize response to get the status of the job  apiResponse = JsonConvert.DeserializeObject(statusResponse.Content);  jobStatus = apiResponse.status;   //Waits 10 seconds..  Thread.Sleep(10000);   //Prints info in console depending on the status  if (jobStatus == "running")  {  Console.WriteLine("Job running...");  }  else  {  Console.WriteLine("Job Finished...");  Console.WriteLine(" ");  Console.WriteLine("Sync details:");  Console.WriteLine(" JobId:" + apiResponse.jobId);  Console.WriteLine(" Status:" + jobStatus);  Console.WriteLine(" Rows/Bytes Synced:" + apiResponse.rowsSynced + "/" + apiResponse.bytesSynced);  Console.WriteLine(" Sync Start Time:" + apiResponse.startTime);  Console.WriteLine(" Sync Last Updated:" + apiResponse.lastUpdatedAt);  Console.WriteLine(" Sync Duration:" + apiResponse.duration);  } }
Please replace
{YourApiKey}
with your actual API key and
jobId
with the ID of the job you want to monitor. This code is written in C# and uses the RestSharp and Newtonsoft.Json libraries. If you're using a different programming language, you'll need to adjust the code accordingly. Please note that the base URL in the RestClient constructor (`https://api.airbyte.com/v1/jobs/`) is an example. If you have deployed Airbyte on a GCP VM with a public IP, you should replace this with the appropriate base URL. For more details, you can refer to this [tutorial](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api).
n
@kapa.ai What should be the URL when the Airbyte is OSS deployed and have its public IP?
k
When Airbyte is deployed as an open-source solution, you can access it via the URL
<http://localhost:8000>
as mentioned in the [Quickstart guide](https://docs.airbyte.com/quickstart/deploy-airbyte). However, if you have a public IP for your Airbyte deployment, you should replace
localhost
with your public IP address. So, the URL would be
http://<your-public-ip>:8000
. Please note that for security reasons, it's recommended not to expose Airbyte on Internet available ports. You can create an SSH tunnel for port 8000 as mentioned in the [AWS EC2 deployment guide](https://docs.airbyte.com/deploying-airbyte/on-aws-ec2). Also, Airbyte Open Source currently does not include any user management or role-based access controls (RBAC) to prevent unauthorized access to the API or UI. You can secure access to Airbyte using methods like deploying Airbyte in a private network, using a firewall, or deploying Airbyte behind a reverse proxy as mentioned in the [Security guide](https://docs.airbyte.com/operator-guides/security).
n
@kapa.ai What should be the URL to get the jobs status and details using API when the Airbyte is OSS deployed and have its public IP?
k
A message from kapa.ai
To get the job status and details using the Airbyte API, you would use the following URL:
<https://api.airbyte.com/v1/jobs/{jobId}>
. Replace
{jobId}
with the actual ID of the job you want to monitor. However, if you have a public IP for your Airbyte deployment, you should replace
<http://api.airbyte.com|api.airbyte.com>
with your public IP address. So, the URL would be
http://<your-public-ip>:8000/api/v1/jobs/{jobId}
. Here is an example of how to use this API endpoint to check the job status:
Copy code
csharp //Checks every 10 seconds the status of the job while(jobStatus == "running") {  //Gets status of the job  client = new RestClient("http://<your-public-ip>:8000/api/v1/jobs/" + jobId);//We add the jobId  request = new RestRequest();  request.AddHeader("accept", "application/json");  request.AddHeader("content-type", "application/json");  request.AddHeader("Authorization", "Bearer {YourApiKey}");  dynamic statusResponse = client.ExecuteGet(request);   //Deserialize response to get the status of the job  apiResponse = JsonConvert.DeserializeObject(statusResponse.Content);  jobStatus = apiResponse.status;   //Waits 10 seconds..  Thread.Sleep(10000);   //Prints info in console depending on the status  if (jobStatus == "running")  {  Console.WriteLine("Job running...");  }  else  {  Console.WriteLine("Job Finished...");  Console.WriteLine(" ");  Console.WriteLine("Sync details:");  Console.WriteLine(" JobId:" + apiResponse.jobId);  Console.WriteLine(" Status:" + jobStatus);  Console.WriteLine(" Rows/Bytes Synced:" + apiResponse.rowsSynced + "/" + apiResponse.bytesSynced);  Console.WriteLine(" Sync Start Time:" + apiResponse.startTime);  Console.WriteLine(" Sync Last Updated:" + apiResponse.lastUpdatedAt);  Console.WriteLine(" Sync Duration:" + apiResponse.duration);  } }
Please replace
<your-public-ip>
,
{jobId}
, and
{YourApiKey}
with your actual public IP, job ID, and API key respectively. This information is based on the tutorial on [Create and Monitor a Data Pipeline Using the Airbyte API](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api).
n
@kapa.ai I got the error:
Copy code
500 Server Error: Internal Server Error for url: http://<my_public_ip>:8006/v1/jobs?jobId=2648
when trying to get the status of the job 2648, why is that?
k
A message from kapa.ai
Hmm, I don't know enough to give you a confident answer yet. The error message you're seeing typically indicates that there's an issue on the server side. It could be due to a variety of reasons such as a problem with the server configuration, an issue with the application running on the server, or even a problem with the specific endpoint you're trying to access. However, based on the information provided in the extracts, I can't determine the exact cause of this error. I would recommend checking the server logs for more detailed error messages which might give you a clue as to what's going wrong. Also, please ensure that you're using the correct port number in the URL. In the extracts, the port number used for Airbyte is 8000, but in your error message, it's 8006. The correct URL should be
http://<your-public-ip>:8000/api/v1/jobs/{{jobId}}
. For more information on how to use the Airbyte API, you can refer to the [Airbyte API tutorial](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api).
n
@kapa.ai As the docs saying, the default port for Airbyte API server is 8006, and the endpoint should be /v1/jobs, why are you recommended the port 8000 and the endpoint /api/v1/jobs ?
k
A message from kapa.ai
I apologize for the confusion. The information provided in the extracts does not specify the default port for the Airbyte API server as 8006. The extracts mention the use of port 8000 for accessing the Airbyte UI. If the Airbyte documentation specifies that the default port for the API server is 8006 and the endpoint is
/v1/jobs
, then you should use
http://<your-public-ip>:8006/v1/jobs/{{jobId}}
to get the job status and details. Please replace
<your-public-ip>
and
{{jobId}}
with your actual public IP and job ID respectively. The error you're encountering could be due to a variety of reasons such as a server-side issue or a problem with the specific endpoint you're trying to access. I recommend checking the server logs for more detailed error messages. I apologize for any confusion caused and appreciate your understanding.
v
@Nguyen Dao (Bach) 👋 Were you able to resolve this issue? I have a similar question.