Nguyen Dao (Bach)
10/29/2023, 4:37 PMkapa.ai
10/29/2023, 4:37 PMcsharp //Checks every 10 seconds the status of the job while(jobStatus == "running") { //Gets status of the job client = new RestClient("<https://api.airbyte.com/v1/jobs/>" + jobId);//We add the jobId request = new RestRequest(); request.AddHeader("accept", "application/json"); request.AddHeader("content-type", "application/json"); request.AddHeader("Authorization", "Bearer {YourApiKey}"); dynamic statusResponse = client.ExecuteGet(request); //Deserialize response to get the status of the job apiResponse = JsonConvert.DeserializeObject(statusResponse.Content); jobStatus = apiResponse.status; //Waits 10 seconds.. Thread.Sleep(10000); //Prints info in console depending on the status if (jobStatus == "running") { Console.WriteLine("Job running..."); } else { Console.WriteLine("Job Finished..."); Console.WriteLine(" "); Console.WriteLine("Sync details:"); Console.WriteLine(" JobId:" + apiResponse.jobId); Console.WriteLine(" Status:" + jobStatus); Console.WriteLine(" Rows/Bytes Synced:" + apiResponse.rowsSynced + "/" + apiResponse.bytesSynced); Console.WriteLine(" Sync Start Time:" + apiResponse.startTime); Console.WriteLine(" Sync Last Updated:" + apiResponse.lastUpdatedAt); Console.WriteLine(" Sync Duration:" + apiResponse.duration); } } Please replace {YourApiKey} with your actual API key and jobId with the ID of the job you want to monitor. This code is written in C# and uses the RestSharp and Newtonsoft.Json libraries. If you're using a different programming language, you'll need to adjust the code accordingly. Please note that the base URL in the RestClient constructor (`https://api.airbyte.com/v1/jobs/`) is an example. If you have deployed Airbyte on a GCP VM with a public IP, you should replace this with the appropriate base URL. For more details, you can refer to this [tutorial](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api).Nguyen Dao (Bach)
10/29/2023, 4:40 PMkapa.ai
10/29/2023, 4:41 PM<http://localhost:8000> as mentioned in the [Quickstart guide](https://docs.airbyte.com/quickstart/deploy-airbyte). However, if you have a public IP for your Airbyte deployment, you should replace localhost with your public IP address. So, the URL would be http://<your-public-ip>:8000. Please note that for security reasons, it's recommended not to expose Airbyte on Internet available ports. You can create an SSH tunnel for port 8000 as mentioned in the [AWS EC2 deployment guide](https://docs.airbyte.com/deploying-airbyte/on-aws-ec2). Also, Airbyte Open Source currently does not include any user management or role-based access controls (RBAC) to prevent unauthorized access to the API or UI. You can secure access to Airbyte using methods like deploying Airbyte in a private network, using a firewall, or deploying Airbyte behind a reverse proxy as mentioned in the [Security guide](https://docs.airbyte.com/operator-guides/security).Nguyen Dao (Bach)
10/29/2023, 4:41 PMkapa.ai
10/29/2023, 4:41 PMkapa.ai
10/29/2023, 4:42 PM<https://api.airbyte.com/v1/jobs/{jobId}>. Replace {jobId} with the actual ID of the job you want to monitor. However, if you have a public IP for your Airbyte deployment, you should replace <http://api.airbyte.com|api.airbyte.com> with your public IP address. So, the URL would be http://<your-public-ip>:8000/api/v1/jobs/{jobId}. Here is an example of how to use this API endpoint to check the job status: csharp //Checks every 10 seconds the status of the job while(jobStatus == "running") { //Gets status of the job client = new RestClient("http://<your-public-ip>:8000/api/v1/jobs/" + jobId);//We add the jobId request = new RestRequest(); request.AddHeader("accept", "application/json"); request.AddHeader("content-type", "application/json"); request.AddHeader("Authorization", "Bearer {YourApiKey}"); dynamic statusResponse = client.ExecuteGet(request); //Deserialize response to get the status of the job apiResponse = JsonConvert.DeserializeObject(statusResponse.Content); jobStatus = apiResponse.status; //Waits 10 seconds.. Thread.Sleep(10000); //Prints info in console depending on the status if (jobStatus == "running") { Console.WriteLine("Job running..."); } else { Console.WriteLine("Job Finished..."); Console.WriteLine(" "); Console.WriteLine("Sync details:"); Console.WriteLine(" JobId:" + apiResponse.jobId); Console.WriteLine(" Status:" + jobStatus); Console.WriteLine(" Rows/Bytes Synced:" + apiResponse.rowsSynced + "/" + apiResponse.bytesSynced); Console.WriteLine(" Sync Start Time:" + apiResponse.startTime); Console.WriteLine(" Sync Last Updated:" + apiResponse.lastUpdatedAt); Console.WriteLine(" Sync Duration:" + apiResponse.duration); } } Please replace <your-public-ip>, {jobId}, and {YourApiKey} with your actual public IP, job ID, and API key respectively. This information is based on the tutorial on [Create and Monitor a Data Pipeline Using the Airbyte API](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api).Nguyen Dao (Bach)
10/29/2023, 11:48 PM500 Server Error: Internal Server Error for url: http://<my_public_ip>:8006/v1/jobs?jobId=2648
when trying to get the status of the job 2648, why is that?kapa.ai
10/29/2023, 11:48 PMkapa.ai
10/29/2023, 11:48 PMhttp://<your-public-ip>:8000/api/v1/jobs/{{jobId}}. For more information on how to use the Airbyte API, you can refer to the [Airbyte API tutorial](https://airbyte.com/tutorials/create-and-monitor-a-data-pipeline-using-the-airbyte-api).Nguyen Dao (Bach)
10/30/2023, 4:27 AMkapa.ai
10/30/2023, 4:27 AMkapa.ai
10/30/2023, 4:27 AM/v1/jobs, then you should use http://<your-public-ip>:8006/v1/jobs/{{jobId}} to get the job status and details. Please replace <your-public-ip> and {{jobId}} with your actual public IP and job ID respectively. The error you're encountering could be due to a variety of reasons such as a server-side issue or a problem with the specific endpoint you're trying to access. I recommend checking the server logs for more detailed error messages. I apologize for any confusion caused and appreciate your understanding.Violet Kester
07/09/2024, 8:36 PM