I tried , but it seems like all the questions are getting shoot to one service only.
I tried these things
step 1: if promptfooconfig.yaml has 1 question questions then for that start the provider service once and stop it
step 2: if promptfooconfig.yaml has multiple question then start provider service for question 1 then stop it then for next question start provider service and then stop, like this it should do.
what i observed , once provider service is getting started then all the questions are getting shoot to single provider.
class ServiceProvider {
constructor(options) {
this.providerId = 'Documentation Service Provider';
this.config = options.config;
this.csharpProcess = null;
}
id() {
return this.providerId;
}
async startCSharpExecutable() {
const command = process.platform === 'win32' ? 'start' : 'open';
const executablePath = 'providerService.exe'; // Update this path
this.csharpProcess = spawn(command, [executablePath], { shell: true, detached: true });
// Implement a way to check if the application has started. This could be a simple delay, a log message check, or a port check.
await this.waitForServiceStart(7000); // Example: waiting for a service to start on port 7000
}
async callApi(prompt) {
// console.log(prompt)
var isServiceRunning = await this.isServiceRunningOnPort(7000);
if (isServiceRunning) {
await this.stopCSharpExecutable();
}
await this.startCSharpExecutable();