At least not for hooks coming from gitlab. Maybe other SCMs can be configured with long timeouts.

@dhollinger??

as a workaround, did you try tuning r10k? download pool size, ssh control socket or using rugged and not shellgit?

I think the best we could do is make sure the command started.

I've got 265 environments, so I don't think any amount of tuning will help me here. But even with a single environment, if you're running r10k and generate types, the process startup time might make consistently sub-second unrealistic.

Should probably configure it to return 202 if using a worker/thread to deploy or 200 if it's waiting for completion

I haven't had time to work on it recently

surely 200 if it's completed.

Async work is slated for a future release, after I get packages, docs, and containers up

and hi!

🤔 Can you parallelize it by batches of environments? Doesn't really help with startup but might help it finish faster overall.

does it need a whole presumably complicated queuing system? What happens if you just move the

c.JSON(http.StatusAccepted, res)

(without the res!) to before the

execute

? I'm presuming the code that follows still gets executed? (eg the chatops stuff)

The problem is that some users of the Webhook tools rely on the return codes to trigger additional steps and want to know when it completes

I suppose we do need a proper queue, because we wouldn't want multiple webhooks to come in anyway. r10k will get pretty upset I think.

And not planning on an overly complicated queuing system, planning on just using goroutines

looks up goroutines

Yeah, you'd need queueing and system load monitoring to ensure that you don't destroy the system (particularly when running generate types)

So it's ok if the webhook API accepts more than one connection at once, so long as its just queuing work for something else.

maybe just return a queue_id and later there can be a separate API if users need to know if their deployment was successful???

How do you detect success?

All beyond my go knowledge though. I guess the advantage of go was a single binary to deploy. We don't want to introduce a proper backend database or redis etc.

(honestly, that one has baffled me for a while)

When I get a clean compile, it worked! Otherwise, so many things could have not worked 😞

Presumably just the exit code from r10k. (Although that's a bit vague as it is IIRC)

Yeah, it is but probably the best you can do

and maybe spool jobs to the filesystem, or an in memory db. Whatever is easiest I guess. Main thing is that I don't think it can be a gorouting per API call. It has to be a global background process/thread for all the queued webhooks as r10k processes really can't be allowed to run concurrently.

They can run concurrently as long as they point at different environments can't they?

From my work where I am at now, there will either need to have webhook: 1. Be single theaded and wait for completion everytime 2. Create a new way to have the goroutine monitor the running deploy and report back 3. Create a script/plugin for r10k/puppet server/bolt/choria that will send a notification message to the appropriate place on success or failure

@tvaughan That was my assumption

maybe?? But module deployments update all the environments that contain that module.