Implementation strategies for long-running process...
# workers-helpm
mjswensen
04/30/2023, 12:33 AMAt a high level, which combination of Workers features would be the best approach for the following use case?
- A long-running process that needs to happen in response to a user action.
- The process takes about 1-2 minutes to complete, which is longer than the 30s HTTP request CPU time limit, but it does fit into the 128MB memory limit.
I have been reading through the docs and have had some ideas, but I'm not sure if they are optimal or possible:
1. On the user's action, send an HTTP request to a Worker, which creates a Durable Object to handle the long process. Subsequent HTTP requests could poll the Durable Object to check for progress on the task. I'm not sure if there is a way to keep a Durable Object running in the background, outside of an active HTTP request?
2. On the user's action, send an HTTP request to a Worker that inserts a message into a Queue. The queue handler would do the long process work. I'm not sure if the queue handler would be limited to 30s as well, or if it would be subject to the 15-minute limit instead?
Am I on track or are there better approaches that I'm not thinking of?
k
kian
04/30/2023, 12:34 AMIs your process going to use 30 seconds of CPU?
kian
04/30/2023, 12:35 AMThat’s a limit of CPU usage specifically, duration has no limit but you have a chance of eviction beyond 30s which is rare but that’s because 30s is your grace period during runtime updates
m
mjswensen
04/30/2023, 12:35 AMYes, it will use 1-2 minutes of CPU time.
mjswensen
04/30/2023, 12:37 AMFrom https://developers.cloudflare.com/workers/platform/limits/#worker-limits I see that workers triggered by cron have CPU limits of 15m. Is it possible to trigger a worker in response to a user action that has that 15m limit, or is cron the only option?
mjswensen
04/30/2023, 12:45 AMIdeally, I'd be able to achieve something like this:
1. User action HTTP request kicks off a long-running background process and returns a response immediately to the user (
200 OKmjswensen
04/30/2023, 12:53 AMI think I found it—looks like queue consumers have the 15-minute limit: https://developers.cloudflare.com/workers/platform/limits/#cpu-runtime
So I should be able to go with option 2 in the original post. 👍
mjswensen
05/02/2023, 6:11 PMUpdate for future readers: It turns out that Queue messages don't get picked up immediately, and it's difficult to get observability on a message's status, and there doesn't appear to be a way to view consumer error logs... may be simply be some rough edges during the beta phase, but I'll end up going with a different vendor/solution for now.