hey folks! Is it possible to get caching at the `W...
# caching
hey folks! Is it possible to get caching at the
level? I have a task which would benefit from using the worker API to parallelize things, but the caching granularity at the task level would be too broad, so currently I'm creating
tasks instead.
I don't think so. Afair you can cache the outputs of a whole task or the outputs of a whole artifact transform (of one artifact).
With configuration-cache tasks are paralelizable, maybe it's worth using tasks instead of worker API (Tricky, it only runs them in parallel if the configuration is cached)
If the tasks still use the worker api and don't explicitly wait for the result, they should also be run in parallel even without configuration cache enabled.
it isn't so obvious to run via tasks since new "work items" are discovered during execution. Using tasks I have to limit this to those I know upfront (which is already not trivial)
👍 1
This is among the plans, but not supported right now.
One reason for not doing this is the overhead each cached work item adds. If you have a task that runs 100 `WorkAction`s and each is cached separately, with the current cache protocol we’d be asking the cache server using 100 HTTP GET requests.
But we want to implement worker action-level caching for native compilation, too.
❤️ 4
I see, thanks for the update!
👍 1