Maybe your remote cache size is too small and is purging all those artifacts by the time you try to reuse them? Have you checked your cache node size and/or tried to increase it already?
Also note that there is a
configurable maximum size per artifact, it might be that those artifacts are rejected (I assume that you should see this in the logs, not sure what is the response in such cases during the storing attempt)