Hello again, I'm trying to understand the cloud ca...
# nextflow-plugins
j
Hello again, I'm trying to understand the cloud cache plugin. I can easily use
nf-cloudcache
with a remote
NXF_CLOUDCACHE_PATH
to cache sessions to a remote bucket. This is using nextflow version 23.10.1.5891 with nf-cloudcache-0.3.0. Where I'm getting confused is by the apparent dependence on the
./.nextflow/history
file if a user makes a new VM to resume a nextflow workflow some other VM started, ideally they can just do the following:
Copy code
export NXF_CLOUDCACHE_PATH=<gs://my-bucket/remote-caches/>
nextflow run . -profile remote -plugins nf-cloudcache -w '<gs://my-bucket/cache-test/workdir/>' -resume bdacbb36-10e9-4214-a1d2-f9ee4be2d091
but in reality, it seems like I'm required to update/fake the new VMs history prior the
nextflow run ..
with some text saved to
./.nextflow/history
like:
Copy code
2024-03-27 14:40:56	6m 56s	jolly_maxwell	OK	f51eaf47bb162ef4fe528c23c35b4238	bdacbb36-10e9-4214-a1d2-f9ee4be2d091	nextflow run . -profile remote -plugins nf-cloudcache -w '<gs://my-bucket/cache-test/workdir/>'
<empty EOF line required here>
Does anyone know if this is intended behavior or if I'm missing something obvious? edit: resolved
Copy code
export NXF_IGNORE_RESUME_HISTORY=true
+ adding
-name jolly_maxwell
to my nextflow run cmd seems to have worked to ignore a history lookup and only use cloud storage to cache restore on a new VM 👍
u
Hi John, I think we may have the same issue, opened a github discussion on this a few days ago: https://github.com/nextflow-io/nextflow/discussions/4876