Hello, I work on an offline server and used to run...
# sarek
s
Hello, I work on an offline server and used to run Sarek 3.4.4, and now trying to run version 3.9.0 and nextflow-25.10.6-dist. I have copied all the plugins, etc to the offline server after test running this version on the dev system. I am running with these params below and it keeps crashing at the validation step (on the offline server). Is there any other params that I can add to stop it from checking these online files?
Copy code
igenomes_ignore : true
no_intervals : true
fasta : '/path/to/fasta'
ERROR:
Copy code
The following invalid input values have been detected:

* --igenomes_base (<s3://ngi-igenomes/igenomes/>): could not validate file format of '<s3://ngi-igenomes/igenomes/>': Unable to execute HTTP request: Connect to <http://ngi-igenomes.s3.us-east-1.amazonaws.com:443|ngi-igenomes.s3.us-east-1.amazonaws.com:443> [<http://ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.244.238|ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.244.238>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.236.217|ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.236.217>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.245.15|ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.245.15>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/52.217.121.226|ngi-igenomes.s3.us-east-1.amazonaws.com/52.217.121.226>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.199.32|ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.199.32>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.223.39|ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.223.39>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.245.84|ngi-igenomes.s3.us-east-1.amazonaws.com/16.15.245.84>, <http://ngi-igenomes.s3.us-east-1.amazonaws.com/52.217.115.82]|ngi-igenomes.s3.us-east-1.amazonaws.com/52.217.115.82]> failed: Connect timed out (SDK Attempt Count: 6)
* --snpeff_cache (<s3://annotation-cache/snpeff_cache/>): could not validate file format of '<s3://annotation-cache/snpeff_cache/>': Unable to execute HTTP request: Connect to <http://annotation-cache.s3.us-east-1.amazonaws.com:443|annotation-cache.s3.us-east-1.amazonaws.com:443> [<http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.252.222|annotation-cache.s3.us-east-1.amazonaws.com/16.15.252.222>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.183.240|annotation-cache.s3.us-east-1.amazonaws.com/16.15.183.240>, <http://annotation-cache.s3.us-east-1.amazonaws.com/52.217.118.178|annotation-cache.s3.us-east-1.amazonaws.com/52.217.118.178>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.253.218|annotation-cache.s3.us-east-1.amazonaws.com/16.15.253.218>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.247.141|annotation-cache.s3.us-east-1.amazonaws.com/16.15.247.141>, <http://annotation-cache.s3.us-east-1.amazonaws.com/52.217.90.184|annotation-cache.s3.us-east-1.amazonaws.com/52.217.90.184>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.229.212|annotation-cache.s3.us-east-1.amazonaws.com/16.15.229.212>, <http://annotation-cache.s3.us-east-1.amazonaws.com/52.217.86.120]|annotation-cache.s3.us-east-1.amazonaws.com/52.217.86.120]> failed: Connect timed out (SDK Attempt Count: 6)
* --vep_cache (<s3://annotation-cache/vep_cache/>): could not validate file format of '<s3://annotation-cache/vep_cache/>': Unable to execute HTTP request: Connect to <http://annotation-cache.s3.us-east-1.amazonaws.com:443|annotation-cache.s3.us-east-1.amazonaws.com:443> [<http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.252.171|annotation-cache.s3.us-east-1.amazonaws.com/16.15.252.171>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.191.132|annotation-cache.s3.us-east-1.amazonaws.com/16.15.191.132>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.223.16|annotation-cache.s3.us-east-1.amazonaws.com/16.15.223.16>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.237.67|annotation-cache.s3.us-east-1.amazonaws.com/16.15.237.67>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.245.111|annotation-cache.s3.us-east-1.amazonaws.com/16.15.245.111>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.207.152|annotation-cache.s3.us-east-1.amazonaws.com/16.15.207.152>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.253.15|annotation-cache.s3.us-east-1.amazonaws.com/16.15.253.15>, <http://annotation-cache.s3.us-east-1.amazonaws.com/16.15.255.229]|annotation-cache.s3.us-east-1.amazonaws.com/16.15.255.229]> failed: Connect timed out (SDK Attempt Count: 6)

 -- Check script /shared/pipelines/sarek/nf-core-sarek_3.9.0/3_9_0/subworkflows/nf-core/utils_nfschema_plugin/main.nf' at line: 72 or see '.nextflow.log' file for more details
Thanks!
p
This is the schema helper to validate that your parameters make sense, could you either point them to some local resource or run with a configuration setting
Copy code
validation.ignoreParams = ['igenomes_base', 'vep_cache']
?
s
Thanks for your suggestions, I'll try that. I wonder why it should validate these values when I'm not going to use them at all? I have igenomes_ignore : true so why it should need to chcek the base path at all? same for the cache paths. I only run the alignment part on my data and the rest is off. it should be more flexible
p
One might think so, but then you'd have all sorts of logic trying to figure out what parts of
params
should be validated rather than just relying on things not used getting the defaults (that normally validate just fine).
I don't recall exactly how nf-schema works (it may be possible to skip some validation in some cases without using
ignoreParams
), but in general, such "easy" things can be messy and time consuming without helping much spending time on.
s
yes, there must be a good reason for that 😁