https://nf-co.re logo
Join Slack
Powered by
Hi all! I tried to add --spades_options '--only-as...
# mag
r
Hi all! I tried to add --spades_options '--only-assembler' to my command, but it gives a nextflow error what would be the correct way to add the --spades_options? I tried:
Copy code
-spades_options '--only-assembler' and -spades_options "--only-assembler"
j
What is the nextflow error and the full command you used?
For example based on your code blockI think you are missing an extra `-`right at the beginning
r
oh sorry, I actually run 
--spades_options '--only-assembler'
j
And you may also need a 
=
between the mag parameter and the spades option itself
--spades_option='--only-assembler'
Something like that
r
oh let me try that!
j
If that works let me know, we should update the parameters docs if so
r
Will do! Running without spades now and will try that later!
👍 1
I did this because it seems like spades was stuck at the Read Error Correction, it seems that 1.fastq.gz was done in 30 minutes, but 2.fastq.gz was stuck there for almost a day. Have you encountered this problem before?
Copy code
okay I think this is the second run:

cat spades.log
Command line: /opt/conda/bin/spades.py	--meta	--threads	40	--memory	450	-1	/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_1.fastq.gz	-2	/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_2.fastq.gz	-o	/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2

System information:
  SPAdes version: 4.1.0
  Python version: 3.13.2
  OS: Linux-4.18.0-553.22.1.el8_10.x86_64-x86_64-with-glibc2.39

Output dir: /project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
  Metagenomic mode
  Reads:
    Library number: 1, library type: paired-end
      orientation: fr
      left reads: ['/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_1.fastq.gz']
      right reads: ['/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_2.fastq.gz']
      interlaced reads: not specified
      single reads: not specified
      merged reads: not specified
Read error correction parameters:
  Iterations: 1
  PHRED offset will be auto-detected
  Corrected reads will be compressed
Assembly parameters:
  k: [21, 33, 55]
  Repeat resolution is enabled
  Mismatch careful mode is turned OFF
  MismatchCorrector will be SKIPPED
  Coverage cutoff is turned OFF
  Assembly graph output will use GFA v1.2 format
Other parameters:
  Dir for temp files: /project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/tmp
  Threads: 40
  Memory limit (in Gb): 450


======= SPAdes pipeline started. Log can be found here: /project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/spades.log

/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_1.fastq.gz: max reads length: 150
/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_2.fastq.gz: max reads length: 150

Reads length: 150


===== Before start started.


===== Read error correction started.


===== Read error correction started.


== Running: /opt/conda/bin/spades-hammer /project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/corrected/configs/config.info

  0:00:00.000     1M / 21M   INFO    General                 (main.cpp                  :  76)   Starting BayesHammer, built from N/A, git revision N/A
  0:00:00.013     1M / 21M   INFO    General                 (main.cpp                  :  77)   Loading config from "/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/corrected/configs/config.info"
  0:00:00.025     1M / 21M   INFO    General                 (main.cpp                  :  79)   Maximum # of threads to use (adjusted due to OMP capabilities): 40
  0:00:00.031     1M / 21M   INFO    General                 (memory_limit.cpp          :  55)   Memory limit set to 450 Gb
  0:00:00.033     1M / 21M   INFO    General                 (main.cpp                  :  87)   Trying to determine PHRED offset
  0:00:00.034     1M / 21M   INFO    General                 (main.cpp                  :  93)   Determined value is 33
  0:00:00.034     1M / 21M   INFO    General                 (hammer_tools.cpp          :  40)   Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]
  0:00:00.034     1M / 21M   INFO    General                 (main.cpp                  : 114)   Size of aux. kmer data 24 bytes
     === ITERATION 0 begins ===
  0:00:00.034     1M / 21M   INFO   K-mer Counting           (kmer_data.cpp             : 284)   Estimating k-mer count
  0:00:00.092   641M / 678M  INFO   K-mer Counting           (kmer_data.cpp             : 289)   Processing "/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_1.fastq.gz"
  0:01:18.804   641M / 684M  INFO   K-mer Counting           (kmer_data.cpp             : 298)   Processed 22602355 reads
  0:01:18.805   641M / 684M  INFO   K-mer Counting           (kmer_data.cpp             : 289)   Processing "/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_2.fastq.gz"
  0:02:39.217   641M / 684M  INFO   K-mer Counting           (kmer_data.cpp             : 298)   Processed 45204710 reads
  0:02:39.229   641M / 684M  INFO   K-mer Counting           (kmer_data.cpp             : 303)   Total 45204710 reads processed
  0:02:39.984   641M / 684M  INFO   K-mer Counting           (kmer_data.cpp             : 306)   Estimated 1028619306 distinct kmers
  0:02:39.985     1M / 684M  INFO   K-mer Counting           (kmer_data.cpp             : 310)   Filtering singleton k-mers
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2726297600 bytes, address: 0x7fd1cadab000, alignment: 67108864, commit: 1)
  0:02:40.208  2599M / 2658M INFO   K-mer Counting           (kmer_data.cpp             : 316)   Processing "/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_1.fastq.gz"
  0:27:05.546  2599M / 2664M INFO   K-mer Counting           (kmer_data.cpp             : 325)   Processed 22602355 reads
  0:27:05.558  2599M / 2664M INFO   K-mer Counting           (kmer_data.cpp             : 316)   Processing "/project2/ckenkel_26/liruiqi/Osedax_meta/A13515_Osedax_knutei/work/54/ca17335ce248371e1e2dca5a9740a2/A13515_O_knutei_run0_phix_removed.unmapped_2.fastq.gz"
right now I am trying to use ``--only-assembler`` in spades_options or just skip spades entirely
but would be nice to know what was going on
j
That you will need to check the SPAdes GitHub repo unfortunately
Normally the problems I've had with error correction has been memory related but that has been almost always related to insufficient memory
r
oh interesting! Thanks for your insights! I am using 450GB for 2x1.2Gb fastq.gz, which should be more than enough?
j
I would assume so, but it depends how youve confighred the pipeline run resource wise
But normally you would get a segmentation fault if you ran out memory
But also how many times has it got stuck?
r
twice, it fails, and retried with increasing mem and cpu
it doesn't fail because of mem
just time limit
but the process was just stuck
j
I'm not sure then :/
r
like no updates in spades log
j
But the spades repo has lot of good reports and repsones
Maybe you can find something there
r
cool! thanks!
--spades_option='--only-assembler'
is the correct command!
This was the issue
I was wondering if you have any insights regarding running spades without the read error correction step, or should I run an alternative software for read error correction, thanks!
j
Awesome thank you for confirming that's how to use the parameter!
No, sorry - no experience there. I've neve had issue with the error correction stuff (except for memory)
r
actually it should be 
--spades_options='--only-assembler'
2 Views
Open in Slack
PreviousNext
Powered by