- I am using a test set of 4 samples only. My runs...
# denovotranscript
c
• I am using a test set of 4 samples only. My runs currently fail on
NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:TRINITY (pooled_reads)
with a timeout error (16h). Is it expected for this step to take that long? Also, this is currently on 4 samples only - I was hoping to do a lot more eventually. Since the samples are merged in the first step will the time requirement for more samples then become prohibitive? Any thoughts would be much appreciated!
a
Running out of resources was a common issue in the v1.0.0 so I have increased the resources for this step in v1.1.0. I've increased it to 200hr. Trinity Wiki suggests ~0.5-1 hr per million pairs of reads. When I tested with 6 samples (~30M paired-end reads each), it took ~21.5h. Happy hear suggestions for changing the defaults if needed 🙂 You also have the option to use a custom config to increase the resources allocation for this step
c
Thank you @Avani Bhojwani! Sorry I had missed your reply, but I will give it a go!
👍🏽 1
Hi! I tried again with v1.1.0. I have 4 samples with around 40 million read pairs each. Trinity timed out seconds after 5d 8h - exactly what had been requested. Not sure why this is different from 200 hr.
I'm also puzzled why it took only a fraction of the time for you. 21 hours is a lot less than 0.5-1 hr per million pairs of reads? It was only a test run for me and we already accumulated quite the AWS cost for it (>$300) which doesn't bode well for the run I actually want to do. Any ideas or suggestions on what could be going wrong?
a
Hi Christel, I ran it on my uni's HPC, rather than on the cloud. I'm not sure if the issue is related to that and fusion. However, there are updates to fusion coming soon which may fix it. Would you be able to send any log files, so we can check if there are errors related to it? I'm also checking with the core team about it
c
Thanks Avani. Unfortunately, there were log files produced, either because the run timed out or because of issues we had with Seqera platform at the time. However, I noticed the following in the error report:
Copy code
Command executed:

  # Note that Trinity needs the word 'trinity' in the outdir
  
  Trinity \
      --seqType fq \
      --max_memory 256G \
      --left input1/pooled_reads_1.merged.fastq.gz --right input2/pooled_reads_2.merged.fastq.gz \
      --output pooled_reads_trinity \
      --CPU 20 \
      --SS_lib_type RF \
      > >(tee pooled_reads.log)
This all makes sense. But then I get the following Command error
Copy code
Wednesday, January 8, 2025: 09:06:28	CMD: /usr/local/opt/trinity-2.15.2/util/support_scripts/write_partitioned_trinity_cmds.pl --reads_list_file /fusion/s3/altos-lab-nextflow/scratch/lSQjeMViiz59/1f/c932da1f264fc107098c4bb0e6c31b/pooled_reads_trinity/partitioned_reads.files.list --CPU 1 --max_memory 1G  --run_as_paired  --SS_lib_type F  --seqType fa --trinity_complete --full_cleanup  --no_salmon  > recursive_tri
The options here are different for CPU and memory, as well as the lib_type. Also, I can't find
--run_as_paired
in the current documentation for Trinity. It seems to be when read pairs are in the same file which is not the case for my run or the parameters above. Do you have an idea what could be going on?
Hi again! I have also tried running denovotranscript with rnaSPAdes. Here, the assembly goes through (in something like 9.5 hours) but it fails afterwards during QC with a file not found error
FileNotFoundError: [Errno 2] No such file or directory: '/fusion/s3/altos-lab-nextflow/scratch/3sqkOOPzAlq74x/3c/8b6233a1dd62053499320e22f12814/input_seqs/*'
Indeed, the input_seqs folder does not contain any transcript fasta even though it is produced by SPADES and further processed in the CAT-CAT step. Thus, BUSCO and rnaQUAST do no know which transcripts to assess. Any help would be much appreciated! Thank you!
t
This may not be of any help, but did you manually name the work directory
scratch/3sqkOOPzAlq74x
?
a
Hi Christel, For the run that fails during QC, could you post the command executed during CAT-CAT? I think I have come across this issue before and it should be resolved in the new release of the pipeline (v1.2.0). I'm looking into the Trinity issue 🙂
Looks like someone raised a similar issue with the Trinity team earlier: https://github.com/trinityrnaseq/trinityrnaseq/issues/263 I can see that the container used for Trinity in the pipeline has the updated code they mentioned for setting the CPU and memory for grid nodes. I'm guessing it's only using 1 CPU and 1G memory because we're not using grid_exec. It seems like Trinity deals with the reads such that by the time it gets to the partitioning stage, it has converted the orientation to F and concatenated left and right reads into one.
c
Thanks for your thoughts!
@Timothy Little, no the naming was automatic
Hi Avani, here is the command that was run
Copy code
cat \
     \
    pooled_reads.transcripts.fa.gz \
     \
    > all_assembled.fa.gz

cat <<-END_VERSIONS > versions.yml
"NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:CAT_CAT":
    pigz: $( pigz --version 2>&1 | sed 's/pigz //g' )
END_VERSIONS
a
sorry, I just realised I would need to see the command from TX2GENE to confirm if the issue is similar to what I fixed in the v1.2.0 release so that the QC steps can be completed. It should look like this:
Copy code
awk \
        'BEGIN { FS="\t"; OFS="\t"; print "transcript_id\tgene_id" } NR>1 { print $1, $3 }' \
         \
        all_assembled.pubids \
        > all_assembled.tx2gene.tsv

    cat <<-END_VERSIONS > versions.yml
    "NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:TX2GENE":
        gawk: $(awk -Wversion | sed '1!d; s/.*Awk //; s/,.*//')
    END_VERSIONS
but before the issue was fixed, it was like this:
Copy code
awk \
    'BEGIN { FS="\t"; OFS="\t"; print "transcript_id\tgene_id" } NR>1 { print $1, $3 }' \
     \
     \
    > all_assembled.tx2gene.tsv

cat <<-END_VERSIONS > versions.yml
"NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:TX2GENE":
    gawk: $(awk -Wversion | sed '1!d; s/.*Awk //; s/,.*//')
END_VERSIONS
f
The command was indeed the older version:
👍🏽 1
a
okay, so in your case the v1.2.0 pipeline should work with rnaSPAdes 🤞🏾 For Trinity, it's hard to be sure what the issue is without logs. You could try running the pipeline with fusion disabled. If using Seqera Platform it can be enabled / disabled when creating a new Compute Environment
f
we are trying the suggestions out now, will let you know how it goes. thanks!
💕 1