The

quantification

is merged now, here is an overview for the next topics that we are planning to add: • Benchmarking: Add a separate input for polyA-Data. As no circRNAs should be present in polyA data, we can classify all of them as false positives. Combined with the results from totalRNA-Seq data we can calculate performance metrics for the pipeline. Results can most likely be visualized through MultiQC, although I don't know how yet • Sponging: I was involved with another pipeline on circRNAs some time ago which was discontinued in favor of this pipeline. The focus there was more on the interaction between miRNAs and circRNAs, not only based on molecular affinity but also on expression. It is planned to accept the output of

nf-core/smrnaseq

as an optional input, which will enable this analysis. @Malte Weyrich will look into this • Isoform-level: As I mentioned in previous messages, all the analyses in the pipeline so far are done on BSJ-level, neglecting splicing within circRNAs. It is planned to add this functionality for paired-end data. For single-end data this won't be possible • Differential expression refactoring: The analyses in the

differential_expression

subworkflow will be updated to make use of the output of the

quantification

subworkflow Always open for thoughts and contributions!

👋 大家好！I am testing the circrna workflow to analysis my data. and I have met some errors ,so I want to show my errors

I am testing my data, and I run 3 control and 3 treat samples, and all of the tasks that labeled "TARGETSCAN" , and "HISAT2_ALIGN" are failed, because of the "Process exceeded running time limit ",but

and the console showed like this: May-04 134708.501 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 90; name: NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:TARGETSCAN (NORMAL_REP3); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (8h); workDir: /home/guozw/hant/project/3_11GSE192902_small_nf_2nd/work/cf/a50c0f08bc81f515f403f4834625f3] task: name=NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:TARGETSCAN (NORMAL_REP3); work-dir=/home/guozw/hant/project/3_11GSE192902_small_nf_2nd/work/cf/a50c0f08bc81f515f403f4834625f3 error [nextflow.exception.ProcessFailedException]: Process

NFCORE_CIRCRNA:CIRCRNA:MIRNA_PREDICTION:TARGETSCAN (NORMAL_REP3)

failed I know that when a task is failed, and it could retry.but why did my failed not retry? and I think maybe the wrong statues " Task completed" status: COMPLETED, and the exit:- , I think maybe there are some errors in this condition,

thanks for your reply! But I had some other errors in 'NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERY:ANNOTATION (NORMAL_REP1)', and the log file content is like this: ERROR ~ Error executing process > 'NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERY:ANNOTATION (NORMAL_REP1)' Caused by: Process

NFCORE_CIRCRNA:CIRCRNA:CIRCRNA_DISCOVERY:ANNOTATION (NORMAL_REP1)

terminated with an error exit status (1) Command executed: annotation.py --input NORMAL_REP1.intersect.bed --exon_boundary 200 --output_bed NORMAL_REP1.circexplorer2.annotated.bed --output_gtf NORMAL_REP1.circexplorer2.annotated.gtf cat <<-END_VERSIONS > versions.yml "NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERY:ANNOTATION": python: $(python --version | sed 's/Python //g') pandas: $(python -c "import pandas; print(pandas.version)") numpy: $(python -c "import numpy; print(numpy.version)") END_VERSIONS Command exit status: 1 Command output: (empty) Command error: WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_NXF_TASK_WORKDIR as environment variable will not be supported in the future, use APPTAINERENV_NXF_TASK_WORKDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_NXF_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_NXF_DEBUG instead /home/guozw/nextflow/nf-core-circrna_dev/dev/bin/annotation.py33 DtypeWarning: Columns (0) have mixed types. Specify dtype option on import or set low_memory=False. df = pd.read_csv(args.input, sep="\t", header=None, usecols=columns.keys()) Traceback (most recent call last): File "/home/guozw/nextflow/nf-core-circrna_dev/dev/bin/annotation.py", line 46, in <module> df['attributes'] = df['attributes'].apply(lambda row: dict([[value.strip(r'"') for value in entry.strip().split(' ')] for entry in row.split(';') if entry])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/series.py", line 4771, in apply return SeriesApply(self, func, convert_dtype, args, kwargs).apply() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/apply.py", line 1105, in apply return self.apply_standard() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/apply.py", line 1156, in apply_standard mapped = lib.map_infer( ^^^^^^^^^^^^^^ File "pandas/_libs/lib.pyx", line 2918, in pandas._libs.lib.map_infer File "/home/guozw/nextflow/nf-core-circrna_dev/dev/bin/annotation.py", line 46, in <lambda> df['attributes'] = df['attributes'].apply(lambda row: dict([[value.strip(r'"') for value in entry.strip().split(' ')] for entry in row.split(';') if entry])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: dictionary update sequence element #18 has length 7; 2 is required Work dir: /home/guozw/hant/project/3_11GSE192902_small_nf_slurm/work/f3/e6e5444c7d7f93cbf9cadd96624246 Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named

.command.sh

when I tested the Test profile,I had met the same error, and I just change the script of 'annotation.py', at line 43, df['attributes'] = df['attributes'].apply(lambda row: dict([[value.strip(r'"') for value in entry.strip().split(' ',1)] for entry in row.split(';') if entry])). and I changed the split parameters with split(' ',1),so it can split at most once, and I think it works. If you do think so, could you fix it in the code?? And the warning 'DEPRECATED USAGE', what should I do or just let it alone??

I want to use the slurm to execute the offline pipeline. And I make a nextflow.config in the workdir. and the content is like this: singularity { enabled=true } process { executor='slurm' queue='normal' clusterOptions = "-N 3" } I set the clusterOptions -N 3,because I want it to run on 3 nodes in slurm. I do not set -n, because different label process has the different cpus,memory... And the base.config has set the cpus, memory, and times based on its label. For example,if a process labeled with process_high, according to the base.config, it will run on 12 cpus, 72G memory, and 16 h. And I set the sbatch -N 3, I think the process_high would run on 3 nodes, it will run on 12 cpus in total. and each node has allocated random cpus, and the total cpus are 12 . But the reality is : each process runs on only one node, and only use one cpu, I see that from the command

htop

. why did this happen?? Did I make something wrong about this?? And how can I tune the config to achieve my idea??😁 because of this?? https://www.nextflow.io/docs/latest/process.html#process-clusteroptions

While you can use the clusterOptions directive to specify options that are supported as process directives (queue, memory, time, etc), you should not use both at the same time, as it will cause undefined behavior. Most HPC schedulers will either fail or simply ignore one or the other.

Maybe I do not understand this ..

Hey everyone, feel free to check out the shiny application I built for downstream analysis. It facilitates dimensionality reduction (PCA, UMAP), pathway analysis and statistical tests (Correlation, differential abundance, both considering inferential uncertainty) You can find a demo of the application here: https://exbio.wzw.tum.de/circest/app/demo

Hello! I am new to nf-core and the circrna pipeline. When I first tried to run it with my data (3 control + 3 treatment samples) I got a “Process exceeded running time limit (8h)” error for the “NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:TARGETSCAN” process. I changed the config for that step to 50 hours and I am getting this: ERROR ~ Error executing process > 'NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:TARGETSCAN (TREATMENT_REP2)' Caused by: Process exceeded running time limit (2d 2h) Command executed: ##format for targetscan cat TREATMENT_REP2.fa | grep ">" | sed 's/>//g' > id cat TREATMENT_REP2.fa | grep -v ">" > seq paste id seq | awk -v OFS="\t" '{print $1, "0000", $2}' > TREATMENT_REP2.targetscan_ts.txt # run targetscan targetscan_70.pl mature.txt TREATMENT_REP2.targetscan_ts.txt TREATMENT_REP2.targetscan.txt cat <-END_VERSIONS versions.yml "NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:TARGETSCAN": awk: $(awk --version | head -n1 | cut -d' ' -f3 | sed 's/,//g' ) targetscan: 7.0 END_VERSIONS Command exit status: - Command output: Processing chr1530906117 30913091+ Processing chr1530918857 30933671+ Processing chr1532887014 32912040- … Command wrapper: Processing chr1530906117 30913091+ Processing chr1530918857 30933671+ Processing chr1532887014 32912040- … The process is stuck at 0% when running: [5f/7fdf7c] process > NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:TARGETSCAN (TREATMENT_REP3) [ 0%] 0 of 6 Any idea why this might be happening? I am running this: nextflow run nf-core/circrna \ -profile docker \ -r dev \ -c '/my/path/next.config' \ --genome GRCh38 \ --input '/my/path/circ_test/input_circ.csv' \ --tool circexplorer2 \ --module circrna_discovery,mirna_prediction,differential_expression \ --phenotype '/my/path/circ_test/phenotype.csv' \ --outdir res_1

Hi, I have one question about the backsplice_gen.sh of "NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERY:FASTA" : In this script,you want to add the splice sequence of the original fasta file, and the splice sequence you think is the first 20 bases of the sequence, is that correct? I think the splice sequence is the first 20 bp the last 20 bp of the fasta, I donot know why you set this ? 😁

I have several questions about the workflow:🤔 (1) about the meaning of "CIRCTEST ": the normal RNAseq gene_count_matrix.csv , and the circRNA count_matrix.txt, (which comes from the

CIRCRNA_DISCOVERY

module), with these tables, then you can do the deseq2 analysis . Why you want to do the CIRCTEST ? the main function of it is to test the variation of circRNAs in respect to host genes. I do not understand the real meaning of this process? (2) In the DIFFERENTIAL_EXPRESSION module, I do not understand the whole process, the process

HISAT2_ALIGN + BAM_SORT_STATS_SAMTOOLS + STRINGTIE_STRINGTIE  + STRINGTIE_PREPDE

is to get the normal RNAseq gene_count_matrix.csv , why would I care about this, the main process is to get the differential genes in circRNA , why would I have to quantification the normal RNAseq ? the relation between this and circRNA differential analysis I do not understand.

Hello! I have a question about the tx and abundance files. Are they supposed to have information on both linear and circular transcripts? My issue is that I am looking for some circRNAs from a specific genomic area (I know for a fact that these circRNAs are there, I just wanted to double-check with the nf core pipeline), but the discovery step itself doesn't seem to be able to capture them (circexplorer2). I can see the ENST and ENSG identifiers that are related to my genes of interest in the tx and abundance files, but I guess that these correspond to the linear forms?

Hello everyone, I've recently been thinking about how we can move forward with the DIFFERENTIAL_EXPRESSION subworkflow. The implementation currently contains some problems, as I pointed out two messages above. Also it is rather hard to maintain in the current state. When refactoring the subworkflow I came to the conclusion that it might be preferable to just provide users with count matrices and refer them to #C045UNCS5R9 for further analysis. There are two things that I would like to preserve: 1. CIRCTEST which I plan to put into a new subworkflow STATISTICAL_TESTS, so that we can add other analyses if necessary 2. The pipeline also provides users with a SummarizedExperiment object, potentially allowing for inferential uncertainty aware DEA using Swish. Maybe we can add this functionality to #C045UNCS5R9. Will discuss with the maintainers over there. Any thoughts on this?

Hey, the pipeline now got a fancy metro map on its website :)

Hi all! I wanted to let you know that I'm changing research institutes soon and will no longer focus on circRNAs (as far as I know atm). I would have liked to contribute more to this pipeline, but sadly, the timing was not right. On July 1st I'll start at the Data Analysis Core of the Institut du Cerveau as a pipeline developer. I was secretly hoping I would still be able to contribute to nf-core, but the reality is that they use Snakemake for all their pipelines 👀 Thanks a lot @Nico Trummer for helping me figure out my small contributions to nf-core/circrna. And also thank you to the whole nf-core core-team and community. It was a real pleasure to discover this corner of the internet! I hope I'll see you all around still 🙂

Hi, I see the big changes about this workflow, and I want to download the new pipeline to run. And I use the command

nohup nextflow run nf-core/circrna -r dev -profile test &

,but it always stopped. From the .nextflow.log, I do not see the error message, and it does not start the analysis. I know the pipeline that I am running is not the latest, and the version is 'vdev-gc29124f' . So,I tried to pull the latest version using

nohup nextflow pull nf-core/circrna > download.log

,and it failed,too. The download.log says "nf-core/circrna contains uncommitted changes -- cannot pull from repository" .So,how can I run the new pipeline. using

git clone <https://github.com/nf-core/circrna.git>

? Thanks for your answers! `Jun-21 115730.079 [main] DEBUG nextflow.validation.SchemaValidator - Starting parameters validation Jun-21 115731.685 [main] DEBUG nextflow.validation.SchemaValidator - Starting validation: 'input': '/nf-core/test-datasets/circrna/sa mples.csv' with '/home/guozw/.nextflow/assets/nf-core/circrna/assets/schema_input.json' Jun-21 115741.925 [main] DEBUG nextflow.validation.SchemaValidator - Validation passed: 'input': '/nf-core/test-datasets/circrna/samp les.csv' with '/home/guozw/.nextflow/assets/nf-core/circrna/assets/schema_input.json' Jun-21 115741.926 [main] DEBUG nextflow.validation.SchemaValidator - Finishing parameters validation Jun-21 115741.931 [main] DEBUG nextflow.validation.SchemaValidator - Starting validation: '/nf-core/test-datasets/circrna/samples.csv ' with '/home/guozw/.nextflow/assets/nf-core/circrna/assets/schema_input.json' Jun-21 115747.504 [main] DEBUG nextflow.validation.SchemaValidator - Validation passed: '/nf-core/test-datasets/circrna/samples.csv' with '/home/guozw/.nextflow/assets/nf-core/circrna/assets/schema_input.json' Jun-21 115747.771 [main] DEBUG nextflow.validation.SchemaValidator - Starting validation: '/nf-core/test-datasets/circrna/samples.csv ' with '/home/guozw/.nextflow/assets/nf-core/circrna/assets/schema_input.json' Jun-21 115753.655 [main] DEBUG nextflow.validation.SchemaValidator - Validation passed: '/nf-core/test-datasets/circrna/samples.csv' with '/home/guozw/.nextflow/assets/nf-core/circrna/assets/schema_input.json' Jun-21 115755.645 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withLabel:process_single

matches labels

process_sin
gle

for process with name NFCORE_CIRCRNACIRCRNACAT_FASTQ Jun-21 115755.650 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Jun-21 115755.650 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Jun-21 115755.654 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local Jun-21 115755.658 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=64; memory=2 51 GB; capacity=64; pollInterval=100ms; dumpInterval=5m Jun-21 115755.660 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local) Jun-21 115755.715 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withLabel:process_medium

matches labels

process_med
ium

for process with name NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:SEQKIT_SPLIT Jun-21 115755.715 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withName:SEQKIT_SPLIT

matches process NFCORE_CIRCRNA CIRCRNAPREPARE_GENOME:SEQKIT_SPLIT Jun-21 115755.717 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Jun-21 115755.717 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Jun-21 115755.722 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withLabel:process_high

matches labels

process_high

for process with name NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:BOWTIE_BUILD Jun-21 115755.747 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withName:HISAT2_BUILD

matches process NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:HISAT2_BUILD Jun-21 115755.749 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Jun-21 115755.749 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Jun-21 115755.757 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withLabel:process_high

matches labels

process_high

for process with name NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:STAR_GENOMEGENERATE Jun-21 115755.758 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withName:STAR_GENOMEGENERATE

matches process NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:STAR_GENOMEGENERATE Jun-21 115755.760 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Jun-21 115755.760 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Jun-21 115755.767 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withLabel:process_high

matches labels

process_high

for process with name NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:SEGEMEHL_INDEX Jun-21 115755.767 [main] DEBUG nextflow.script.ProcessConfig - Config settings

withName:SEGEMEHL_INDEX

matches process NFCORE_CIRCRNACIRCRNAPREPARE_GENOME:SEGEMEHL_INDEX Jun-21 115755.768 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Jun-21 115755.768 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' `

I had met several questions about the new pipeline, and I hope can get your help. (1) NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERYANNOTATIONANNOTATE (fust1_3:circexplorer2) and NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERYANNOTATIONANNOTATE (fust1_2:merged) , I can not see the difference from the two, about the command line, and the results are also similar. (2) NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERYANNOTATIONCOMBINE_GTFS. I think there maybe some errors? the command is like

sort -k 1,1 -k3,3n -k4,4n -u fust1_2.gtf fust1_1.gtf cel_N2_1.gtf cel_N2_2.gtf cel_N2_3.gtf fust1_3.gtf > annotation.combined.gtf

, but the third column is type of circRNA, not the start . I think there maybe

sort -k 1,1 -k4,4n -k5,5n

. NFCORE_CIRCRNACIRCRNAQUANTIFICATION:COMBINE_TRANSCRIPTOME_GTFS also have the same mistakes.

CIRC_Georgetown has removed themselves from this channel.

Hi, I tried to test the test_full profile, and my command is like

nextflow run nf-core/circrna -profile test,full,singularity -r dev -bg

And there is an error:

Empty hisat2 bam generated

.How did that happen?? ERROR ~ Error executing process > 'NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERYCIRIQUANTMAIN (fust1_2)' Caused by: Process

NFCORE_CIRCRNA:CIRCRNA:CIRCRNA_DISCOVERY:CIRIQUANT:MAIN (fust1_2)

terminated with an error exit status (1) Command executed: BWA=

which bwa

HISAT2=

which hisat2

STRINGTIE=

which stringtie

SAMTOOLS=

which samtools

BWA_FILE=

ls bwa/*.bwt

BWA_PREFIX=

basename $BWA_FILE .bwt

HISAT2_FILE=

ls hisat2/*.1.ht2

HISAT2_PREFIX=

basename $HISAT2_FILE .1.ht2

printf "name: ciriquant\ntools:\n bwa: $BWA\n hisat2: $HISAT2\n stringtie: $STRINGTIE\n samtools: $SAMTOOLS\n\nreference:\n fasta: chrI.fa\n gtf: chrI.gtf\n bwa_index: bwa/$BWA_PREFIX\n hisat_index: hisat2/$HISAT2_PREFIX" > config.yml CIRIquant \ -t 2 \ -1 fust1_2_1_val_1.fq.gz \ -2 fust1_2_2_val_2.fq.gz \ --config config.yml \ --no-gene \ -o fust1_2 \ -p fust1_2 \ \ cat <<-END_VERSIONS > versions.yml "NFCORE_CIRCRNACIRCRNACIRCRNA_DISCOVERYCIRIQUANTMAIN": bwa: $(echo $(bwa 2>&1) | sed 's/^.*Version: //; s/Contact:.*$//') ciriquant: $(echo $(CIRIquant --version 2>&1) | sed 's/CIRIquant //g' ) samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//') stringtie: $(stringtie --version 2>&1) hisat2: 2.1.0 END_VERSIONS Command exit status: 1 Command output: (empty) Command error: WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_NXF_TASK_WORKDIR as environment variable will not be supported in the future, use APPTAINERENV_NXF_TASK_WORKDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_NXF_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_NXF_DEBUG instead INFO: fuse: warning: library too old, some operations may not work [Tue 2024-06-25 195040] [INFO ] Input reads: fust1_2_1_val_1.fq.gz,fust1_2_2_val_2.fq.gz [Tue 2024-06-25 195040] [INFO ] Library type: unstranded [Tue 2024-06-25 195040] [INFO ] Output directory: fust1_2, Output prefix: fust1_2 [Tue 2024-06-25 195040] [INFO ] Config: ciriquant Loaded [Tue 2024-06-25 195040] [INFO ] 64 CPU cores availble, using 2 [Tue 2024-06-25 195040] [INFO ] Align RNA-seq reads to reference genome .. Traceback (most recent call last): File "/usr/local/bin/CIRIquant", line 10, in <module> sys.exit(main()) File "/usr/local/lib/python2.7/site-packages/CIRIquant/main.py", line 155, in main hisat_bam = pipeline.align_genome(log_file, thread, reads, outdir, prefix) File "/usr/local/lib/python2.7/site-packages/CIRIquant/pipeline.py", line 52, in align_genome raise utils.PipelineError('Empty hisat2 bam generated, please re-run CIRIquant with -v and check the fastq and hisat2-index.') CIRIquant.utils.PipelineError: Empty hisat2 bam generated, please re-run CIRIquant with -v and check the fastq and hisat2-index.

With the most recent updates on the dev branch, it seems the ciriquant + Hisat2 index building problem is fixed. However I ran into a downstream problem. Here is the error message: Jul-23 105626.634 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > ‘NFCORE_CIRCRNACIRCRNAQUANTIFICATION:JOIN_GENE_COUNTS (gene_counts)’ Caused by: Process

NFCORE_CIRCRNA:CIRCRNA:QUANTIFICATION:JOIN_GENE_COUNTS (gene_counts)

terminated with an error exit status (255) Command executed: csvtk \ join \ -f 1,2 -t \ --num-cpus 1 \ --out-file gene_counts.csv \ PDX5.gene_counts.tsv cat <<-END_VERSIONS > versions.yml “NFCORE_CIRCRNACIRCRNAQUANTIFICATIONJOIN GENE COUNTS” csvtk: $(echo $( csvtk version | sed -e “s/csvtk v//g” )) END_VERSIONS Command exit status: 255 Command output: (empty) Command error: [ERRO] two or more files needed

Hi! This pipeline didn't get a template update with tools 3.0.0 due to an incorrect

version

entry in

nextflow.config

in the dev branch. It should be e.g.

1.2.3dev

, no leading`v`, not just

dev

. Once you changed that in your

dev

branch you can get a template sync PR by starting the sync GHA action with your pipeline name as an input https://github.com/nf-core/tools/actions/workflows/sync.yml

Hi all, I’m busy trying out your circRNA pipeline. The nf-core page pointed me to this slack channel with the text:

For further information or help, don't hesitate to get in touch on the Slack #circrna channel (you can join with this invite).

Hope this is the correct place to ask questions? I’m wondering if there are any pre-build indexes for the various tools used inside this pipeline (

bwa

,

hisat2

, etc) that I can point nextflow to when using the

--genome GRCh38

command?

I’m also not getting this pipeline to work. Here is my test `samplesheet.csv`:

cat samplesheet.csv
sample,fastq_1,fastq_2
cel_N2_1,<https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/N2_rep1_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/N2_rep1_2.fastq.gz>
cel_N2_2,<https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/N2_rep2_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/N2_rep2_2.fastq.gz>
cel_N2_3,<https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/N2_rep3_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/N2_rep3_2.fastq.gz>
fust1_1,<https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/fust1_rep1_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/fust1_rep1_2.fastq.gz>
fust1_2,<https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/fust1_rep2_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/fust1_rep2_2.fastq.gz>
fust1_3,<https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/fust1_rep3_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/circrna/fastq/fust1_rep3_2.fastq.gz>

and my `params.yaml`:

cat params.yaml
# <https://nf-co.re/circrna/dev/parameters/>
input: 'samplesheet.csv'
outdir: 'results'
#gtf: 'references/gencode_v27_annotation.gtf'
#fasta: 'references/GRCh38_primary_assembly_genome.fa'
#genome: 'GRCh38'
#profile: 'docker'
email: 'leendertcloete@gmail.com'
multiqc_title: 'test_circRNA_nfcore'
tools: 'ciriquant,circexplorer2,find_circ,circrna_finder,mapsplice,dcc'
bsj_reads: 5
min_tools: 3

And with this cmd:

docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):$(pwd) -w $(pwd) nextflow/nextflow:24.10.3 bash -c 'nextflow run -r dev nf-core/circrna -params-file params.yaml -profile docker --genome GRCh38'

I get this error. What am I doing wrong here?

Hey @Leendert Cloete you are in the correct place. Using the

--genome

parameter you specify the pipeline to use iGenomes. You should read this before you move on. In the beginning of the pipeline, it filters out all GTF entries that are located on sequences that do not exist in the FASTA file. This is most likely because the chromosomes are annotated differently. Moving forward I suggest you to download your genome GTF and FASTA from a source of your choice, e.g. here. All the tool indices will be generated if you don't provide them, so no need to worry. If you want to re- use them in future runs, you can use the

save_reference

parameter, copy them from the pipeline output dir to a permanent location and reference them from there. Let me know if you have any other question!

Hi, I’m trying to use the nf-core/circrna pipeline and I’m not sure what I’m doing wrong but it keeps crashing half way. Could somebody please help? https://github.com/nf-core/circrna/issues/184#issue-2807834539

Hi there! Thank you for developing the pipeline, I wanted to report that the pipeline fails to pull a image when using singularity profile

Jan-29 09:33:17.490 [Actor Thread 7262] DEBUG nextflow.Session - Session aborted -- Cause: java.lang.IllegalStateException: Failed to pull singularity image
  command: singularity pull  --name depot.galaxyproject.org-singularity-pygtftk-1.6.2--py39h4e691d4_2.img.pulling.1738103595466 <docker://depot.galaxyproject.org/singularity/pygtftk:1.6.2--py39h4e691d4_2> > /dev/null
  status : 255
  hint   : Try and increase singularity.pullTimeout in the config (current is "20m")
  message:
    FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for <docker://depot.galaxyproject.org/singularity/pygtftk:1.6.2--py39h4e691d4_2>: pinging container registry depot.galaxyproject.org: StatusCode: 404, "<html>\r\n<head><title>404 Not Found</title></head>\r..."

The image

pygtfk:1.6.2

exist though on the depot, and manually changing the docker protocol into http allows it to pull the image and fixes the problem. I'm fairly new to using nf-core, so I'm not sure if this is an intended behavior or not. script used to submit job:

#!/bin/bash 
#PBS -l select=1:ncpus=16:mem=123gb
#PBS -N nfcircrna 
#PBS -l walltime=11:59:00 
#PBS -M sxxxxxxxx
#PBS -m ae 
#PBS -j oe 
module load nextflow/24.04.4
module load java/21.0.3_9-openjdk
nextflow run nf-core/circrna -r dev --input samplesheet.csv --outdir /srv/scratch/z5628486/Analysis/Results  --phenotype phenotype.csv --fasta GRCh38.primary_assembly.genome.fa --gtf gencode.v47.annotation.gtf -profile singularity --tools circexplorer2,ciriquant -c /scratch/Analysis/resourcelimits.config -resume -with-report /scratch/run_1.html

Hi there! Thanks for all your work on this pipeline. New user here with a question about the quantification workflow. I am getting the following error during

QUANTIFICATION:CIRIQUANT:MAIN

$ nextflow run $nfhome -profile test,singularity --tools ciriquant
$ cat work/00/e87601a183fb216db669f3d8e3acc9/.command.log
INFO:  Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
INFO:  Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
INFO:  underlay of /etc/localtime required more than 50 (73) bind mounts
[Wed 2025-08-20 16:40:47] [INFO ] Input reads: fust1_1_1_val_1.fq.gz,fust1_1_2_val_2.fq.gz
[Wed 2025-08-20 16:40:47] [INFO ] Library type: unstranded
[Wed 2025-08-20 16:40:47] [INFO ] Output directory: /****/nf_circ_test/work/00/e87601a183fb216db669f3d8e3acc9/fust1_1, Output prefix: fust1_1
[Wed 2025-08-20 16:40:47] [INFO ] Config: ciriquant Loaded
[Wed 2025-08-20 16:40:47] [INFO ] 80 CPU cores availble, using 4
[Wed 2025-08-20 16:40:47] [INFO ] Align RNA-seq reads to reference genome ..
[Wed 2025-08-20 16:40:53] [INFO ] Estimate gene abundance ..
[Wed 2025-08-20 16:40:55] [INFO ] Using user-provided circRNA bed file: /****/nf_circ_test/work/00/e87601a183fb216db669f3d8e3acc9/all.annotated.bed12
[Wed 2025-08-20 16:40:55] [INFO ] Extract circular sequence
[Wed 2025-08-20 16:40:55] [100% ] [##################################################]
[Wed 2025-08-20 16:40:55] [INFO ] Building circular index ..
[Wed 2025-08-20 16:40:55] [INFO ] De novo alignment for circular RNAs ..
[Wed 2025-08-20 16:41:09] [INFO ] Detecting reads containing Back-splicing signals
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
 File "/opt/conda/lib/python3.12/multiprocessing/pool.py", line 125, in worker
  result = (True, func(*args, **kwds))
          ^^^^^^^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.12/site-packages/CIRIquant/circ.py", line 364, in denovo_worker
  parser = CIRC[contig][circ_id]
       ~~~~~~~~~~~~^^^^^^^^^
KeyError: 'circ_chrI:1039055-1039884:+'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/opt/conda/bin/CIRIquant", line 8, in <module>
  sys.exit(main())
       ^^^^^^
 File "/opt/conda/lib/python3.12/site-packages/CIRIquant/main.py", line 201, in main
  out_file = circ.proc(log_file, thread, bed_file, hisat_bam, rnaser_file, reads, outdir, prefix, anchor, lib_type,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.12/site-packages/CIRIquant/circ.py", line 659, in proc
  cand_bsj = proc_denovo_bam(denovo_bam, thread, circ_info, anchor, lib_type)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.12/site-packages/CIRIquant/circ.py", line 321, in proc_denovo_bam
  tmp_cand = job.get()
        ^^^^^^^^^
 File "/opt/conda/lib/python3.12/multiprocessing/pool.py", line 774, in get
  raise self._value
KeyError: 'circ_chrI:1039055-1039884:+'

1. Could somebody kindly confirm whether the following is the expected format of the circRNA ID (in the 4th column of the BED file called “`all.annotated.bed12`” —I think it is output by

NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:ANNOTATE_COMBINED:RENAME

)? Specifically the “circ_” prefix before “chrI”. I think the "chrI" in the first field being discrepant from the "circ_chrI" in the 4th field would cause problems in CIRIquant.

$ head work/67/9ea9165027dc485e9517f6eaa6016b/all.annotated.bed12

chrI  1039055 1039884 circ_chrI:1039055-1039884:+   0    +    1039055 1039055 0,0,0  3    120,80,544   0,162,285    0    circRNA Y34D9A.3    NM_001306323
  9,10,11 chrI:1037716-1039055|chrI:1039884-1040116
chrI  1056063 1060673 circ_chrI:1056063-1060673:-   0    -    1056063 1056063 0,0,0  4    145,219,164,221 0,1633,3319,4389    0    circRNA vps-4  NM_058415

1. Likewise, when running

NFCORE_CIRCRNA:CIRCRNA:QUANTIFICATION:CIRI_QUANT:MAIN

, is this the expected appearance of the circRNA ID’s appearing in the SN field of the header of “`*_denovo.sorted.bam`”?

Apptainer> samtools view -H test/circ/test_denovo.sorted.bam | head
@HD	VN:1.0	SO:coordinate
@SQ	SN:circ_chrI:1039055-1039884:+	    LN:1660
@SQ	SN:circ_chrI:1056063-1060673:-	    LN:9222
@SQ	SN:circ_chrI:1071472-1071936:+	    LN:930

Thank you!!

Hi team, I am getting an error in nf-core/circRNA. Fatal LIMIT error: the number of junctions to be inserted on the fly =1054268 is larger than the limitSjdbInsertNsj=1000000 Fatal LIMIT error: the number of junctions to be inserted on the fly =1054268 is larger than the limitSjdbInsertNsj=1000000 SOLUTION: re-run with at least --limitSjdbInsertNsj 1054268 Where do I edit to allow to 1500000 as the limit. Am I able to add a params file? This is my bash nextflow run nf-core/circrna \ -profile conda \ -c custom.config \ -resume \ --input ADsample.csv \ --outdir /srv/scratch/grace/nextflow/AD_samples \ -r dev \ --tools ciriquant,circexplorer2,find_circ,circrna_finder \ --quantification_tools ciriquant,psirc,sum,max \ --gtf /srv/scratch/grace/gencode.v47.annotation.gtf \ --fasta /srv/scratch/grace/Homo_sapiens/UCSC/hg38/Sequence/WholeGenomeFasta/genome.fa \ --tools ciriquant,circexplorer2,find_circ,circrna_finder,mapsplice,dcc \

Hi! I am getting an error during `'NFCORE_CIRCRNACIRCRNACAT_FASTQ`:

ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (HC01)'

Caused by:
  Process `NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (HC01)` terminated with an error exit status (126)


Command executed:

  cat input1/01-001_S1_L001_R1_001.fastq.gz > HC01_1.merged.fastq.gz
  cat input2/01-001_S1_L001_R2_001.fastq.gz > HC01_2.merged.fastq.gz
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ":
      cat: $(echo $(cat --version 2>&1) | sed 's/^.*coreutils) //; s/ .*$//')
  END_VERSIONS

Command exit status:
  126

Command output:
  (empty)

Command error:
  /bin/bash: line 10: .command.run: Permission denied

Work dir:
  /scratch/azabala/network/circRNA/work_circRNA/aa/5a34573f047cf813261ee17db91ea1

Container:
  /home/azabala/.apptainer_cache/quay.io-nf-core-ubuntu-20.04.img

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

Hi! I am getting an error during `'NFCORE_CIRCRNACIRCRNAMIRNA_PREDICTION:COMPUTE_CORRELATIONS (batch_1)'`:

ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:MIRNA_PREDICTION:COMPUTE_CORRELATIONS (batch_1)'

Caused by:
  Process `NFCORE_CIRCRNA:CIRCRNA:MIRNA_PREDICTION:COMPUTE_CORRELATIONS (batch_1)` terminated with an error exit status (1)


Command executed [/home/azabala/.nextflow/assets/nf-core/circRNA/./workflows/circrna/../../subworkflows/local/../../modules/local/compute_correlations/templates/compute_correlations.R]:

  #!/usr/bin/env Rscript
  
  library(fishpond)
  suppressMessages(library(SummarizedExperiment))
  
  tx_expression <- readRDS('experiments.merged.rds')
  mi_expression <- read.table('mirna.normalized_counts_filtered.tsv', header=TRUE, row.names=1, sep='\t')
  interactions <- read.table('mirna.targets.1.tsv', sep='\t')
  
  tx_expression <- scaleInfReps(tx_expression)
  tx_expression <- labelKeep(tx_expression) # Here one can perform custom filtering
  
  if (!any(mcols(tx_expression)$keep)) {
      stop('No transcripts left after filtering')
  }
  
  result_cols <- c('stat', 'log2FC', 'pvalue', 'locfdr', 'qvalue')
  
  # Iterate rows of interactions
  for (i in 1:nrow(interactions)) {
      # Get miRNA and target gene
      miRNA <- interactions[i, 1]
      targets <- unlist(strsplit(interactions[i, 2], ','))
  
      mirna_expression <- mi_expression[miRNA,]
      transcript_expression <- tx_expression[targets,]
  
      if (!any(mcols(transcript_expression)$keep)) {
          print(paste('No transcripts left after filtering for miRNA', miRNA))
          next
      }
  
      # Add miRNA expression to colData so that it can be used for correlation
      colData(transcript_expression) <- cbind(
          colData(transcript_expression),
          t(mirna_expression[, rownames(colData(transcript_expression))])
      )
  
      result <- rowData(swish(transcript_expression, miRNA, cor = "pearson"))[, result_cols]
      result <- result[complete.cases(result), ]
      write.table(result, paste0(miRNA, '.tsv'), sep = '\t')
  }
  
  ################################################
  ################################################
  ## VERSIONS FILE                              ##
  ################################################
  ################################################
  
  r.version <- strsplit(version[['version.string']], ' ')[[1]][3]
  
  writeLines(
      c(
          '"NFCORE_CIRCRNA:CIRCRNA:MIRNA_PREDICTION:COMPUTE_CORRELATIONS":',
          paste('    r-base:', r.version)
      ),
  'versions.yml')
  
  ################################################
  ################################################
  ################################################
  ################################################

Command exit status:
  1

Command output:
  (empty)

Command error:
  Error in h(simpleError(msg, call)) : 
    error in evaluating the argument 'x' in selecting a method for function 't': undefined columns selected
  Calls: cbind ... standardGeneric -> eval -> eval -> eval -> t -> [ -> [.data.frame
  Execution halted

Work dir:
  /scratch/azabala/network/circRNA/work_circRNA/fb/8cd9537a6b1538f793cd1c4367e95d

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>

 -- Check '.nextflow.log' file for details

Hello! I ran the pipeline successfully on three test samples with following parameters: nextflow run nf-core/circrna -r dev \ -profile singularity \ -w circ_RNA_seq_test_2/work \ --input samplesheet_test.csv \ --outdir circ_RNA_seq_test_2/results \ --genome GRCh38 \ --tools circexplorer2,ciri,find_circ \ --min_tools 2 \ --save_intermediates \ --bsj_reads 2 \ --mirna_tools miranda,targetscan \ --mirna_min_tools 2 \ -resume I noticed that the work directory is very large with more than 800GB. I need to run this on 53 samples, which will most likely hit the limit of my scratch quota. As I understand, the --save-intermediates just influences the result directory and not the work directory. The biggest files are the different sam files produced by different tools. Is there a way to reduce this memory footprint? Thank you for your help in advance!