Hi all, does anyone know why InterProScan (via nf-...
# funcscan
p
Hi all, does anyone know why InterProScan (via nf-core/funcscan) is only returning three hits in my run? Thanks!
j
hi @Philipp Glasmacher, that's extremely hard to say without more context sorry 😅
p
Hi @James Fellows Yates, apologies for the late reply. I noticed on the usage page the directory should look like this - but for me it looks like this. Could that be the issue. Thanks.
j
Ok in that case maybe it's a version issue? The pipeline has version 5.59_91.... I don't know if interproscan matches it's database to the tool itself (I didn't add it to the pipeline) But regardless, what I meant more was you didn't say what version of the pipeline, what command you used ,what inputs etc. So in the future when asking for help I recommend in the first case posting
moreinfo
s
To help we'd love a few more details please: 1. The Nextflow command you used to launch the pipeline 2. Any custom Nextflow config files 3. The full error message 4. The corresponding
.nextflow.log
file, where you are running the analysis. Remember to use triple backslashes on their own lines for code blocks. Also take a look at https://nf-co.re/usage/troubleshooting for more information on debugging failed pipelines. Thanks!
j
Because then we can immediately start help debugging and you don't have to wait so long for a useful answer:)
p
@James Fellows Yates Apologies, will do in the future. Thank you so much! 🙂
Hi James — sorry, updating the database to match the nf-core version did not resolve the issue. Here are the full details.
Copy code
1) Job script

#!/bin/bash -l
#$ -S /bin/bash
#$ -N funcscan_MAG30
#$ -cwd
#$ -pe smp 1
#$ -l h_rt=48:00:00
#$ -l mem=8G
#$ -o logs/funcscan_MAG30.$JOB_ID.out
#$ -e logs/funcscan_MAG30.$JOB_ID.err

set -euo pipefail

module purge
module load java/temurin-17/17.0.2_8

export NXF_OPTS='-Xms1g -Xmx6g'

export NXF_HOME="$HOME/Scratch/.nextflow"
export NXF_TEMP="$HOME/Scratch/.nextflow_tmp"
mkdir -p "$NXF_HOME" "$NXF_TEMP"

export APPTAINER_CACHEDIR="$HOME/Scratch/.apptainer"
export NXF_SINGULARITY_CACHEDIR="$HOME/Scratch/.apptainer/pull"
mkdir -p "$APPTAINER_CACHEDIR" "$NXF_SINGULARITY_CACHEDIR"

nextflow run nf-core/funcscan \
  -r 3.0.0 \
  -profile ucl_myriad \
  -resume \
  -work-dir "$HOME/Scratch/funcscan_MAG30_work" \
  --input samplesheet.csv \
  --outdir results_funcscan_3.0.0 \
  --run_amp_screening \
  --run_arg_screening \
  --run_bgc_screening \
  --run_protein_annotation \
  --protein_annotation_interproscan_db /myriadfs/home/zcbepgl/Scratch/databases/interproscan_5.59/interproscan-5.59-91.0 \
  --protein_annotation_interproscan_applications Pfam,Hamap,PANTHER,TIGRFAM,sfld,ProSiteProfiles,ProSitePatterns \
  --bgc_mincontiglength 3000 \
  --bgc_antismash_contigminlength 3000
2) Samplesheet
Copy code
sample,fasta
MAG30,/home/zcbepgl/Scratch/MAG30_ipr/MAG30.fa
3) InterProScan process status
Copy code
The INTERPROSCAN process completes successfully (exit 0):

Process NFCORE_FUNCSCAN:FUNCSCAN:PROTEIN_ANNOTATION:INTERPROSCAN (MAG30); status: COMPLETED; exit: 0
However, the only error i see in the nextflow log is:
Copy code
Skipping output binding because one or more optional files are missing.
4) Output behaviour Pyrodigal predicts ~6000 proteins, but the InterProScan TSV output appears heavily dominated by PF00004 (AAA ATPase family) and looks very repetitive overall.
Copy code
Example:

k141_5296188_14 ... Pfam PF00004 ATPase family associated with various cellular activities (AAA)
k141_2743843_13 ... TIGRFAM TIGR01027 proB: glutamate 5-kinase
k141_8488096_5 ... PANTHER PTHR23076 METALLOPROTEASE M41 FTSH
5) Pfam run separately For debugging, I also ran Pfam separately on the same predicted protein FASTA outside the pipeline, and it produced many diverse domain hits (not just PF00004). So Pfam itself appears to work correctly. Given that: InterProScan exits cleanly (exit 0) No fatal errors are reported Standalone Pfam behaves normally But the pipeline TSV output seems skewed / repetitive (only 37 hits in total) Could this be: An output parsing or aggregation issue? A filtering step in the pipeline? Something related to optional output binding? Or something still misconfigured in the InterProScan integration? Thanks again — really appreciate the help.
j
I'm not sure unforauntely at this stage now 😞, I'm not familiar with InterProScan myself... @Evangelos Karatzas @Martin Beracochea what about you guys? Any suggestions?
e
Hmm it would be a good test to try running the #C084HP11VPF pipeline at this point for the same protein input. The latest version uses the same exact version as #C02K5GX2W93 , but I made some updates to the module because it used to be buggy (i.e., it won't run correctly given more than 1 cpu for multiple databases). Can you try out and let me know? you can override the
--interproscan_db
parameter with your already downloaded InterProScan db version
m
Also, can you share the interproscan process .command.log and .command.sh please?
p
Hi @Evangelos Karatzas and @Martin Beracochea, I ran #C084HP11VPF on the same protein input, overriding
--interproscan_db
to point to my already downloaded local InterProScan database (same setup as in #C02K5GX2W93). I observe the same behaviour there as well. In the InterProScan task logs, I consistently see:
Error in running [ps -o pid,vsize,rss,command -p 99]           followed by:
Copy code
java.lang.NullPointerException
    at uk.ac.ebi.interpro.scan.io.cli.CommandLineConversationMonitor.getJavaCmdCount(...)
For reference, here is the
.command.sh
from the failing InterProScan task:
interproscan.sh \
--cpu 6 \
--input MEGAHIT-MetaBAT2Refined-BelfastCompost.40_cleaned.faa.fastq \
--applications PANTHER,ProSiteProfiles,ProSitePatterns,Pfam \
--disable-precalc --disable-residue-annot \
--enable-tsv-residue-annot --formats tsv \
--output-file-base MEGAHIT-MetaBAT2Refined-BelfastCompost.40_interproscan.faa
e
--cpu 6 is not right with this version, it should be --cpu 1
and in #C084HP11VPF this should be taken care of by default with:
Copy code
withName: 'NFCORE_PROTEINANNOTATOR:PROTEINANNOTATOR:FUNCTIONAL_ANNOTATION:INTERPROSCAN' {
        publishDir = [
            path: { "${params.outdir}/functional_annotation/interproscan/${meta.id}/" },
            mode: params.publish_dir_mode,
            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
        ]
        ext.args = [
            "--applications ${params.interproscan_applications}",
            params.interproscan_enableprecalc ? '' : '--disable-precalc'
        ].join(' ').trim()
        cpus = 1 // bugs with more
    }
Thatswhere the NullPointerExpression comes from
p
Awesome — thanks. I’ll set InterProScan to
--cpu 1
and rerun. I’m not 100% sure this will fix the hit content issue (right now I’m basically only seeing approx 20 hits, whereas when I ran Pfam/InterProScan independently I get many more hits per MAG in the thousands), but I’ll definitely give it a shot and report back. Thanks so much @Evangelos Karatzas 🙏
e
Let me know!
🙏 1
j
@Evangelos Karatzas should we harcoce teh cpus1 in funcsacn then?
e
Yep, until we decide if we port over to the latest docker-only version which I think is fixed (just no conda there)
👍 1
p
Hey @Evangelos Karatzas, unfortunately still no. Is there anything else you can think of that may incur this. As you can see when i run interpro independently, i get a lot more hits.
e
Can you send me a couple of your hit proteins so I can try running those myself as well? And I guess you are using the same db version and member databases for both funcscan and manually?
p
@Evangelos Karatzas Yes, im consistent with database versions! This is one of the files im running in nf core funcscan.
j
Also what command are you running when you run it manually @Philipp Glasmacher?
e
Can you tell me the IDs of one or two of these proteins that should have hits?
j
I just saw Anan added a bunch of hardcoded parameters...
Maybe that could influence it?
e
(I only kept these for #C084HP11VPF:
Copy code
ext.args = [
            "--applications ${params.interproscan_applications}",
            params.interproscan_enableprecalc ? '' : '--disable-precalc'
        ].join(' ').trim()
)
👍 1
p
I got the same issue in proteinannotator though, so should be an issue on my end!
e
Copy code
Can you tell me the IDs of one or two of these proteins that should have hits?
^
👍 1
p
k141_9919578_9 # 8071 # 9273 # 1 # ID=138_9;partial=00;start_type=ATG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.574	400
k141_9919578_8 # 6069 # 8063 # 1 # ID=138_8;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.599	664
k141_9919578_7 # 4648 # 6057 # 1 # ID=138_7;partial=00;start_type=ATG;rbs_motif=AGGA;rbs_spacer=5-10bp;gc_cont=0.606	469
k141_9919578_6 # 3469 # 4431 # -1 # ID=138_6;partial=00;start_type=ATG;rbs_motif=AGGA/GGAG/GAGG;rbs_spacer=11-12bp;gc_cont=0.595	320
k141_9919578_5 # 2997 # 3311 # -1 # ID=138_5;partial=00;start_type=ATG;rbs_motif=AGxAGG/AGGxGG;rbs_spacer=5-10bp;gc_cont=0.635	104
k141_9919578_4 # 2520 # 2882 # -1 # ID=138_4;partial=00;start_type=ATG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.565	120
k141_9919578_3 # 1874 # 2386 # -1 # ID=138_3;partial=00;start_type=ATG;rbs_motif=AGGAG;rbs_spacer=5-10bp;gc_cont=0.598	170
k141_9919578_2 # 972 # 1877 # -1 # ID=138_2;partial=00;start_type=ATG;rbs_motif=4Base/6BMM;rbs_spacer=13-15bp;gc_cont=0.579	301
k141_9919578_17 # 15485 # 19378 # 1 # ID=138_17;partial=01;start_type=ATG;rbs_motif=AGxAG;rbs_spacer=5-10bp;gc_cont=0.646	1298
k141_9919578_16 # 15120 # 15446 # 1 # ID=138_16;partial=00;start_type=ATG;rbs_motif=GGAGG;rbs_spacer=5-10bp;gc_cont=0.587	108
[zcbepgl@login12 cb5deb4cbae322d2561673c736bccf]$
e
None of these are in the file you sent xD Any chance there was a mistake with inputs?
p
Sorry @Evangelos Karatzas looked at wrong file so this should have: >k141_2639666_1 # 445 # 5781 # -1 # ID=6_1;partial=00;start_type=ATG;rbs_motif=GGA/GAG/AGG;rbs_spacer=5-10bp;gc_cont=0.570
k141_2639666_2 # 6471 # 6923 # 1 # ID=6_2;partial=00;start_type=ATG;rbs_motif=AGGA;rbs_spacer=5-10bp;gc_cont=0.598
e
Ok! I'll run the pipeline a little later and will let you know
p
And my inputs are just fasta files from nf-core MAG DAS tool . Amazing thank you so much!
And @James Fellows Yates this is the command when i run it manually: and i just point it to the db where the sh command donwloaded with the db.
e
I cant get anything on these two proteins while running ips
5.59_91.0
against db version
5.73-104.0
...It might be that you are using another version of ips or the db, outside the pipeline I guess? When I put the test protein alongside those two and search for
sfld
matches, it correctly returns the match for that one:
Copy code
GI|225038609|EFDID|719595|FULL	079fff43a0270e432d339ea71b6f0acf	350	SFLD	SFLDS00057	Glutaminase/Asparaginase	17	347	0.0	T	16-02-2026	-	-
🙏 1