Hey, guys! I have a question about developing my o...
# tools
v
Hey, guys! I have a question about developing my own nf-core repository with custom subworkflows and modules. I'm trying to create a subworkflow that uses both personal processes and nf-core modules, which can be imported as an nf-core subworkflow into any Nextflow repository using a command like this:
Copy code
nf-core subworkflows --git-remote git@github.com:MyProject/Modules.git --branch main install clean_and_assembly
However, I'm facing two issues: 1. I can't install nf-core modules directly in my repository since it's a clone of
nf-core/modules
.
Copy code
ERROR    You cannot install a bowtie2/align in a clone of nf-core/modules
2. When I install a subworkflow that uses nf-core modules, the personal modules are installed correctly, but the original nf-core modules are missing:
Copy code
INFO     Installing 'clean_and_assembly'                                                                              
ERROR    Module 'bowtie2/align' not found in list of available modules.                                               
INFO     Use the command 'nf-core modules list' to view available software                                            
ERROR    Module 'genomescope2' not found in list of available modules.                                                
INFO     Use the command 'nf-core modules list' to view available software                                            
ERROR    Module 'quast' not found in list of available modules.                                                       
INFO     Use the command 'nf-core modules list' to view available software                                            
INFO     Use the following statement to include this subworkflow:                                                     
                                                                                                                      
 include { CLEAN_AND_ASSEMBLY } from '../subworkflows/bmc/clean_and_assembly/main'
Then, when I try to install any other modules, I get an error due to these missing nf-core modules:
Copy code
ERROR    'bowtie2/align'
What do you recommend me to do?
j
Hi @Vinícius Castro Santos, creating subworkflows which use both nf-core and custom modules is not supported
you will have to copy the nf-core modules into your repository, as if they were custom modules, to be able to use them
this might be supported in the future, you can follow https://github.com/nf-core/tools/pull/3083, https://github.com/nf-core/tools/issues/3175 or https://github.com/nf-core/tools/issues/3174 to be updated with the discussion/development
v
I see. Thank you very much for your response, Julia! I will keep myself updated on these discussions
Hi again, @Júlia Mir! May I ask you another question about the same topic, please? I'm facing issues when updating and installing new modules/subworkflows across different users. Sometimes, nf-core detects the updates, but other times it does not. We usually use the following command to update a module:
Copy code
nf-core modules --git-remote https://github.com/MyProject/Project.git --branch feat/new_modules update module
However, the behavior is inconsistent. We are all using the latest nf-core version (3.2.0), but not all of us can update our pipeline with the new modules.
j
This command looks light, can you share the output that you see on the terminal when you run the command? You can also run it with
nf-core --verbose modules ...
to get a more complete log
and can you also check your
modules.json
file? the versions of the modules are listed there, this file shouldn't be updated manually, but the nf-core modules commands use it as a reference and update it when required to handle the modules in a pipeline
v
Hi, @Júlia Mir. I figured it out! The HTTPS authentication didn’t work. I had to use SSH, and then it worked smoothly:
git@github.com:MyProject/Project.git
My own mistake cause we only use SSH on Github
Thank you for the fast response anyway!