https://nf-co.re logo
Join Slack
Powered by
today I was a little bored so I've started nf-csve...
# shameless-promo
j
today I was a little bored so I've started nf-csvext plugin. Basically why to create a process to manipulate csv files with bash tools when we can do it "natively" in our pipelines? still under development but hope to have a first release candidate soon https://incsteps.github.io/nf-csvext/
💯 1
👀 3
m
does this work with tsv too?
j
yes (will do)
m
So definitively love this, and I totally agree that no such things should be done in bash and waste a process. I usually try to do this kind of things in groovy so it runs on the head node, but a plugin would be nicer. Any other plans for additional functionalities? Why not adding it to Nextflow directly?
j
I'm thinking to include more functions as leftJoin & rightJoin for example, transpose (can be little complicated but...), tansform and so on
About to include in nextflow from my side there is no problem but maybe can be better to implement these functions as "incubator" by the moment
🚀 1
m
Perfect answers
What about csv creation?
j
ummmm as for example ?
m
I know that such things can be improved, especially with the workflow outputs
and that this is merely a hack, but since you're doing an amazing work, I'm just trying to see what you think
j
good idea, so can be a 
csv_create
operator (instead a function) and it "consumes" maps and saves them to the csv file ...
m
Oooh, I like where this is going
😄 1
j
Sooo I think I may have needed this like 2 weeks ago 😆 (splitting a bunch of CSVs, redistributing the rows, and merging them back together again) https://github.com/nf-core/mag/pull/835 and this mess https://github.com/nf-core/mag/blob/40f40c0a5b8495f02a07dd4ad968683a78e44b9b/subworkflows/local/PYDAMAGE_BINS/main.nf
😢 1
j
so sorry, 2 weeks ago I was busy deploying first version of nf-aspera https://incsteps.github.io/nf-aspera/
<end of spam/>
j
Haha, no worries, I meant more I'm very excited because it's very 'timely' !
And nf-aspera I'm also excited,m that should be added to 
fetchngs
And get stuff from ENA rather than NCBI
👀 1
m
I think I mentioned that too
👍 1
j
The 'officiakl' ENA downloading tool uses it even: https://github.com/enasequence/ena-ftp-downloader
j
csv_trim
implemented also started 
csv_create
but I need to work a little more on it https://incsteps.github.io/nf-csvext/
98d3d6553a6e4e38.png
👀 1
j
@Mahesh Binzer-Panchal any thoughts on this btw? Given I've adopted your tricks from the sample sheet generation?
But that looks really nice @Jorge!
💯 1
👍 1
j
about ENA, some progress, using these parameters
Copy code
remote_host : '<http://fasp.ebi.ac.uk|fasp.ebi.ac.uk>',
ssh_port : 33001,
remote_user : "fasp-public",
running the pipeline: 
println file('<aspera://ena/pub/databases/ena/doc/FT_current.txt').text>
👀 1
Copy code
nextflow run <http://readme_ena.nf|readme_ena.nf> 

 N E X T F L O W   ~  version 25.04.7

Launching `<http://readme_ena.nf|readme_ena.nf>` [astonishing_wozniak] DSL2 - revision: f9a2d22501


The DDBJ/ENA/GenBank 
Feature Table 
Definition

Version 11.3 October 2024



DNA Data Bank of Japan, Mishima, Japan.
EMBL-EBI, European Nucleotide Archive, Cambridge, UK.
GenBank, NCBI, Bethesda, MD, USA.


1 Introduction
2 Overview of the Feature Table format
👀 1
j
Ooooh exciting!
The thing for ENA and fetchNGS is if you give fetchNGS an ERR accession code, you can tell it to use the corresponding aspera link to download from the DNS. For example, be able to download the oldest Neanderthal oral microbiome: https://www.ebi.ac.uk/ena/browser/view/ERR3579688 (you might need to use the 'show column selection' to find the `fastq_aspera`column
j
Maybe it's time to create an áspera channel (?)
m
You mean #C09G6NN5F6X (
#aspera
on the nf-core slack)
You know you should ask me for channels
Open in Slack
PreviousNext
Powered by