Hi Please, could somebody give me a hand about som...
# random
m
Hi Please, could somebody give me a hand about some infrastructure information? Having 256 prokaryote samples, (doing RNASeq/ TnSeq), how terabits I need if the project running for 6 years? Thanks a lot
i
It's quite hard to estimate the required disk space from this limited information. It depends on the sequencing platform, the target depth, the number of runs that you intend to perform, and the approach to raw sequencer files (i.e., how aggressively you're planning to remove/archive them). If you're preparing a data management plan, you should look at the stored outputs of previous runs and extrapolate to find the amount of short-term and long-term storage that you will need.
🙌 1
p
You might find this helpful: https://nf-co.re/blog/2020/data_management
🙌 1
m
So, keeping all the files for 6 years, 60 TB is exaggerative?
p
It depends on what the samples are and what your context is 🙂
🙌 1
But probably, yes. I suspect it'd be cheaper to repeat the experiments..
Or at least repeat the analysis runs
m
It also depends on the data you wish to keep around. If you plan to store the raw sequencing information long-term, then convert them from
fastq.gz
to a better archival format. There are a ton of tools out there, that bring the size down quite significantly, please see the results from an NGI internal benchmark that I did (short-read RNA-seq data in this case, results vary for high-complexity libraries or inconsistent read lengths such as Nanopore reads). If you keep the container image of the tool, you can be sure that you can always convert your data back, if it were to disappear. Most are on Bioconda though, so should be available in the future still.
👌 2
🙌 1
m
Sorry, so for a 6 years project by welcome Trust, two back up is required or arbitrarily?
i
You have to check with the grant provider to be sure, but 3-2-1 backup is always recommended for valuable data
m
For non-clinical/sensitive data (let's say RNAseq of bacteria), Is two physically separate copies enough for project’s requirements, or does that (usually) require a third copy (3-2-1 backup rule)? From where, I can check this from, please?