hi again, my first trial was a success but for the...
# chipseq
m
hi again, my first trial was a success but for the real run i an having problems. there was this specific sample: 3k27ac_ms_b_ENCSR364OIK. At first it was giving error, so i redownloaded everything. all my 153 samples were uploaded and aligned but this time it gives a trimming problem. this is the error: trimming : ERROR ~ Error executing process > 'NFCORE_CHIPSEQCHIPSEQFASTQC_TRIMGALORE:TRIMGALORE (h3k27ac_ms_b_ENCSR364OIK_T1)' Caused by: Process
NFCORE_CHIPSEQ:CHIPSEQ:FASTQC_TRIMGALORE:TRIMGALORE (h3k27ac_ms_b_ENCSR364OIK_T1)
terminated with an error exit status (1) Command executed: [ ! -f h3k27ac_ms_b_ENCSR364OIK_T1.fastq.gz ] && ln -s h3k27ac_ms_b_ENCSR364OIK.fastq.gz h3k27ac_ms_b_ENCSR364OIK_T1.fastq.gz trim_galore \ --fastqc \ --cores 4 \ --gzip \ \ \ h3k27ac_ms_b_ENCSR364OIK_T1.fastq.gz cat <<-END_VERSIONS > versions.yml "NFCORE_CHIPSEQCHIPSEQFASTQC_TRIMGALORE:TRIMGALORE": trimgalore: $(echo $(trim_galore --version 2>&1) | sed 's/^.*version //; s/Last.*$//') cutadapt: $(cutadapt --version) END_VERSIONS Command exit status: 1 Command output: pigz 2.6 Command error: ERROR: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 559, in run for chunk_index, chunk in enumerate(dnaio.read_chunks(f, self.buffer_size)): File "/usr/local/lib/python3.9/site-packages/dnaio/chunks.py", line 81, in read_chunks bufend = f.readinto(memoryview(buf)[start:]) + start # type: ignore File "/usr/local/lib/python3.9/gzip.py", line 300, in read return self._buffer.read(size) File "/usr/local/lib/python3.9/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/local/lib/python3.9/gzip.py", line 487, in read if not self._read_gzip_header(): File "/usr/local/lib/python3.9/gzip.py", line 435, in _read_gzip_header raise BadGzipFile('Not a gzipped file (%r)' % magic) gzip.BadGzipFile: Not a gzipped file (b'Fi') ERROR: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 559, in run for chunk_index, chunk in enumerate(dnaio.read_chunks(f, self.buffer_size)): File "/usr/local/lib/python3.9/site-packages/dnaio/chunks.py", line 81, in read_chunks bufend = f.readinto(memoryview(buf)[start:]) + start # type: ignore File "/usr/local/lib/python3.9/gzip.py", line 300, in read return self._buffer.read(size) File "/usr/local/lib/python3.9/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/local/lib/python3.9/gzip.py", line 487, in read if not self._read_gzip_header(): File "/usr/local/lib/python3.9/gzip.py", line 435, in _read_gzip_header raise BadGzipFile('Not a gzipped file (%r)' % magic) gzip.BadGzipFile: Not a gzipped file (b'Fi') ERROR: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 626, in run raise e gzip.BadGzipFile: Not a gzipped file (b'Fi') Traceback (most recent call last): File "/usr/local/bin/cutadapt", line 10, in <module> sys.exit(main_cli()) File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 848, in main_cli main(sys.argv[1:]) File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 913, in main stats = r.run() File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 825, in run raise e gzip.BadGzipFile: Not a gzipped file (b'Fi') Cutadapt terminated with exit signal: '256'. Terminating Trim Galore run, please check error message(s) to get an idea what went wrong... Work dir: /cta/users/merve.kaftancioglu/alternative_scenario/for_concat/work/b9/7deb506bb49a2adcd1cc8f49a4b288 Tip: you can replicate the issue by changing to the process work dir and entering the command
bash .command.run
-- Check '.nextflow.log' file for details executor > local (159) [a4/217800] process > NFCORE_CHIPSEQCHIPSEQPREP... [100%] 1 of 1 ✔️ [e0/2a007b] process > NFCORE_CHIPSEQCHIPSEQPREP... [100%] 1 of 1 ✔️ [98/7c9cb0] process > NFCORE_CHIPSEQCHIPSEQPREP... [100%] 1 of 1 ✔️ [4a/a028fe] process > NFCORE_CHIPSEQCHIPSEQINPU... [100%] 1 of 1 ✔️ [67/a94877] process > NFCORE_CHIPSEQCHIPSEQFAST... [100%] 153 of 153 ✔️ [b9/7deb50] process > NFCORE_CHIPSEQCHIPSEQFAST... [ 0%] 1 of 153, failed: 1 [- ] process > NFCORE_CHIPSEQCHIPSEQALIG... - [- ] process > NFCORE_CHIPSEQCHIPSEQALIG... - [- ] process > NFCORE_CHIPSEQCHIPSEQALIG... - [- ] process > NFCORE_CHIPSEQCHIPSEQALIG... - [- ] process > NFCORE_CHIPSEQCHIPSEQALIG... - [- ] process > NFCORE_CHIPSEQCHIPSEQALIG... - [- ] process > NFCORE_CHIPSEQCHIPSEQPICA... - [- ] process > NFCORE_CHIPSEQCHIPSEQMARK... - [- ] process > NFCORE_CHIPSEQCHIPSEQMARK... - [- ] process > NFCORE_CHIPSEQCHIPSEQMARK... - [- ] process > NFCORE_CHIPSEQCHIPSEQMARK... - [- ] process > NFCORE_CHIPSEQCHIPSEQMARK... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQFILT... - [- ] process > NFCORE_CHIPSEQCHIPSEQPRES... - [- ] process > NFCORE_CHIPSEQCHIPSEQPICA... - [- ] process > NFCORE_CHIPSEQCHIPSEQPHAN... - [- ] process > NFCORE_CHIPSEQCHIPSEQMULT... - [- ] process > NFCORE_CHIPSEQCHIPSEQBEDT... - [- ] process > NFCORE_CHIPSEQCHIPSEQUCSC... - [- ] process > NFCORE_CHIPSEQCHIPSEQDEEP... - [- ] process > NFCORE_CHIPSEQCHIPSEQDEEP... - [- ] process > NFCORE_CHIPSEQCHIPSEQDEEP... - [- ] process > NFCORE_CHIPSEQCHIPSEQDEEP... - [- ] process > NFCORE_CHIPSEQCHIPSEQMACS... - [- ] process > NFCORE_CHIPSEQCHIPSEQFRIP... - [- ] process > NFCORE_CHIPSEQCHIPSEQMULT... - [- ] process > NFCORE_CHIPSEQCHIPSEQHOME... - [- ] process > NFCORE_CHIPSEQCHIPSEQPLOT... - [- ] process > NFCORE_CHIPSEQCHIPSEQPLOT... - [- ] process > NFCORE_CHIPSEQCHIPSEQMACS... - [- ] process > NFCORE_CHIPSEQCHIPSEQHOME... - [- ] process > NFCORE_CHIPSEQCHIPSEQANNO... - [- ] process > NFCORE_CHIPSEQCHIPSEQSUBR... - [- ] process > NFCORE_CHIPSEQCHIPSEQDESE... - [- ] process > NFCORE_CHIPSEQCHIPSEQIGV - [- ] process > NFCORE_CHIPSEQCHIPSEQCUST... - [- ] process > NFCORE_CHIPSEQCHIPSEQMULTIQC - Execution cancelled -- Finishing pending tasks before exit ERROR ~ Error executing process > 'NFCORE_CHIPSEQCHIPSEQFASTQC_TRIMGALORE:TRIMGALORE (h3k27ac_ms_b_ENCSR364OIK_T1)' Caused by: Process
NFCORE_CHIPSEQ:CHIPSEQ:FASTQC_TRIMGALORE:TRIMGALORE (h3k27ac_ms_b_ENCSR364OIK_T1)
terminated with an error exit status (1) Command executed: [ ! -f h3k27ac_ms_b_ENCSR364OIK_T1.fastq.gz ] && ln -s h3k27ac_ms_b_ENCSR364OIK.fastq.gz h3k27ac_ms_b_ENCSR364OIK_T1.fastq.gz trim_galore \ --fastqc \ --cores 4 \ --gzip \ \ \ h3k27ac_ms_b_ENCSR364OIK_T1.fastq.gz cat <<-END_VERSIONS > versions.yml "NFCORE_CHIPSEQCHIPSEQFASTQC_TRIMGALORE:TRIMGALORE": trimgalore: $(echo $(trim_galore --version 2>&1) | sed 's/^.*version //; s/Last.*$//') cutadapt: $(cutadapt --version) END_VERSIONS Command exit status: 1 Command output: pigz 2.6 Command error: ERROR: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 559, in run for chunk_index, chunk in enumerate(dnaio.read_chunks(f, self.buffer_size)): File "/usr/local/lib/python3.9/site-packages/dnaio/chunks.py", line 81, in read_chunks bufend = f.readinto(memoryview(buf)[start:]) + start # type: ignore File "/usr/local/lib/python3.9/gzip.py", line 300, in read return self._buffer.read(size) File "/usr/local/lib/python3.9/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/local/lib/python3.9/gzip.py", line 487, in read if not self._read_gzip_header(): File "/usr/local/lib/python3.9/gzip.py", line 435, in _read_gzip_header raise BadGzipFile('Not a gzipped file (%r)' % magic) gzip.BadGzipFile: Not a gzipped file (b'Fi') ERROR: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 559, in run for chunk_index, chunk in enumerate(dnaio.read_chunks(f, self.buffer_size)): File "/usr/local/lib/python3.9/site-packages/dnaio/chunks.py", line 81, in read_chunks bufend = f.readinto(memoryview(buf)[start:]) + start # type: ignore File "/usr/local/lib/python3.9/gzip.py", line 300, in read return self._buffer.read(size) File "/usr/local/lib/python3.9/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/local/lib/python3.9/gzip.py", line 487, in read if not self._read_gzip_header(): File "/usr/local/lib/python3.9/gzip.py", line 435, in _read_gzip_header raise BadGzipFile('Not a gzipped file (%r)' % magic) gzip.BadGzipFile: Not a gzipped file (b'Fi') ERROR: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 626, in run raise e gzip.BadGzipFile: Not a gzipped file (b'Fi') Traceback (most recent call last): File "/usr/local/bin/cutadapt", line 10, in <module> sys.exit(main_cli()) File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 848, in main_cli main(sys.argv[1:]) File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 913, in main stats = r.run() File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 825, in run raise e gzip.BadGzipFile: Not a gzipped file (b'Fi') Cutadapt terminated with exit signal: '256'. Terminating Trim Galore run, please check error message(s) to get an idea what went wrong... Work dir: /cta/users/merve.kaftancioglu/alternative_scenario/for_concat/work/b9/7deb506bb49a2adcd1cc8f49a4b288 Tip: you can replicate the issue by changing to the process work dir and entering the command
bash .command.run
-- Check '.nextflow.log' file for details
I excluded the sample which created the problem, but now, another one appeared. i re-downloaded and checked these samples. what could be wrong, do you have any suggestions please?
a
What is your filename that you are inputting? ;Looks like you are not gzipping your input files per this error message:
Copy code
line 825, in run raise e gzip.BadGzipFile: Not a gzipped file (b'Fi') Cutadapt terminated with exit signal: '256'.
command to use:
Copy code
gzip filename.fa
m
Thank you so much for returning. The file name h3k27ac_ms_b_ENCSR364OIK. The original files i downloaded were all gunzipped. All i did was concatanate to make them one file. Once i encountered this problem, i redownloaded them, and concatanated these new files from the beginning. So i am suspecting of some corruption of the files or there is a problem during upload of the inputs.
a
If they were all g-unzipped, then it sounds like you need to gzip the file (concatenated or not)
m
I concatanated as they are gunzipped and all of them are. Interestingly only some of these files are giving problem. So far 60 of them gone without trouble
when i re-gunzip this is the error: gzip: h3k27ac_ms_b_ENCSR364OIK.fastq.gz already has .gz suffix -- unchanged
a
I think you are meaning to say gzipped? So your files end in .fastq.gz?
Can you tell me exactly what input you are uploading and how you exactly prepared that input? Starting from your download command.
m
they are gzipped all the time, from download to uploading to input. they had never been ungunzipped. i downloaded, i concatenated, and i uploaded. that's all
a
I guess I don't fully understand what you are doing - can you please provide your nextflow run nf-core/chipseq command that you are using?
m
the text which is attached is for the run. it worked perfectly for the 2 samples i tried. if i need to explain what i had done so far: i got the sequence such as this: wget https://www.encodeproject.org/files/ENCFF594QFH/@@download/ENCFF594QFH.fastq.gz but every sample has 30-40 fastq.gz files, so i needed to concatenated them. i did concatenated each of them with cat command, then i put them under a folder.