Running into something a bit interesting. The following error in markduplicates:

WARNING	2026-02-19 01:05:51	AbstractOpticalDuplicateFinderCommandLineProgram	A field field parsed out of a read name was expected to contain an integer and did not. Read name: RMHC001_haystack_tumour:1. Cause: String 'RMHC001_haystack_tumour' did not start with a parsable number.

and then the subsequent error after processing some of the reads (only getting into chr3 or so)

[Thu Feb 19 01:12:13 GMT 2026] picard.sam.markduplicates.MarkDuplicates done. Elapsed time: 6.42 minutes.
Runtime.totalMemory()=16005464064
To get help, see <http://broadinstitute.github.io/picard/index.html#GettingHelp>
htsjdk.samtools.SAMException: Value was put into PairInfoMap more than once.  3: RGRMHC001_haystack_tumour:12818437
	at htsjdk.samtools.CoordinateSortedPairInfoMap.ensureSequenceLoaded(CoordinateSortedPairInfoMap.java:133)
	at htsjdk.samtools.CoordinateSortedPairInfoMap.remove(CoordinateSortedPairInfoMap.java:86)
	at picard.sam.markduplicates.util.DiskBasedReadEndsForMarkDuplicatesMap.remove(DiskBasedReadEndsForMarkDuplicatesMap.java:61)
	at picard.sam.markduplicates.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:560)
	at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:270)
	at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:280)
	at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:37)
	at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:166)
	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:209)
	at org.broadinstitute.hellbender.Main.main(Main.java:306)

this case was sequenced across two lanes, and one of them is the top-up, but after some grepping I realise that there are reads with the same name in some of the parallelised .bam files - even from the same lane

Hey! Markduplicates does not like that. is one lane already completely contained in the second? Otherwise I have use seqkit in the past to remove reads of the same name (in my case for some reason some reads truly were in two files down to the last nucleotide and quality value)

it's the same library, run twice - and labelled with different lanes on the samplesheet

right?

are those 5 important?

Is any read truly important? 🤔

hm but that shouldn't duplicate reads. It just divides the fastq files for faster alignment (unless fastp is doing something crazy). and if they are duplicated in your original files already..

well I did just grep one read name

ah ok. That makes it a bit better tbh 😄 I mean you could turning off splitting (how big are your fastq files?) but I really don't think it would change anything. GATK MArkdup needs unique read names cause it puts them in a map. I found this entry with two suggestions: Using AddOrReplaceReadgroups (sadly not implemented in sarek) or double checking that the demultiplexing worked fine: https://gatk.broadinstitute.org/hc/en-us/community/posts/4408717387803-SAMException-Value-was-put-into-PairInfoMap-more-than-once

Here are the offending reads, if that's helpful (4, not 5 - one was supplemental alignment which I don't show here)

RMHC001_haystack_tumour:12818437	65	chr2	29213724	60	150M	chr4	107480126	0	CTGAAATCCTAGCATTTCAGGAGGCCAAGGTGGGAGGATTACTTGAGCCCAGAAATTCGAGACCAGCCTGGGCAACAAAGTGAGACCCTGTCTCTCACAAAAAAACAATAAAACAAAGCTGAATCATCCTACATCCAAATGGCTCTGGAG	EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE	NM:i:1	MD:Z:43G106	MC:Z:76M74S	MQ:i:60	AS:i:146	XS:i:48	RG:Z:null.RMHC001_haystack_tumour.L001
RMHC001_haystack_tumour:12818437	129	chr4	107480126	60	76M74S	chr2	29213724	0	TGTAAATATATATTCACACAGCACTGGAGTACCCAGATTTATAAAGCAAACATTCCTAGATCTAAAAAGAAAGACTTCTGCACGTGGCTCGGGACATTGCCTGTGGCTGTCAGTATTTGGAGGAAAACCACTTCATCCACCGGTGAGTCA	EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE8EEEEEEEEEEEE	NM:i:0	MD:Z:76	MC:Z:150M	MQ:i:60	AS:i:76	XS:i:37	RG:Z:null.RMHC001_haystack_tumour.L001	SA:Z:chr2,29213976,-,76M74S,60,0;

RMHC001_haystack_tumour:12818437	129	chr2	32916476	0	5S145M	chr4	3441943	0	NNGNNGGGGGGGGGGGGNGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG	##8##8EE8EE8EE88E#8EE8E88EEEEEEEEEEEEEEEEEEEEEEEEEE8EE8EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE8EEEEEEEE8E8EEE8EE8EEE88EEEEEEEEEEEEEEEEEEEEEEEEE8EEEEEEEEEEEEEE	NM:i:6	MD:Z:10A1G68A23C5C10A22	MC:Z:150M	MQ:i:60	AS:i:123	XS:i:122	RG:Z:null.RMHC001_haystack_tumour.L002	XA:Z:chr2,+32916242,7S105M5D38M,8;chr2,+32916230,6S117M5D27M,9;
RMHC001_haystack_tumour:12818437	65	chr4	3441943	60	150M	chr2	32916476	0	NNCCTGCCTTGGCCGCTCTGCTCCCGCCTCCCACTGCCCCTCAGGCCAGCTCAGGAGCCATGGGGCGCTGGGCCTGGGTCCCCAGCCCCTGGCCCCCACCGGGGCTGGGCCCCTTCCTCCTCCTCCTCCNGCTGCTGCTGCTGCTGCCAC	##888EEEEEEEEEEEEEEEEEEEEEEEEEEEE8EEEEEEEEEEEEEEEEEE8EEE8EEEEEEE8EEE8E8EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE8EEE#EE8EEEEEEEEEEEE8EEEE	NM:i:3	MD:Z:0C0A127T20	MC:Z:5S145M	MQ:i:0	AS:i:146	XS:i:20	RG:Z:null.RMHC001_haystack_tumour.L002

yep probably the best. afaik no need to do duplicate marking when using UMIs right @Simon Pearce?

also I realised that I'm on 3.5.0, so uh, time to update

if using the fgbio consensus workflow, duplicate marking doesn't make sense

I'm using the parameter: --umi_read_structure '9M+T +T' As I have appended the UMI strings to the start of the R1 fastq files.