https://nf-co.re logo
Join Slack
Powered by
Could it be that --mle , contrasts and --publish_d...
# crisprseq
j
Could it be that --mle , contrasts and --publish_dir_mode move is broken? I have 4 different contrasts defined as 4 different rows in the same file, which all work fine with RRA. When I use mle everything is fine as well, except for that it only ever outputs a random contrast out of the contrasts file (I assume). When I then turn on --publish_dir_mode move to test if it changes sth, I get an error. Can MLE only run on one contrast as opposed to rra?
m
Hi, this sounds definitely like a bug... which version of the pipeline are you using (main or dev)?
j
nf-core/crisprseq 2.3.0 main [db/4a6e63] process > NFCORE_CRISPRSEQCRISPRSEQ SCREENINGMAGECK_TEST (CD8_OVA_tumor_ms67,CD8_OVA_tumor_ms68,CD8_OVA_tumor_ms69_CD8_spleen_ms67,CD8_spleen_ms68,CD8_spleen_ms69) [100%] 4 of 4 ✔️ [66/9064aa] process > NFCORE_CRISPRSEQCRISPRSEQ SCREENINGMAGECK_GRAPHRRA (CD8_OVA_tumor_ms67,CD8_OVA_tumor_ms68,CD8_OVA_tumor_ms69) [100%] 4 of 4 ✔️ [ef/0f8ec5] process > NFCORE_CRISPRSEQCRISPRSEQ SCREENINGMATRICESCREATION (2) [100%] 4 of 4 ✔️ [c8/698b59] process > NFCORE_CRISPRSEQCRISPRSEQ SCREENINGMAGECK_MLE (CD8_OVA_tumor_ms67,CD8_OVA_tumor_ms68,CD8_OVA_tumor_ms69_vs_CD8_Ctrl_tumor_ms67,CD8_Ctrl_tumor_ms68,CD8_Ctrl_tumor_ms69) [100%] 1 of 1 ✔️ You see that with the same input mageck test runs four times, mageck MLE only once.
m
probably there are issues with input channel of the MAGECK_MLE process, but since I made significant changes to the code it would help if you test whether you encounter this problem with the development version as well
coudl you rerun the pipeline with the -r dev option?
if you still encounter the same problem, tomorrow I can take a look at the code and see whether I can fix the bug
j
For some reason MLE wont even run now, but also does not throw an error. I am running it locally on my computer. Before, I used a local version where I made some adaptations so i could use more of the native parameters of mageck rra and mle
m
Ok, can you share the contrast matrix and the header of the count matrix?
j
[no preview available]
screen3.csv
It butchered the formatting of the contrast file a bit
image.png
m
Hi, I think I found the bug with the dev version... when I recently fixed an issue I did not consider the possibility that the user could provide the count matrix instead of the samplesheet. I have pushed a bug fix in this branch: https://github.com/nf-core/crisprseq/tree/bugfix_mle_from_constrast_with_count_table ... you can go ahead and try it (it will take a couple of days to merge the PR to the dev branch) However, this may not be the end of it... since you already had problems with the main, I suspect there might be another problem. In case, feel free to reach out and I will try to help you.
j
Thanks a lot, I will try the fix. I tried to fix it myself, but I am not verse enough yet with nextflow to understand channels etc enough 😄
👍🏻 1
Unfortunately the results didn#t change
image.png
base) joachim@joachim-NP5x-6x-7x-SNx:~/Projects/external/crisprseq_dev/crisprseq$ git status On branch bugfix_mle_from_constrast_with_count_table Your branch is up to date with 'origin/bugfix_mle_from_constrast_with_count_table'. nothing to commit, working tree clean
m
I was kind of expecting that... it is the same issue you had with the master branch... let me look into this, I will let you know asap
j
Cheers!
It's so weird because it obviously works for MAGECK_TEST, and I wasn't able to find the crucial difference to MAGECK_MLE as to why it would behave differently
m
very weird... I have tried a test run with a setup similar to yours, but it seems I cannot reproduce the problem: I have two lines in my contrast file and I correctly get two different MLE runs
it is hard for me to make any test without being able to reproduce your issue
j
Can I have your test files so I can see where the difference is to my data and whether I can reproduce it on my end?
m
yes, of course
params.yaml
Copy code
count_table: "<https://raw.githubusercontent.com/nf-core/test-datasets/crisprseq/testdata/count_table.tsv>"
analysis: 'screening'
contrasts: "rra_contrasts.txt"
outdir: "output"
mle: true
rra_constrasts.txt
Copy code
treatment;reference
control;treatment
control1;treatment1
command line (you have to adapt it, obviously)
Copy code
nextflow run ../source/nf-core-crisprseq/ -params-file params.yaml -profile local_docker -c ../.nextflow/config
j
I got it, the data gave me an idea
NFC…RISPRSEQCRISPRSEQ SCREENINGMAGECK_MLE (CD8_OVA_tumor_ms67_CD8_OVA_tumor_ms68_CD8_OVA_tumor_ms69_CD8_OVA_tumor_ms70_vs_CD8_Ctrl_tumor_ms67_CD8_Ctrl_tumor_ms68_CD8_Ctrl_tumor_ms69_CD8_Ctrl_tumor_ms70) [100%] 4 of 4 ✔️
❤️ 1
The difference is you did not provide --mle_control_sgrna /home/joachim/Projects/crispr_kirsten/data/control_genes_sg2.txt
so with --mle_control_sgrna /home/joachim/Projects/crispr_kirsten/data/control_genes_sg2.txt it fails
m
ah perfect! this help me a lot... give me one minute to check what is going on with this option
j
MAGECK_MLE (ch_mle, INITIALISATION_CHANNEL_CREATION_SCREENING.out.mle_control_sgrna)
MAGECK_TEST runs only on one parameter
maybe if flattens the channel somehow
m
ah found the bug... it is a trivial channel mistake... in the line that you wrote, 
ch_mle
has one instance for each line of the contrast file, whereas 
INITIALISATION_CHANNEL_CREATION_SCREENING.out.mle_control_sgrna
is a single instance queue channel, so after the first instance is consumed you cannot run the MLE job any longer
give me a minute to fix it
j
I figured something like that, but I lack the nextflow skills to fix it 😄 teamwork makes the dreamwork, thanks alot !
🙌🏻 1
m
ok, I pushed another commit to the same repo, give it a try!
j
seems to work!
🍌 1
Can I ask why Channel.fromPath(params.mle_control_sgrna) behaves differently from Channel.value(file(params.mle_control_sgrna, checkIfExists: true)) ?
m
this is one of the core concept of nf (and unfortunately quite difficult to grasp)... there are two main types of channel, queue channels and value channels, the first ones are supposed to contain elements on which you want to repeat some operations and are "consumed" when they are passed to the job, while a value channel is a sort of immutable value that is passed to all the instances of a job... it is easier to see in practice... if you want to understand this, my suggestion is to go to the nextflow training: https://training.nextflow.io/2.4.0/archive/basic_training/channels/
j
Cheers !
Open in Slack
PreviousNext
Powered by