Hi, I am having troubles with the download GHA. <I...
# tools
n
Hi, I am having troubles with the download GHA. It fails. I do not know why, there is no meaningful error. Other tests that run work fine, could someone take a look at what I am doing wrong?
For example, there is this: `WARN: nf-core pipelines do not accept positional arguments. The positional argument
true
has been detected.` but the pipeline is called as
nextflow run ./genomeassembler/$( sed 's/\W/_/g' <<< dev) -stub -profile test,singularity --outdir ./results
and i dont see any true or positional argument, I also do not really know what the sed part is doing tbh
It is also not failing at the download step, but at the run pipeline step..?
m
that warning is a bit misleading. the “problem” is that
-stub
doesn’t have any value and is therefore triggering tthis warning . can be ignored. can you download the pipeline locally and run it with the command in the CI locally?
n
I do not have singularity setup so I have not tested this
m
does your pipelilne support stub runs?
n
The pipeline runs fine with docker / conda in the GHA.
does your pipelilne support stub runs?
maybe I have not tested stub runs, I have added stubs for the modules that did not have one and are part of the test, this did not help. But it should run afterwards in a non-stub, correct? That step also fails without any error message
By without error message I mean
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting
m
ah, there is an actual error in the step above:
Cannot invoke method clone() on null object
means one of the modules doesn’t have a stub mode (yet), see discussion here https://nfcore.slack.com/archives/C07BDPBPLG3/p1734347460850619
n
Ah, ok so I will add stubs. But why does the run in non-stub mode fail?
So for stub to work all modules need a stub?
m
yep
but i don’t think it is required for this action to actually pass, so just a side thing
i will try to replicate this in a gitpod
n
thanks
It is probably something stupid I am doing..
m
okay, it worked on gitpod … 🤔 let’s restart the action, maybe it just had some hiccup
n
Ok, thanks for taking a look at this
It failed. Is it possible that the gha is running on
dev
and not the branch that is in the PR? I.e. I would only see if I fixed it after merging? I have enabled GHA to trigger on PRs (to see if it works before merging) but it is not unlikely that I forgot to change something and it is still running dev..
m
you can trigger it manually from whatever branch you want https://github.com/nf-core/genomeassembler/actions/workflows/download_pipeline.yml
n
the branch is on my fork of the repo, I have no runners
I will try to move the branch..
So even if i trigger it from another branch, it will still test dev? In any case, it is very strange that it works for your in a gitpod, but never worked as an action
Well, that was not the reason it fails, I guess I will have to live with that test failing for now..
m
@Matthias Zepper do you see anything suspicious causing the pipeline run to fail after a successful download (even though it runs in gitpod with singularity)
m
I also do not really know what the sed part is doing tbh
Generating the correct path to the download directory. If you are downloading the
dev
revision of genome assembler, the pipeline code will be in
genomeassembler/dev/
. But if you are downloading the revision
v2.10
, it will be
genomeassembler/v2_10/
Because you can download arbitrary branches, I can’t hardcode the full output path in the action. But when it tries running
$( sed 's/\W/_/g' <<< dev)
, you are indeed downloading and testing
dev
and not your featurebranch. In your more recent runs, it is then
gha_download
. Runs in your own fork should be available, since it is an open-source repo. You can launch them e.g. like so:
Copy code
gh workflow run -R "nschan/genomeassembler" --ref "fix_ci_download"  --field "testbranch=nschan:fix_ci_download" "download_pipeline.yml"
-R
is the repo, therefore
nf-core/genomeassembler
or
nschan/genomeassembler
etc.
--ref
is the branch from which you are using the action and
--field
specifies the branch you would like to test downloading.
Regarding the error. Unfortunately the latest addition to the action, making the
.nextflow.log
file available as artifact has not yet been rolled out. That should in the future make it easier to troubleshoot a failing pipeline. To me the
killed
implies that the process may be terminated by the executor. Does the
ASSEMBLE:HIFIASM
module correctly implement the CPU/memory limits and are they also applied to the downloaded pipeline correctly e.g. with a correct config base?
If you like, add this step at the end to one of those branches that you use to experiment with the action:
Copy code
- name: Upload Nextflow logfile for debugging purposes
        uses: actions/upload-artifact@v4
        with:
          name: nextflow_logfile.txt
          path: .nextflow.log*
          include-hidden-files: true
n
Resource limits for all processess are set in the test profile, I dont see why HIFIASM wouldnt respect those, other tests using the test profile work fine.
m
Just a shot in the dark from my side, I didn't look at the module code etc. If it works on Gitpod and also the regular pipeline test works, I think it can be ignored ?!? Otherwise, try to get the .nextflow.log with that additional step and see if it gives you more information.