Hi! I am wondering how the hash values are (or use...
# nf-test
t
Hi! I am wondering how the hash values are (or used to be?) created when passing files to a snapshot assert? E.g. the `default.nf.test` does something like
Copy code
def stable_name = getAllFilesFromDir(params.outdir, relative: true, includeDir: true, ignore: ['pipeline_info/*.{html,json,txt}'])
def stable_file = getAllFilesFromDir(params.outdir, ignoreFile: 'tests/.nftignore')

{ assert snapshot(
    removeNextflowVersion("$outputDir/pipeline_info/qcflow_software_mqc_versions.yml"),
    stable_name,
    stable_file
).match() }
and somehow the hashes magically appear in the snapshot behind the corresponding file, like
Copy code
[
    "Sample1.bam:md5,b659fef4c129faf27bb69210b6a9fcc7",
    "Sample1.bam.bai:md5,da403861301abccd8f6314c6390f7989",
    ...
]
for the
stable_file
. However, if I now do this again in another test, i.e. passing a few files collected via
getAllFilesFromDir()
or
getAllFilesFromPath()
to the
snapshot()
, then I don't have the hashes but only the relative file paths like in
stable_name
. Can you remind me, what is the mechanics that adds the hashes?
m
that's internal nf-test that does md5sum snapshot
theses getAllFilesFromPath and getAllFilesFromDir functions are just sugar coating syntax to simplify the code and capture multiple files in one go
check https://nf-co.re/nft-utils for more info about these functions
do not hesitate to open issues or PRs if documentation is not clear enough
we're trying to be good, but I find it particularly hard to write docs for things I write myself
t
No blame, there is plenty of documentation and I read the nf-test and nft-utils doc to understand what's going on. ✨
But I do not see why
stable_name
files are not getting the hash suffix in the snapshot, but
stable_files
do.
m
er, if I remember well it's probably a question of getting a relative path or not
and that's probably lacking in the docs indeed
t
hmmm... I tested your hypothesis
Copy code
def baiFiles = getAllFilesFromPath(params.outdir, relative: true, include: ["**/bam/*.bai"])
def baiFilesMD5 = getAllFilesFromPath(params.outdir, relative: false, include: ["**/bam/*.bai"])
but it did not just not print the hashes but also repeated the relative paths... 😞 Result:
Copy code
[
                "path/to/my/Sample1.bam.bai",
                "path/to/my/Sample2.bam.bai",
                "path/to/my/Sample3.bam.bai",
                "path/to/my/Sample4.bam.bai",
            ],
            [
                "path/to/my/Sample1.bam.bai",
                "path/to/my/Sample2.bam.bai",
                "path/to/my/Sample3.bam.bai",
                "path/to/my/Sample4.bam.bai",
            ],
[nft-utils-1.0.0]
m
What about includeDir?
t
🤷 it's bit frustrating to test the parameters until I find the one that does the trick 😅 I'll keep trying another day. It's late in my timezone... thanks for your input though!
I leave this note here in case people Ctrl+F this issue at some point. I'll open an issue on nft-utils since it is genuinely a shortcoming of the new
getAllFilesFromPath()
function (and
getAllFilesFromDir()
is depreciated). TL;DR: use the old function for the time being and specify
relative:true
. So, @Maxime Garcia you were kind of correct on this one 🙂
m
I'll have a look right away
t