<@U03CKGEU3LZ> file exists checking is default isn...
# meta-omics
j
@Nicolas Vannieuwkerke file exists checking is default isn't it?
n
Nope you need to add
"exists":true
to the schema for exist checking to happen
s
yeah, that's in the input schema here 🙂
j
Or is that wrong place
s
that's what I'm referencing
n
You need to add it to the first
anyOf
?
That way it will only run through that check if it's a file 😛
I think some weird things happen if you don't put it in there
j
n
yes! 😛
j
NGL I don't find Json that intuitive
What are the rules guiding what does inside anyOf and not
n
I think this is mainly something that could have been implemented better in nf-schema (need to take a look at that some time), it basically checks if the format is one of the file paths but it doesn't find that format in this case because it's nested inside the
anyOf
j
As in validation option of the (single) object itself should go into any of?
n
I'm not following what you mean 😁
j
Sorry, I guess I'm not getting what you mean too 😅
To summraise I hope: I don't understand which attributes go into anyOf and vice versa
n
I'd say if you use
anyOf
you should add as much into it as you can 😛 This will make sure nf-schema works optimally
j
So then in my PR, I should move the type/maxLength also into anyOf.?
n
Yeah true, you can just remove the
anyOf
at this point 😛.
uniqueItems
will also do nothing in this case because
fasta_aa
is not a list 😛
I'll try and explain it as best as I can in the bytesize next week 😛
j
I'm completley lost now
So I don't need anyOf at all?
Or I move uniqItems outside of anyOf?
(I thought maxLength was the length of the string?)
uniqueItems will also do nothing in this case because fasta_aa is not a list 😛
Ah have I forgot to move the uniqeItems thing 😆
facepalm
n
Yeah you won't need
anyOf
in 99% of the cases 😛
It mostly complicates stuff that could actually be super simple
Copy code
"fasta_aa": {
                "type": "string",
                "format": "file-path",
                "pattern": "^\\S+\\.(fasta|fas|fa|faa)(\\.gz)?$",
                "exists": true,
                "errorMessage": "FASTA file for amino acid reference sequence cannot contain spaces and must have a valid FASTA extension (fasta, fna, fa, fas, faa), optionally gzipped"
            }
This should be enough in your case 😛
j
Updated accrodingly @Nicolas Vannieuwkerke https://github.com/nf-core/createtaxdb/pull/62/files But now I'm not understanding how to check that e.g. all file-paths in my e.g.
fasta_dna
column are unique?
https://nextflow-io.github.io/nf-schema/latest/nextflow_schema/nextflow_schema_specification/#uniqueentries So uniqueItems you earlier implied only works if you have a list of objects in a single 'cell' of the input samplesheet (is that correct?)
But uniqueEntries is for combinations of two columns
n
No `uniqueItems`will check the uniqueness of full rows. I don't think this is useful in 99% of the samplesheets but that's a JSON schema native feature
👍 1
indeed using
uniqueEntries
is the way to go here 😛
"uniqueEntries": ["fasta_dna"]
should work for the example you gave 😛
j
And if I want to do the same with fasta_aa?
Without checking the 'combination'
n
Than you can use an `allOf`:
Copy code
"allOf": [
  { "uniqueEntries":["fasta_dna"] },
  { "uniqueEntries":["fasta_aa"] }
]
This simply checks if all these schemas pass 😛
j
Ah ok, so unique uniqueEntries per column, got it
🎉 1
n
which in this case means that it will check if all values in
fasta_dna
are unique and the same thing for column
fasta_aa
👍 1
(I know it can get very confusing, sorry, I didn't design JSON schema 😁)
j
YEah no worries!
I just can't tell what is JSON schema and nf-schema unforuantely
n
If you see this tag under the documentation of a keyword in the nf-schema docs, the keyword is nf-schema specific:
j
Ah ok
n
All other keywords are JSON schema specific (we don't explain all in the docs of course)
👍 1
j
n
Hmmm good point, can you open an issue for that real quick so I don't forget about it? 😛
salute 1
j
And the example doesn't amake sense to me... I think maybe my issue is I'm going to nf-schema veryspecifically with samplehseets in my mind, which I guess is not the only thing it can do
But having more practical examples will help a lot, as I think that's why I've been confused in a lot of the docs
the explanations don't map with my 'understanding'
n
but yeah this is maybe not that clear 🤔
j
Yeah I know I've seen them but very often they are much too simple
It's always RNAseq examples which IMO is quite bad 😆
No uniqeEtnries there for example
No example of combinations
etc. etc. (if you see what I mean)
n
Good point! I need to create some examples anyway for the bytesize next week. I'll try and create some more complex ones and add them to the docs after the bytesize 😛
j
Hrm, @Nicolas Vannieuwkerke don't think the uniqueEtnries thing is working
Copy code
Process `NFCORE_CREATETAXDB:CREATETAXDB:CAT_CAT_AA` input file name collision -- There are multiple input files for each of the following file names: sarscov2.faa
The pipeline should'nt have got rthat far
Uh ok the schea input isn't working at all now wtf
n
Can you show me your schema? 😛
j
No wait that might be me, let me check first
👍 1
It's all there
Or have I put everything in the wrong place
n
You need to put your
uniqueEntries
up a level 😛
j
facepalm
n
You are annotating the properties this way, but
properties
doesn't do anything with
uniqueEntries
so it gets skipped 😛
j
ooookkk
n
I'm wondering how I can easily explain this stuff 🤔
The bytesize might be more of a challenge than I first expected it to be 😁
j
I think it's more about finding out from people what the most common things they want to valid for
validate*
and make sure you have realisitc examples in all cases
I think the schema is now working properly 😄
🎉 1
n
Yeah you are right! I'll post something in nf-schema to ask for common pitfalls and gaps in the examples 😛
👍 1