So what I'm looking it at is that I have an object which is bascially my initial graph state that I will be running. one of the fields it has is

description

- which is a ground-truth, human made description. My agent is producing a field in output called

generated_description

which is of course the description agent did. I would like to run the

similar

metric on the two, ideally using the default test. How can I configure the test assertion so that it would reference

description

from the variables of the test case instead of hard-coding it? something like this:

defaultTest:
  assert:
    - type: similar
      provider: azure:embeddings:text-embedding-3-small
      value: {{input_state.description}}
      transformm: 'output.generated_description'

Yes! Variable references in assertions are fully supported. Your config just needs two small fixes:

defaultTest:
  assert:
    - type: similar
      provider: azure:embedding:your-deployment-name  # Use your Azure deployment name
      value: '{{input_state.description}}'           # :white_check_mark: Must be quoted
      transform: 'output.generated_description'      # :white_check_mark: Fixed typo (was transformm)
      threshold: 0.7

Key points: 1. Add quotes around the variable template: '{{input_state.description}}' 2. Fix typo: transform not transformm 3. Azure syntax: Use your deployment name, not the model name This uses Nunjucks templating and works with any variable in test.vars, including nested objects like {{input_state.description}}. Works with all assertion types: similar, contains, equals, regex, etc.

thank you! fantastic; somehow I have missied that in the docs.