dev_nerd_41284
09/14/2024, 5:32 AMdev_nerd_41284
09/14/2024, 5:32 AMmichaelmichaelmichael
09/14/2024, 7:55 AMdev_nerd_41284
09/14/2024, 4:51 PMelsatch
09/15/2024, 11:46 PMyaml
tests:
- vars:
query: How did the student calculate the volume of the mold?
context: file://mold_volume.md
assert:
- type: context-relevance
threshold: 0.9
provider: openai:completion:explodinggradients/Ragas-critic-llm-Qwen1.5-GPTQ
- type: context-faithfulness
threshold: 0.9
provider: openai:completion:explodinggradients/Ragas-critic-llm-Qwen1.5-GPTQ
- type: answer-relevance
threshold: 0.9
provider:
text:
id: openai:completion:explodinggradients/Ragas-critic-llm-Qwen1.5-GPTQ
embedding:
id: ollama:embeddings:nomic-embed-text:latestelsatch
09/15/2024, 11:48 PMyaml
evaluateOptions:
maxConcurrency: 1
prompts:
- file://rag_prompt.txt
providers:
- id: ollama:chat:command-r:latest
defaultTest:
options:
provider:
text:
id: ollama:chat:command-r:latest
embedding:
id: ollama:embeddings:nomic-embed-text:latest
tests:
- vars:
query: What types of silicone did the student use for molding?
context: file://silicone_types.md
assert:
- type: context-relevance
threshold: 0.9
- type: context-faithfulness
threshold: 0.9
- type: answer-relevance
threshold: 0.9michaelmichaelmichael
09/15/2024, 11:56 PMelsatch
09/16/2024, 12:03 AMpython
import pandas as pd
import yaml
import os
# Read the CSV file
df = pd.read_csv('your_file.csv')
# Create a directory for context files if it doesn't exist
os.makedirs('context_files', exist_ok=True)
# Initialize the YAML structure
yaml_data = {'tests': []}
# Process each row in the DataFrame
for _, row in df.iterrows():
# Extract the id, query, and context
id = row['id']
query = row['query']
context = row['context']
# Create the context file
context_filename = f"{id:03d}_context.txt"
with open(os.path.join('context_files', context_filename), 'w') as f:
f.write(context)
# Create the test entry for YAML
test_entry = {
'vars': {
'query': query,
'context': f"file://{context_filename}"
},
'assert': [
{'type': 'context-relevance', 'threshold': 0.9},
{'type': 'context-faithfulness', 'threshold': 0.9},
{'type': 'answer-relevance', 'threshold': 0.9}
]
}
# Add the test entry to the YAML data
yaml_data['tests'].append(test_entry)
# Write the YAML file
with open('output.yaml', 'w') as f:
yaml.dump(yaml_data, f, default_flow_style=False)
print("Processing complete. Context files and YAML file have been created.")
Note 1: I haven't tested the code yet, but looks mostly ok to me.
Note 2: Iterrows is the slowest way to iterate over a large csv file, but you could improve the performance working with vectorial functions if needed.