https://nf-co.re logo
Join Slack
Powered by
The <multiqc docs> state: &gt; • Anthropic model n...
# multiqc
n
The multiqc docs state:
• Anthropic model names must begin with 
claude
◦ Default: 
claude-3-5-sonnet-latest
.
◦ Tested with Sonnet 3.5 and Haiku 3.5. See the Anthropic docs.
• OpenAI model names must being with 
gpt
◦ Default: 
gpt-4o
.
◦ Tested with GPT-4o and GPT-4o-mini. See the OpenAI docs.
Anyone know if there are any issues with newer models (e.g, claude 3.7, gpt-4.1, or o4-mini)?
p
I would guess / hope that they should work..
Have you tried?
eg, 
claude-3-7-sonnet-latest
?
n
No, but I'm planning on trying later today: all of the models that I list above & gpt-4.1-mini
👍🏻 1
https://docs.seqera.io/multiqc/ai#configuring-within-nextflow states:
Copy code
env {
   MULTIQC_AI_SUMMARY_FULL = 1              // Enable long summaries during report generation
   MULTIQC_AI_PROVIDER = "openai"           // Select OpenAI as provider
   OPENAI_API_KEY = secrets.OPENAI_API_KEY  // Access key for OpenAI
}
Could this be a security risk since the key is exposed for all processes in the entire pipeline via the 
env
scope? The key could appear in logs if environment variables are dumped (e.g., debug mode of nextflow).
Also, I was hoping to dynamically set the model based on whether the user provides an openai or anthropic API key:
Copy code
process MULTIQC_basic{
    secret "OPENAI_API_KEY"
    secret "ANTHROPIC_API_KEY"
    
    input:
    path multiqc_files, stageAs: "?/*"

    output:
    path "multiqc_output/*", emit: all
    
    script:
    """
    exit 1
    # set the provider and enable AI summaries if API key is set
    if [ -n "${ANTHROPIC_API_KEY}" ]; then
        # if ANTHROPIC_API_KEY is set, use Anthropic as provider
        MULTIQC_AI_PROVIDER="anthropic"
    elif [ -n "${OPENAI_API_KEY}" ]; then
        # if OPENAI_API_KEY is set, use OpenAI as provider
        MULTIQC_AI_PROVIDER="openai"
    fi

    # if OPENAI_API_KEY or ANTHROPIC_API_KEY is set, enable long summaries
    if [ -n "${OPENAI_API_KEY}" ] || [ -n "${ANTHROPIC_API_KEY}" ]; then
        MULTIQC_AI_SUMMARY_FULL=1
    fi
    
    multiqc
    [...]
}
However, it appears that secrets MUST be set: 
Required secret is missing: 'ANTHROPIC_API_KEY'
Any suggestions on how to get around this?
I settled on the following:
Copy code
process MULTIQC_basic{
    secret "AI_API_KEY"
    
    input:
    path multiqc_files, stageAs: "?/*"
    path multiqc_config
    path multiqc_summary

    output:
    path "multiqc_output/*", emit: all
    
    script:
    """
    AI_SETTINGS=""
    if [[ "${params.ai_model}" =~ ^(o[1-9]-.*|gpt-.*) ]]; then
        # if model starts with "o[3-9]-" or "gpt-" then use OpenAI
        AI_SETTINGS="--ai-model ${params.ai_model} --ai-summary --ai-provider openai"
        export OPENAI_API_KEY="\${AI_API_KEY}"
    elif [[ "${params.ai_model}" =~ ^(claude-.*) ]]; then
        # if model starts with "claude-" then use Anthropic
        AI_SETTINGS="--ai-model ${params.ai_model} --ai-summary --ai-provider anthropic"
        export ANTHROPIC_API_KEY="\${AI_API_KEY}"
    fi
    
    multiqc \${AI_SETTINGS} [...]
👍🏻 1
@Phil Ewels It appears that OpenAI reasoning models do not work with multiqc 1.28:
Copy code
ai | Failed to get a response from OpenAI (attempt 1/1). Error: 400 Client Error: Bad Request for url: <https://api.openai.com/v1/chat/completions>. Retrying in 1s...

                ai | Failed to get a response from OpenAI. Error: 400 Client Error: Bad Request for url: <https://api.openai.com/v1/chat/completions>
                ai | Failed to interpret report with OpenAI: 400 Client Error: Bad Request for url: <https://api.openai.com/v1/chat/completions>
Also, one cannot currently set the reasoning effort
Back to:
Could this be a security risk since the key is exposed for all processes in the entire pipeline via the 
env
scope?
When using:
Copy code
env {
   MULTIQC_AI_SUMMARY_FULL = 1              // Enable long summaries during report generation
   MULTIQC_AI_PROVIDER = "openai"           // Select OpenAI as provider
   OPENAI_API_KEY = secrets.OPENAI_API_KEY  // Access key for OpenAI
}
as stated in the docs, every 
.command.run
will include the unencrypted secret. This is a big security risk.
p
I'm not massively surprised about the reasoning models. They didn't exist when we built this. Feel free to put in a GitHub issue about it.
Need to think more about secrets. But it's a fairly well established thing to need to pass API keys, eh. #CGFUX04HZ does it with some of its tools. Could check how they're doing it.
n
But it's a fairly well established thing to need to pass API keys
I though that the point of 
nextflow secrets
was to avoid the security issues of just using env variables via the 
env
scope. Using 
env { OPENAI_API_KEY = secrets.OPENAI_API_KEY }
just seems to bypass the use of secrets, except for the fact that you are not hardcoding the secret directly into the codebase.
p
I have a feeling that secrets are screened out of log files and the like? But honestly, Nextflow secrets are one of the parts of Nextflow that I know least about
if you have a suggestion for a better way to pass the API key then I'd be happy to update the docs 👍🏻
I'll ask around a bit internally too
n
Secrets are definitely a challenge with Nextflow, especially since they are not supported for GCP and AWS. The fact that the API key, when used as an env variable in Nextflow, will be in all 
.command.run
scripts for the run is definitely a problem for our organization. We don't want unauthorized individuals racking up a large OpenAI/Anthropic API bill before we notice.
a
Using 
env { OPENAI_API_KEY = secrets.OPENAI_API_KEY }
just seems to bypass the use of secrets
Correct, but secrets are only really supported with local executors so this is a clunky workaround sadpanda
You could also do something like this in your script:
Copy code
export OPENAI_API_KEY=$(aws secretsmanager get-secret-value --secret-id OpenAiApiKey)
However, it appears that secrets MUST be set: 
Required secret is missing: 'ANTHROPIC_API_KEY'
This is why we ended up with this solution for Sentieon
you can set a process level secret in config via this:
Copy code
process.secret = ["OPENAI_API_KEY"]
Copy code
process {
    withName: MULTIQC { 
        secret = ["OPENAI_API_KEY"]
    }
}
☝️ that will give you the most specific secret solution
p
The downside of this is that the process script needs editing though, right? I liked the environment var approach as it was a simple config switch and it'd magically work for any pipeline.
a
you can use my config block up there as a config item!
errr dun do words good, let me rephrase
That block can go in your nextflow.config
`main.nf`:
Copy code
process MY_SUPER_SECRET_KEY {

    debug true

    output:
        stdout

    script:
    """
    echo \$MY_SECRET
    """
}

workflow {
    MY_SUPER_SECRET_KEY()
}
`nextflow.config`:
Copy code
process {
    withName: MY_SUPER_SECRET_KEY { 
        secret = ["MY_SECRET"]
    }
}
console:
Copy code
N E X T F L O W   ~  version 25.04.1

Launching `./main.nf` [shrivelled_joliot] DSL2 - revision: 4599e938a3

executor >  local (1)
[ee/1753e5] process > MY_SUPER_SECRET_KEY [100%] 1 of 1 ✔
oh_no_you_revealed_the_secret
n
That process MUST have the secret set, correct? So, you'd need to have 2 very similar processes: one that includes the secret and one for if the user does not provide the secret
Correct, but secrets are only really supported with local executors so this is a clunky workaround
Yeah. It's a big limitation of using Nextflow
a
That process MUST have the secret set, correct? So, you'd need to have 2 very similar processes: one that includes the secret and one for if the user does not provide the secret
or use the 
def
solution we use for Sarek. Or don’t apply that configuration above.
Open in Slack
PreviousNext
Powered by