https://www.runatlantis.io/ logo
Powered by
Title
a

Andrew Best

04/06/2023, 1:15 AM
Im doing a small poc of Atlantis on ACI with Azure DevOps. I am following the bouncing ball for installation. I am now trying to test the webhooks I created in AzDO per https://www.runatlantis.io/docs/configuring-webhooks.html#next-steps. AzDO is reporting a 401 status code though when it tries to fire the webhook into Atlantis. Before I dive into sharing configs and log outputs, can anyone suggest what the best troubleshooting path forward here is? Im just about to turn up the Atlantis debug level but figured I would see what the brains-trust here recommends.
Okay, so debug logs turned on. They didnt actually point me to the underlying issue thats causing the 
401
error. 
{"level":"debug","ts":"2023-04-06T01:21:07.715Z","caller":"server/middleware.go:45","msg":"POST /events – from 10.92.0.5:60147","json":{}}
{"level":"debug","ts":"2023-04-06T01:21:07.718Z","caller":"events/events_controller.go:134","msg":"handling AzureDevops post","json":{}}
{"level":"warn","ts":"2023-04-06T01:21:07.722Z","caller":"logging/simple_logger.go:161","msg":"ValidatePayload authentication failed","json":{},"stacktrace":"<http://github.com/runatlantis/atlantis/server/logging.(*StructuredLogger).Log|github.com/runatlantis/atlantis/server/logging.(*StructuredLogger).Log>\n\tgithub.com/runatlantis/atlantis/server/logging/simple_logger.go:161\ngithub.com/runatlantis/atlantis/server/controllers/events.(*VCSEventsController).respond\n\tgithub.com/runatlantis/atlantis/server/controllers/events/events_controller.go:716\ngithub.com/runatlantis/atlantis/server/controllers/events.(*VCSEventsController).handleAzureDevopsPost\n\tgithub.com/runatlantis/atlantis/server/controllers/events/events_controller.go:263\ngithub.com/runatlantis/atlantis/server/controllers/events.(*VCSEventsController).Post\n\tgithub.com/runatlantis/atlantis/server/controllers/events/events_controller.go:135\nnet/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:2109\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\tgithub.com/gorilla/mux@v1.8.0/mux.go:210\ngithub.com/urfave/negroni/v3.Wrap.func1\n\tgithub.com/urfave/negroni/v3@v3.0.0/negroni.go:59\ngithub.com/urfave/negroni/v3.HandlerFunc.ServeHTTP\n\tgithub.com/urfave/negroni/v3@v3.0.0/negroni.go:33\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\tgithub.com/urfave/negroni/v3@v3.0.0/negroni.go:51\ngithub.com/runatlantis/atlantis/server.(*RequestLogger).ServeHTTP\n\tgithub.com/runatlantis/atlantis/server/middleware.go:70\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\tgithub.com/urfave/negroni/v3@v3.0.0/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Recovery).ServeHTTP\n\tgithub.com/urfave/negroni/v3@v3.0.0/recovery.go:210\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\tgithub.com/urfave/negroni/v3@v3.0.0/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Negroni).ServeHTTP\n\tgithub.com/urfave/negroni/v3@v3.0.0/negroni.go:111\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2947\nnet/http.(*conn).serve\n\tnet/http/server.go:1991"}
{"level":"debug","ts":"2023-04-06T01:21:07.722Z","caller":"server/middleware.go:72","msg":"POST /events – respond HTTP 401","json":{}}
At least from what I can see in the above anyway.
FYI, im using Atlantis v0.22.3 because of https://github.com/runatlantis/atlantis/issues/3291
p

PePe Amengual

04/06/2023, 3:47 AM
you are using the test button on ADO to test the webhook?
a

Andrew Best

04/06/2023, 3:47 AM
Negative
p

PePe Amengual

04/06/2023, 3:47 AM
so you set it all up and you got that error, correct?
a

Andrew Best

04/06/2023, 3:48 AM
Correct.
p

PePe Amengual

04/06/2023, 3:48 AM
the Test webhook thing is just terrible
a

Andrew Best

04/06/2023, 3:48 AM
the Test webhook Azure DevOps thing is just terrible
ftfy
🙂 1
Its not clear to me which part of the Atlantis config is causing it to reject the auth request with the AzDO webhook. Sadly the debug logs doesnt shed any light on which particular config in Atlantis is causing the issue.
p

PePe Amengual

04/06/2023, 6:00 AM
I will make sure the secret for the webhook is correct
I do not know how you are feeding it but it could contain special chars and might noe being parsed correctly etc
a

Andrew Best

04/06/2023, 6:09 AM
Im wondering how up to date the Atlantis docs are for use with AzDO.
Because this is a PoC im not putting too much effort into automating the setup of Atlantis in Azure. Ive ended up going with the shell script at https://github.com/jplane/atlantis-on-aci/blob/master/atlantis-on-aci.sh
And now im seeing a few issues in that script more broadly. Lemme see if I can hack at my script a little more.
That script is >12 months old and is using server flags that dont appear in the docs.
eg 
# create and run the Atlantis container
az container create \
    --name $ATLANTIS_CONTAINER_GROUP_NAME \
    --resource-group $ATLANTIS_RG_NAME \
    --location $ATLANTIS_LOCATION \
    --assign-identity \
    --scope /subscriptions/$SUB_ID \
    --azure-file-volume-account-name $ATLANTIS_STORAGE_NAME \
    --azure-file-volume-account-key $STORAGE_KEY \
    --azure-file-volume-share-name "atlantis-cert-share" \
    --azure-file-volume-mount-path /mnt/atlantis-certs \
    --image "<http://ghcr.io/runatlantis/atlantis:v0.22.3|ghcr.io/runatlantis/atlantis:v0.22.3>" \
    --os-type Linux \
    --restart-policy OnFailure \
    --cpu 1 \
    --memory 2 \
    --ports 4141 \
    --dns-name-label $ATLANTIS_CONTAINER_DNS_NAME \
    --command-line "atlantis server \
        --azuredevops-user=$AZUREDEVOPS_USER \
        --azuredevops-webhook-user=$AZUREDEVOPS_WEBHOOK_USER \
        --repo-allowlist=$REPO_ALLOWLIST \
        --ssl-cert-file=/mnt/atlantis-certs/atlantis.crt \
        --ssl-key-file=/mnt/atlantis-certs/atlantis.key \
        --web-basic-auth \
        --web-username={REDACTED}" \
    --environment-variables \
        ARM_USE_MSI=true \
        ARM_SUBSCRIPTION_ID=$SUB_ID \
    --secure-environment-variables \
        ARM_ACCESS_KEY=$STORAGE_KEY \
        ATLANTIS_AZUREDEVOPS_TOKEN=$AZUREDEVOPS_TOKEN \
        ATLANTIS_AZUREDEVOPS_WEBHOOK_PASSWORD=$AZUREDEVOPS_WEBHOOK_SECRET \
        ATLANTIS_WEB_PASSWORD=$ATLANTIS_WEB_PASSWORD
The setup docs say to create the webhook secret but then when configuring the AzDO webhook triggers, it says to use the basic-auth values.
wait.....
So, in the AzDO webhook setup, you specify the 
Basic authentication username
and 
Basic authentication password
. I had entered these as the values I had configured for the 
--web-username
and 
ATLANTIS_WEB_PASSWORD
. I just configured the AzDO webhooks to use the 
--azuredevops-webhook-user
and 
ATLANTIS_AZUREDEVOPS_WEBHOOK_PASSWORD
values. I commented on a PR and AzDO reports it as successful.
What that means is, the Atlantis setup doco for AzDO needs to make it clearer that when you are configured the Webhooks in AzDO, you need to enter your 
webhook-user
and 
webhook-password
into the AzDO fields labelled 
Basic authentication username
and 
Basic authentication password
.
The page in question here https://www.runatlantis.io/docs/configuring-webhooks.html#azure-devops
The docs currently say:
No mention there of what I found above.
p

PePe Amengual

04/06/2023, 3:00 PM
please feel free to create a PR, it would be greatly appreciated
#communityJoin Slack
Powered by