This message was deleted Permit #opal

Join Slack

This message was deleted.

# opal

Slackbot

12/15/2022, 11:43 AM

This message was deleted.

Or Weis

12/15/2022, 11:53 AM

I’m actually just in the middle of a PR to add more https://github.com/permitio/opal/pull/342

Sumesh Sukumaran

12/15/2022, 12:13 PM

Great 👍

Or Weis

12/15/2022, 1:27 PM

@Sumesh Sukumaran Can you provide an example of the webhook from Azure-Git so I can make sure the configuration will support that too? 🙂

Or Weis

12/15/2022, 3:55 PM

I merged the PR - you can already try it from master Once we ship a new version of OPAL it will be part of the released image. Note this new config to control how the web hook is parsed:

Copy code

OPAL_POLICY_REPO_WEBHOOK_PARAMS= {
           "secret_header_name": "X-Gitlab-Token",
           "secret_type": "token",
           "secret_parsing_regex": "(.*)",
           "event_header_name": "X-Gitlab-Event",
           "push_event_value": "Push Hook",
       }

This is the config for Gitlab - it would be great if you can share the config you end up using for Azure (or let us know if something is missing)

Sumesh Sukumaran

12/15/2022, 4:33 PM

Hi @Or Weis, Have a look at the config for Azure Git.

Or Weis

12/15/2022, 4:37 PM

Oh it seems pretty coustomizable, at least for the hedaers, you can actually configure it to look like whatever you want. How does the body of the webhook look?

Sumesh Sukumaran

12/15/2022, 5:31 PM

This the request body I got when I tested "Headers":{ "Authorization":"Basic ******** x-api-key":"******** Content-Type":application/json; charset=utf-8 }"Content":{ "subscriptionId":"00000000-0000-0000-0000-000000000000", "notificationId":48, "id":"03c164c2-8912-4d5e-8009-3707d5f83734", "eventType":"git.push", "publisherId":"tfs", "message":{ "text":"Jamal Hartnett pushed updates to Fabrikam-Fiber-Git:master.", "html":"Jamal Hartnett pushed updates to Fabrikam-Fiber-Git:master.", "markdown":"Jamal Hartnett pushed updates to `Fabrikam-Fiber-Git`:`master`." }, "detailedMessage":{ "text":"Jamal Hartnett pushed a commit to Fabrikam-Fiber-Git:master.\n - Fixed bug in web.config file 33b55f7c", "html":"Jamal Hartnett pushed a commit to <a href=\"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/\">Fabrikam-Fiber-Git</a>:<a href=\"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/#version=GBmaster\">master</a>.\n<ul>\n<li>Fixed bug in web.config file <a href=\"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/commit/33b55f7cb7e7e245323987634f960cf4a6e6bc74\">33b55f7c</a>\n</ul>", "markdown":"Jamal Hartnett pushed a commit to [Fabrikam-Fiber-Git](https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/):[master](https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/#version=GBmaster).\n* Fixed bug in web.config file [33b55f7c](https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/commit/33b55f7cb7e7e245323987634f960cf4a6e6bc74)" }, "resource":{ "commits":[ { "commitId":"33b55f7cb7e7e245323987634f960cf4a6e6bc74", "author":{ "name":"Jamal Hartnett", "email":"fabrikamfiber4@hotmail.com", "date":"2015-02-25T190100Z" }, "committer":{ "name":"Jamal Hartnett", "email":"fabrikamfiber4@hotmail.com", "date":"2015-02-25T190100Z" }, "comment":"Fixed bug in web.config file", "url":"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git/commit/33b55f7cb7e7e245323987634f960cf4a6e6bc74" } ], "refUpdates":[ { "name":"refs/heads/master", "oldObjectId":"aad331d8d3b131fa9ae03cf5e53965b51942618a", "newObjectId":"33b55f7cb7e7e245323987634f960cf4a6e6bc74" } ], "repository":{ "id":"278d5cd2-584d-4b63-824a-2ba458937249", "name":"Fabrikam-Fiber-Git", "url":"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249", "project":{ "id":"6ce954b1-ce1f-45d1-b94d-e6bf2464ba2c", "name":"Fabrikam-Fiber-Git", "url":"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/projects/6ce954b1-ce1f-45d1-b94d-e6bf2464ba2c", "state":"wellFormed", "visibility":"unchanged", "lastUpdateTime":"0001-01-01T000000" }, "defaultBranch":"refs/heads/master", "remoteUrl":"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_git/Fabrikam-Fiber-Git" }, "pushedBy":{ "displayName":"Jamal Hartnett", "id":"00067FFED5C7AF52@Live.com", "uniqueName":"fabrikamfiber4@hotmail.com" }, "pushId":14, "date":"2014-05-02T191713.3309587Z", "url":"https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/pushes/14" }, "resourceVersion":"1.0", "resourceContainers":{ "collection":{ "id":"c12d0eb8-e382-443b-9f9c-c52cba5014c2" }, "account":{ "id":"f844ec47-a9db-4511-8281-8b63f4eaf94e" }, "project":{ "id":"be9b3917-87e6-42a4-a549-2bc06a7a878f" } }, "createdDate":"2022-12-15T172823.1937259Z" }

Or Weis

12/15/2022, 5:46 PM

Mmmm… this won’t work. They nest the repository object under another key (resource), and they have the event type in the body of the webhook request, instead of in a header. We’ll need to add specific support for this. I can try and get to it next week; would that be okay for you, @Sumesh Sukumaran ? You’re welcome to do a PR of your own, of course, if you can’t wait. The needed changes as I see them: • Here - add parsing of “resource.repository” in addition to repository ◦ or better yet, add config of which paths to check • Here - add option to read event from body in a certain path instead of header • Here - Add specific test for Azure Git

Sumesh Sukumaran

12/15/2022, 6:10 PM

Yes @Or Weis that's fine. Thank you for looking at it. Actually, I don't have prior experience in python I am a .Net person. Still, let me check whether I can do something 🙂.

Or Weis

12/15/2022, 6:11 PM

💪🤘

Pentry Yurhadi

12/21/2022, 4:17 AM

i'm using gitlab and using gitlab ci/cd like this

Copy code

stages:
  - deploy 
deploy:
  stage: deploy
  before_script:
    - apk add --update curl && rm -rf /var/cache/apk/*
  script: 
    - |
      curl --location --request POST '<opal-server-url>/webhook' \
      --header 'x-hub-signature-256: sha256=9d08c9c0b4c2862b282ae23c43cdd3b14a1965fdcf55feb2207ae21cd21e0a93' \
      --header 'X-GitHub-Event: push' \
      --header 'Content-Type: application/json' \
      --data-raw '{
          "action" : "push",
          "repository": {
              "ssh_url" : "<gitlab-ssh-url>"
          }
      }'

maybe you can also try it in azure

Or Weis

12/21/2022, 7:09 AM

Oh wow @Pentry Yurhadi that's a cool solution. Thanks for sharing

Or Weis

12/21/2022, 7:12 AM

We did just add built-in support for Gitlab, maybe you'd still want to try it.

Pentry Yurhadi

12/21/2022, 7:27 AM

@Or Weis cool, thanks for the information

Or Weis

01/02/2023, 12:19 AM

@Sumesh Sukumaran , @Jack Geek I’ve finished a PR to add support for Azure-Git, and improved the docs for the feature Docs preview here: https://deploy-preview-351--opal-docs.netlify.app/tutorials/track_a_git_repo#working-with-different-git-services Hopefully @Ro'e Katz can review this tomorrow- and this will be part of the upcoming OPAL release (@Asaf Cohen)

Sumesh Sukumaran

01/05/2023, 9:51 AM

Thanks @Or Weis, I will try and configure with my Azure Git.

👍 1

Sumesh Sukumaran

01/06/2023, 6:28 AM

Hi all, I am trying to setup Rego policies in Azure git(private git). So how should I configure it? Is it possible to use the Personal Access token or should we go with SSH keys?

Or Weis

01/06/2023, 6:36 AM

SSH key :) It is the generic way of securely connecting to a Git repository

👍 1

Sumesh Sukumaran

01/31/2023, 9:59 AM

Hi @Or Weis, I was trying to configure Azure private git using keys generated by 'rsa' algorithm instead of default 'ed25519' algorithm. Please let us know whether it works with keys generated rsa algorithm(currentlt Azure git is supporting only rsa keys). And wanted to check if the latest opal docker images in docker hub has Azure git support related changes.

Or Weis

01/31/2023, 10:02 AM

Yeah as long as it's a valid SSH key there shouldn't be a problem

Or Weis

01/31/2023, 10:03 AM

https://docs.opal.ac/getting-started/running-opal/run-opal-server/security-parameters

Sumesh Sukumaran

01/31/2023, 10:05 AM

Thanks. And wanted to check if the latest opal docker images in docker hub has Azure git support related changes.

Or Weis

01/31/2023, 10:10 AM

I’m sorry to say - that not yet - we’ve had delays with a release here. I’ll ask @Asaf Cohen to let us know when’s the next release planned

Asaf Cohen

01/31/2023, 10:11 AM

Hey @Sumesh Sukumaran, We are waiting for one more PR by @Ori Shavit and will hopefully make a release later today.

👍 1

Sumesh Sukumaran

01/31/2023, 11:20 AM

Please let us know once you released

👍 1

Sumesh Sukumaran

02/06/2023, 3:08 PM

Hi folks, Do we need to add OPAL_POLICY_REPO_WEBHOOK_SECRET on Azure git? With that I wanted to know can I ignore the optional fields in the below settings?

Oded Bd

02/06/2023, 3:29 PM

Just add

x-api-key=YOUR_WEBHOOK_SECRET

to your HTTP headers As you can see in this docs https://docs.opal.ac/tutorials/track_a_git_repo#option-2-using-a-webhook under Azure-Git section

Sumesh Sukumaran

02/06/2023, 3:36 PM

Thank you Oded 👍

Sumesh Sukumaran

02/07/2023, 4:31 AM

Hi @Oded Bd, I have added the configuration as per the documentation. But I am getting unauthorized 401 error while sending the webhook. Attaching my logs and azure git settings.

azure webhook logs.txt

Or Weis

02/07/2023, 8:06 AM

Hi @Sumesh Sukumaran, Seems like the secrect check failed. We'd need more information here. What's the rest of the configuration you've provided to OPAL? Can you try the webhook on a dummy server of your own to see how the incoming request looks

Sumesh Sukumaran

02/07/2023, 9:11 AM

Yes I am sharing the OPAL settings, webhook request and event information here. Request Data

Method: POST

URI: https://1730-122-50-209-214.in.ngrok.io/webhook

HTTP Version: 1.1

Headers:

{

x-api-key: ********

Content-Type: application/json; charset=utf-8

}

Content:

{

"subscriptionId": "440a9037-46f6-43fb-9d8c-33dcfe3886b5",

"notificationId": 13,

"id": "2c0a54da-9ba2-4293-aa7c-d349f09b91d4",

"eventType": "git.push",

"publisherId": "tfs",

"message": {

"text": "Sumesh S pushed updates to OPAL-Policy-Repo:master\r\n(https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/#version=GBmaster)",

"html": "Sumesh S pushed updates to <a href=\"https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/\">OPAL-Policy-Repo</a>:<a href=\"https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/#version=GBmaster\">master</a>",

"markdown": "Sumesh S pushed updates to [OPAL-Policy-Repo](https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/):[master](https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/#version=GBmaster)"

},

"detailedMessage": {

"text": "Sumesh S pushed a commit to OPAL-Policy-Repo:master\r\n - adsa 982ef9ee (https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/commit/982ef9ee66b3349cd0e09b3b3a326bec8047aab8)",

"html": "Sumesh S pushed a commit to <a href=\"https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/\">OPAL-Policy-Repo</a>:<a href=\"https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/#version=GBmaster\">master</a>\r\n<ul>\r\n<li>adsa <a href=\"https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/commit/982ef9ee66b3349cd0e09b3b3a326bec8047aab8\">982ef9ee</a></li>\r\n</ul>",

"markdown": "Sumesh S pushed a commit to [OPAL-Policy-Repo](https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/):[master](https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/#version=GBmaster)\r\n* adsa [982ef9ee](https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo/commit/982ef9ee66b3349cd0e09b3b3a326bec8047aab8)"

},

"resource": {

"commits": [

{

"commitId": "982ef9ee66b3349cd0e09b3b3a326bec8047aab8",

"author": {

"name": "sumesh.s@alegeus.com",

"email": "sumesh.s@alegeus.com",

"date": "2023-02-07T041314Z"

},

"committer": {

"name": "sumesh.s@alegeus.com",

"email": "sumesh.s@alegeus.com",

"date": "2023-02-07T041314Z"

},

"comment": "adsa",

"url": "https://dev.azure.com/Alegeus-Technologies/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1/commits/982ef9ee66b3349cd0e09b3b3a326bec8047aab8"

}

],

"refUpdates": [

{

"name": "refs/heads/master",

"oldObjectId": "fa5a8e4fe82837576df8da012987a3104270a851",

"newObjectId": "982ef9ee66b3349cd0e09b3b3a326bec8047aab8"

}

],

"repository": {

"id": "7286c5d3-b7c1-4ca8-8928-c82930a30bb1",

"name": "OPAL-Policy-Repo",

"url": "https://dev.azure.com/Alegeus-Technologies/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1",

"project": {

"id": "d7388abd-50f6-4da7-8b36-409edceba8a9",

"name": "ArchitectureCouncil",

"url": "https://dev.azure.com/Alegeus-Technologies/_apis/projects/d7388abd-50f6-4da7-8b36-409edceba8a9",

"state": "wellFormed",

"visibility": "unchanged",

"lastUpdateTime": "0001-01-01T000000"

},

"defaultBranch": "refs/heads/master",

"remoteUrl": "https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo"

},

"pushedBy": {

"displayName": "Sumesh S",

"url": "https://spsprodcus1.vssps.visualstudio.com/Adbba024b-4985-41b1-86e1-fb7a27d5055f/_apis/Identities/bfd07cea-8c9e-60dc-801c-efb0f661f638",

"_links": {

"avatar": {

"href": "https://dev.azure.com/Alegeus-Technologies/_apis/GraphProfile/MemberAvatars/aad.YmZkMDdjZWEtOGM5ZS03MGRjLTgwMWMtZWZiMGY2NjFmNjM4"

}

},

"id": "bfd07cea-8c9e-60dc-801c-efb0f661f638",

"uniqueName": "sumesh.s@alegeus.com",

"imageUrl": "https://dev.azure.com/Alegeus-Technologies/_api/_common/identityImage?id=bfd07cea-8c9e-60dc-801c-efb0f661f638",

"descriptor": "aad.YmZkMDdjZWEtOGM5ZS03MGRjLTgwMWMtZWZiMGY2NjFmNjM4"

},

"pushId": 396757,

"date": "2023-02-07T041319.6262277Z",

"url": "https://dev.azure.com/Alegeus-Technologies/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1/pushes/396757",

"_links": {

"self": {

"href": "https://dev.azure.com/Alegeus-Technologies/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1/pushes/396757"

},

"repository": {

"href": "https://dev.azure.com/Alegeus-Technologies/d7388abd-50f6-4da7-8b36-409edceba8a9/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1"

},

"commits": {

"href": "https://dev.azure.com/Alegeus-Technologies/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1/pushes/396757/commits"

},

"pusher": {

"href": "https://spsprodcus1.vssps.visualstudio.com/Adbba024b-4985-41b1-86e1-fb7a27d5055f/_apis/Identities/bfd07cea-8c9e-60dc-801c-efb0f661f638"

},

"refs": {

"href": "https://dev.azure.com/Alegeus-Technologies/d7388abd-50f6-4da7-8b36-409edceba8a9/_apis/git/repositories/7286c5d3-b7c1-4ca8-8928-c82930a30bb1/refs/heads/master"

}

},

"resourceVersion": "1.0",

"resourceContainers": {

"collection": {

"id": "02e6d2e0-ac7c-4ce9-b5a6-3bfaea7cddfa",

"baseUrl": "https://dev.azure.com/Alegeus-Technologies/"

},

"account": {

"id": "dbba024b-4985-41b1-86e1-fb7a27d5055f",

"baseUrl": "https://dev.azure.com/Alegeus-Technologies/"

},

"project": {

"id": "d7388abd-50f6-4da7-8b36-409edceba8a9",

"baseUrl": "https://dev.azure.com/Alegeus-Technologies/"

}

},

"createdDate": "2023-02-07T041326.9652352Z"

}

event data.txt

Or Weis

02/07/2023, 9:52 AM

Hi @Sumesh Sukumaran I’m missing the

OPAL_POLICY_REPO_WEBHOOK_PARAMS

from you config screenshot It should be, something like this (as per the docs):

Copy code

{
    "secret_header_name": "x-api-key",
    "secret_type": "token",
    "secret_parsing_regex": "(.*)",
    "event_header_name": None,
    "event_request_key": "eventType",
    "push_event_value": "git.push",
}

Sumesh Sukumaran

02/07/2023, 11:00 AM

Hi @Or Weis, You mean to say about the value in the header right?

Or Weis

02/07/2023, 11:05 AM

No, I mean the config value for OPAL:

OPAL_POLICY_REPO_WEBHOOK_PARAMS

Or Weis

02/07/2023, 11:05 AM

It’s needed to tell OPAL to expect a webhook from Azure-Git, and not another Git service

Sumesh Sukumaran

02/07/2023, 1:09 PM

Yes, I have added it, got little confused with the documentation. But now my OPAL client is not getting updated with the webhook changes. I am attaching the log file also

new url mismatch logs.txt

Or Weis

02/07/2023, 1:25 PM

1. Hi @Sumesh Sukumaran - you put the header value instead of the header name in (“x-api-key”) 2. As you can see in the error- the repo that you configured for OPAL and the one being reported but the webhook do not match

👍 1

Or Weis

02/07/2023, 1:27 PM

the event_header_name needs to be null instead of an empty string

🤘 1

Sumesh Sukumaran

02/07/2023, 3:38 PM

Thanks I made the necessary changes. And regarding the error I am getting actually its the same repo only. I have given the SSH url on OPAL server but through webhook request its sending the https url.

Copy code

<mailto:git@ssh.dev.azure.com|git@ssh.dev.azure.com>:v3/Alegeus-Technologies/ArchitectureCouncil/OPAL-Policy-Repo

<https://dev.azure.com/Alegeus-Technologies/ArchitectureCouncil/_git/OPAL-Policy-Repo>

Or Weis

02/07/2023, 3:59 PM

Oh. not sure if that flow works. Let me check. We might need to add code here. @Ori Shavit - you’ve worked on something similar not to long ago - think we can just drop the schema ?

🙂 1

Ori Shavit

02/07/2023, 4:06 PM

For GitHub webhook handler, I made changes so look in all the provided URLs in webhook payload, maybe the same change is needed in azure git handlers, I'll take a look

Or Weis

02/07/2023, 4:06 PM

Thanks @Ori Shavit

Sumesh Sukumaran

02/18/2023, 3:00 PM

Hi @Or Weis, instead of configuring the webhook can we call it from a pipeline as API. I mean can we send a post request with json body.

generated (1).json

Or Weis

02/18/2023, 3:04 PM

Yes for sure. As long as the incoming http post request to the route (default

/webhook

) meets the configuration you set in the webhook config params it would work.

Sumesh Sukumaran

02/18/2023, 3:06 PM

Great 🤘

Open in Slack

Previous Next