Hello we re seeing regular gateway timeouts when trying to u Permit #permit-saas

Hello, we're seeing regular gateway timeouts when ...

Lewis Youl

08/27/2025, 11:49 AM

Hello, we're seeing regular gateway timeouts when trying to use the condition sets API, as well as create condition sets via the UI. Our implementation uses condition sets extensively and we can't afford for this endpoint to regularly time out. Is this on your radar to fix? Or can you recommend a work around? Please see the attached for an example of it failing via your UI. This is the body that was sent in the request:

Copy code

{
  "key": "megatest",
  "type": "resourceset",
  "resource_id": "document",
  "name": "megatest",
  "description": "megatest",
  "conditions": {
    "allOf": [
      {
        "allOf": [
          {
            "resource.documentTypeId": {
              "equals": "roflcopter"
            }
          },
          {
            "resource.owner": {
              "equals": "123456"
            }
          }
        ]
      }
    ]
  },
  "parent_id": null
}

Ask Inkeep

08/27/2025, 11:50 AM

Hi @Lewis Youl, I'm an AI assistant for Permit.io. I'll analyze available sources and get back to you shortly.

Ask Inkeep

08/27/2025, 11:50 AM

Feel free to tag me with additional questions. I also appreciate feedback, it helps me improve. 👍 button 👎 button

Or Weis

08/27/2025, 12:14 PM

Hi @Lewis Youl - thanks for reaching out. Few follow up questions? • How often would you say these happen? • Are you operating behind a gateway / VPN / proxy ? • Can you share logs / timestamps of these occurrences ? • Please share your workspace / Project / Env names

Lewis Youl

08/27/2025, 12:21 PM

• They happen at least 50% of the time when trying to create resource sets via the api • No I'm not. • Here are two that just happened:

Copy code

{"level":50,"time":"2025-08-27T12:16:32.146Z","label":"<http://Permit.io|Permit.io>","msg":"Got error status code: 504, err: \"<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body>\\r\\n<center><h1>504 Gateway Time-out</h1></center>\\r\\n</body>\\r\\n</html>\\r\\n\""}
Failed to create condition set document-08a7b643-272f-460e-89fb-dba8b8848c48-c8217bd9-ed07-4898-b455-f9c9eb615977-resource-set: 3 | import { BASE_PATH } from '../openapi/base';
4 | import { API_ACCESS_LEVELS, ApiContextLevel, ApiKeyLevel, PermitContextError } from './context';
5 | export class PermitApiError extends Error {
6 |     originalError;
7 |     constructor(message, originalError) {
8 |         super(message);
            ^
Error: 
 originalError: AxiosError {
  originalLine: 6,
  originalColumn: 26,
  stack: "Error\n    at settle (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/axios/lib/core/settle.js:19:16)\n    at handleStreamEnd (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/axios/lib/adapters/http.js:599:11)\n    at emit (node:events:84:22)\n    at endReadableNT (internal:streams/readable:862:50)\n    at processTicksAndRejections (native:7:39)\n    at <anonymous> (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/axios/lib/core/Axios.js:45:41)\n    at processTicksAndRejections (native:7:39)",
  message: "Request failed with status code 504",
  name: "AxiosError",
  code: "ERR_BAD_RESPONSE",
  config: [Object ...],
  request: [Object ...],
  response: [Object ...],
  status: 504,
  toJSON: [Function: toJSON],
  isAxiosError: true,
  toString: [Function: toString],
},

      at new PermitApiError (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/permitio/build/module/api/base.mjs:8:9)
      at handleApiError (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/permitio/build/module/api/base.mjs:126:19)
      at <anonymous> (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/permitio/build/module/api/condition-sets.mjs:105:18)

undefined

---

Copy code

{"level":50,"time":"2025-08-27T12:15:40.322Z","label":"<http://Permit.io|Permit.io>","msg":"Got error status code: 504, err: \"<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body>\\r\\n<center><h1>504 Gateway Time-out</h1></center>\\r\\n</body>\\r\\n</html>\\r\\n\""}
Failed to create condition set document-08a7b643-272f-460e-89fb-dba8b8848c48-c8217bd9-ed07-4898-b455-f9c9eb615977-resource-set: 3 | import { BASE_PATH } from '../openapi/base';
4 | import { API_ACCESS_LEVELS, ApiContextLevel, ApiKeyLevel, PermitContextError } from './context';
5 | export class PermitApiError extends Error {
6 |     originalError;
7 |     constructor(message, originalError) {
8 |         super(message);
            ^
Error: 
 originalError: AxiosError {
  originalLine: 6,
  originalColumn: 26,
  stack: "Error\n    at settle (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/axios/lib/core/settle.js:19:16)\n    at handleStreamEnd (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/axios/lib/adapters/http.js:599:11)\n    at emit (node:events:84:22)\n    at endReadableNT (internal:streams/readable:862:50)\n    at processTicksAndRejections (native:7:39)\n    at <anonymous> (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/axios/lib/core/Axios.js:45:41)\n    at processTicksAndRejections (native:7:39)",
  message: "Request failed with status code 504",
  name: "AxiosError",
  code: "ERR_BAD_RESPONSE",
  config: [Object ...],
  request: [Object ...],
  response: [Object ...],
  status: 504,
  toJSON: [Function: toJSON],
  isAxiosError: true,
  toString: [Function: toString],
},

      at new PermitApiError (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/permitio/build/module/api/base.mjs:8:9)
      at handleApiError (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/permitio/build/module/api/base.mjs:126:19)
      at <anonymous> (/Users/lewisyoul/Repositories/TrustFlight/1-tf/services/user-upstream/node_modules/permitio/build/module/api/condition-sets.mjs:105:18)

There's also the one that I shared the image of that happened via your UI which happened at

Wed, 27 Aug 2025 11:41:43 GMT

Lewis Youl

08/27/2025, 12:22 PM

workspace = alex-trustflight-sandbox project = sandbox env = lewisdevelopment

Or Weis

08/27/2025, 12:28 PM

Thanks, let us review these and circle back to you.

Lewis Youl

08/27/2025, 12:29 PM

For further context, we have also found the check to find out whether a condition set already exists sometimes returns false, even when it does exist, as such we need to rely on the 409 that comes back from the create endpoint.

Lewis Youl

08/27/2025, 12:30 PM

Thanks

Lewis Youl

08/27/2025, 2:16 PM

Another instance of this happening, this time on our hosted environment:

Copy code

│ {"level":50,"time":"2025-08-27T14:11:16.279Z","label":"<http://Permit.io|Permit.io>","msg":"Got error status code: 504, err: \"<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body>\\r\\n<center>< │
│ Failed to remove condition rule for document:read: 3 | import { BASE_PATH } from '../openapi/base';                                                                                                 │
│ 4 | import { API_ACCESS_LEVELS, ApiContextLevel, ApiKeyLevel, PermitContextError } from './context';                                                                                                │
│ 5 | export class PermitApiError extends Error {                                                                                                                                                     │
│ 6 |     originalError;                                                                                                                                                                              │
│ 7 |     constructor(message, originalError) {                                                                                                                                                       │
│ 8 |         super(message);                                                                                                                                                                         │
│             ^                                                                                                                                                                                       │
│ Error:                                                                                                                                                                                              │
│  originalError: AxiosError {                                                                                                                                                                        │
│   originalLine: 6,                                                                                                                                                                                  │
│   originalColumn: 26,                                                                                                                                                                               │
│   stack: "Error\n    at settle (/app/node_modules/axios/lib/core/settle.js:19:16)\n    at handleStreamEnd (/app/node_modules/axios/lib/adapters/http.js:599:11)\n    at emit (node:events:86:22)\n  │
│   message: "Request failed with status code 504",                                                                                                                                                   │
│   name: "AxiosError",                                                                                                                                                                               │
│   code: "ERR_BAD_RESPONSE",                                                                                                                                                                         │
│   config: [Object ...],                                                                                                                                                                             │
│   request: [Object ...],                                                                                                                                                                            │
│   response: [Object ...],                                                                                                                                                                           │
│   status: 504,                                                                                                                                                                                      │
│   toJSON: [Function: toJSON],                                                                                                                                                                       │
│   isAxiosError: true,                                                                                                                                                                               │
│   toString: [Function: toString],                                                                                                                                                                   │
│ },                                                                                                                                                                                                  │
│                                                                                                                                                                                                     │
│       at new PermitApiError (/app/node_modules/permitio/build/module/api/base.mjs:8:9)                                                                                                              │
│       at handleApiError (/app/node_modules/permitio/build/module/api/base.mjs:126:19)                                                                                                               │
│       at <anonymous> (/app/node_modules/permitio/build/module/api/condition-set-rules.mjs:81:18)                                                                                                    │
│                                                                                                                                                                                                    
│ undefined

Lewis Youl

08/27/2025, 2:17 PM

workspace = alex-trustflight-sandbox project = default env = dev

Or Weis

08/27/2025, 2:17 PM

@Omer Zuarets is on the case

👍 1

Lewis Youl

08/27/2025, 2:18 PM

This was for trying to use

this.permit.api.conditionSetRules.delete

. So it's a different endpoint but related to condition sets again.

Lewis Youl

08/27/2025, 2:21 PM

We're also seeing your UI time out when trying to remove condition set rules. A colleague (Yilmaz)is recording a video at the moment.

Yilmaz Alizadeh

08/27/2025, 2:21 PM

This is also happening in UI

2025-08-27 15-19-56.mkv

Or Weis

08/27/2025, 2:22 PM

Thank you, hopefully this information would help us to pinpoint the underlying issue quickly

Lewis Youl

08/27/2025, 2:36 PM

Thank you for the quick response. It would be great if this could be resolved as it is hamstringing us rather at the moment.

Or Weis

08/27/2025, 2:40 PM

I’m confident it would be resolved (my intuition is this is some underlying DB issue), but I’ll let Omer do his work/magic 😉

🤞 1

Alon Boshi

08/27/2025, 4:11 PM

Hey @Lewis Youl, I went through this with @Omer Zuarets, and we found the issue. It’s being reviewed and tested now, and we’re hoping it’ll be in production by tomorrow or early next week. Really appreciate all the details you shared—they were super helpful! :)

🙌 1

Lewis Youl

08/27/2025, 4:15 PM

That's great news, thank you for the update and for looking into it!

Lewis Youl

08/27/2025, 4:15 PM

Will you let us know when it is live?

Lewis Youl

08/27/2025, 4:15 PM

(Or is there some other way we can find out?)

Alon Boshi

08/27/2025, 4:43 PM

We will let you know when it's in production, And you can use the

/permit-ticket-status

with the ticket PER-12695 to track the status also 🙏🏼

👍 1

Lewis Youl

08/29/2025, 8:54 AM

Is there an an update on when this will be live? The ticket status doesn't seem to have moved in a couple of days and as such our implementation cannot function.

Or Weis

08/29/2025, 8:58 AM

Likely early next week. It's in the acceptance (testing and review) phase, we take testing with extra care here.

Lewis Youl

08/29/2025, 9:10 AM

OK thanks. To help with your testing, the endpoint that is consistently failing (504) is the one to remove a condition set rule. In our system specifically, it is failing when the resource set is for the

document

resource. We are able to remove condition set rules for other resources such as

document_type

. The one to create resource sets fails intermittently for the same reason.

Omer Zuarets

08/29/2025, 9:21 AM

Hey @Lewis Youl The endpoints you attached until now were the create endpoint if I understood correctly which is what we fixed aim to release on Sunday. To make sure we look at the right one regarding the delete, are you talking on a delete conditionset or delete set rules ?

Lewis Youl

08/29/2025, 9:29 AM

Deleting a condition set rule.

Lewis Youl

08/29/2025, 9:29 AM

I'll get you some examples.

Omer Zuarets

08/29/2025, 9:30 AM

Thanks, we’ll aim to release this also on Sunday

Lewis Youl

08/29/2025, 10:51 AM

Copy code

DELETE /v2/facts/9aa8bcdc454049cd8c1459f6d1215b4f/890e358221b34dee9ac6fb6103b762f8/set_rules - 504 Gateway Timeout

body:

{ 
  "user_set": "c91cddb1-e571-46e5-821d-056b6e0100f4-user-set",  
  "resource_set": "document-50c415ef-5713-431a-96ac-2a1c7ce128d7-c8217bd9-ed07-4898-b455-f9c9eb615977-resource-set",
  "permission":"document:update"
}

workspace: alex-trustflight-sandbox
project: sandbox
env: lewisdevelopment

I've attached an image for an identical request for

document_type

that is made just before the one for

document

. The

document_type

one always succeeds so I can see it in the api logs. The one for

document

always fails and I can't see it in the logs. This happens in different environments too, not just my one. > The endpoints you attached until now were the create endpoint I did mention this endpoint earlier in the thread too https://permit-io.slack.com/archives/C02RTML2WNN/p1756304306634849?thread_ts=1756295397.608009&cid=C02RTML2WNN

Alon Boshi

08/31/2025, 4:15 PM

Hey @Lewis Youl, The fixes for creating and updating condition sets are now live in production. The fix for deleting condition set rules is in progress and should be deployed in the next few days. Thanks again for flagging this — we’ll update you once it’s live!

Lewis Youl

09/01/2025, 8:18 AM

OK thanks for the update. We really need these resolved as it essentially renders permit unusable right now for us. Users can't toggle their permissions.

Omer Zuarets

09/01/2025, 8:27 AM

We let you know as soon as it’s fixed, the delete is the only one left, right ?

Lewis Youl

09/01/2025, 8:31 AM

As well as this one I'm also talking with you on https://permit-io.slack.com/archives/C02RTML2WNN/p1756487273851679.

Omer Zuarets

09/01/2025, 8:33 AM

• Delete set_rules • PATCH resource instance Anything else ? We’ll also try to go over other APIs around that to see if we see any other performance lacks there but we’ll prioritize fixing and deploying those two before others

Lewis Youl

09/01/2025, 8:36 AM

Yes those two. We're manually testing our integration heavily at the moment so I will flag anything else we spot as soon as I can.

Lewis Youl

09/01/2025, 2:15 PM

What is your confidence level in having the "Delete set_rules" issue resolved in the next couple of days?

Alon Boshi

09/01/2025, 2:16 PM

Hey @Lewis Youl , it’s now on reviewing and testing. I believe tomorrow it’s gonna be live in production

Lewis Youl

09/01/2025, 2:22 PM

Excellent, thank you for the prompt response.

Lewis Youl

09/02/2025, 12:52 PM

Is this live?

Alon Boshi

09/02/2025, 12:55 PM

It will happen in the coming hours, I’ll keep you updated.

Lewis Youl

09/02/2025, 12:56 PM

Thank you

Omer Zuarets

09/02/2025, 7:04 PM

Hey @Lewis Youl the fix for those two APIs is now being deployed to production. You should be able to experience them shortly

Yilmaz Alizadeh

09/03/2025, 3:58 PM

@Omer Zuarets I tested them this morning and they work as expected. thanks.

💜 2

Lewis Youl

09/03/2025, 4:16 PM

Indeed, thank you!

💜 1

Open in Slack

Previous Next