@Vladi Stevanovic Hi ! I know that is a community-only chan. But I think this is closely related to this Prisma Cloud sunset. Can you help us for this transition?

something related with metrics-eu1.prisma.io

It would be great that this URL return a 200 (for every route)

I can set a

PROMETHEUS_PUSHGATEWAY_ENDPOINT

sys env and I can create a "black hole" but I think that I am not the only one with this need

Gentle up @Vladi Stevanovic 🙂

Can you double check if your deployment is fully set up as a custom server btw? https://v1.prisma.io/docs/1.34/faq/migrate-to-self-hosted-fq15/#custom-servers

I don't think we have any custom configuration and I can confirm that my deployment is fully set up as a custom server (we use prisma server in a docker container in our infrastructure since 2018). Here is our Ansible template, unchanged for years:

env:
      PRISMA_CONFIG: |
        managementApiSecret: "{{ prisma_management_api_secret }}"
        port: 4466
        databases:
          default:
            rawAccess: true
            connector: mysql
            migrations: true
            host: "{{ prisma_db_host }}"
            port: 3306
            user: "{{ prisma_db_user }}"
            password: "{{ prisma_db_pass }}"

I understand that you don't change anything but maybe this is an outage? You can try by yourself: http://metrics-eu1.prisma.io/ : any route for this domain returns a

503 Service Unavailable: Back-end server is at capacity

. Not a 4xx. I suspect this is not attended. I understand that I am the only one to report this to you. But I am maybe the only one with a monitor on the error logs of the prisma1 server. Because it still works as expected, maybe this error exists for others but they don't have noticed it 😅

How sure are you this has not been failing the same way since much longer? Was the logging in place all the time already and you are sure, before it just worked?

@janpio I am not sure that the metrics "worked" but I am 100% sure that no error was printed. We have some "legit" error every day for years.

@andrewicarlson I don't have a node environment in my container (no npx then)

(I was confused because the error is not related to schema deployment, but I understand this is a way to validate that we don't use prisma cloud 🙂 )

Thank you @Mike B. for this confirmation 🙏

@Mike B. is your

logConfiguration

that's using

awsfirelens

pointing to your own New Relic instance?

FWIW, inside our

"environment"

options, we previously had:

{
  "name": "LOG_LEVEL",
  "value": "INFO"
}

I removed that section today, redeployed, and same result.

Thanks for giving that a go – still digging in!

@Mike B. another attempt – based on line 13 here you might try adding a blank

PROMETHEUS_PUSHGATEWAY_ENDPOINT

environment variable (like

""

or

localhost

etc) I think it might throw a Metrics initialization error (hitting line 36)

@Mike B. Awesome thanks for trying that. We're moving in the right direction! I'm trying to avoid setting up an actual black hole to capture those requests.

@Mike B. I wonder if you push a non-string value if it'll hit that line 36. For example:

PROMETHEUS_PUSHGATEWAY_ENDPOINT: false

If you or @CHaBou can push out a minimally reproducible example I can test some of these theories before sending them out to you!