https://discord.cloudflare.com logo
Join Discord
Powered by
# logs-engine
  • crossposting here https discord com
    c

    cole | data

    03/13/2023, 9:07 PM
    What are you specifying as
    dataset
    value in the create job payload? Non-ENT plans only have access to
    workers_trace_events
    .
    a
    b
    • 3
    • 8
  • Anyone knows how to parse log files
    c

    cole | data

    04/25/2023, 4:06 PM
    Depends on your definition of parse. In general, the logs are newline delimited json. The entire file is then compressed using gzip. Some JSON libraries support NDJSON and some don't so you'll have to first split on newlines before parsing the json.
    s
    m
    • 3
    • 16
  • Logs Engine Client
    c

    cole | data

    04/25/2023, 4:07 PM
    Hey @Wallacy - nothing officially supported by us outside of the REST API. Happy to see any community built solutions!
  • Logpush integrations
    c

    cole | data

    04/27/2023, 9:30 PM
    Hey there, we recommend you use some form of authentication provided in HTTP headers, unguessable URL or a combination of both. For example, datadog uses HTTP authorization headers. 1. All possible logpush IP's are listed here https://www.cloudflare.com/ips/. It's best to add some form of authentication to your endpoint. 2. mTLS for logpush is not currently supported. 3. You could use Cloudflare Access and provide a service token in your destinationConf. https://developers.cloudflare.com/cloudflare-one/identity/service-tokens/#connect-your-service-to-access
    "destinationConf": "https://your.endpoint.com/ingest?header_cf-access-client-id=123&header_cf-access-client-secret=secret"
    b
    • 2
    • 1
  • m

    mz

    05/10/2023, 11:27 PM
    Hi: Can object expiry lifecycle rules (announced today) be applied to R2 bucket backing worker trace events (logs)?
  • u

    Unsmart | Tech debt

    05/10/2023, 11:28 PM
    it can be applied to any R2 bucket
  • m

    mz

    05/10/2023, 11:38 PM
    Thanks. Just to confirm: Without any consequences to the way workers trace events works, correct?
  • u

    Unsmart | Tech debt

    05/10/2023, 11:40 PM
    Wont have any affect on that its separate processes
  • h

    haltbill

    05/14/2023, 9:45 AM
    I'm getting this while adding
    new relic
    We cannot access your storage provider. Please check that you have granted Cloudflare access and provided the correct configuration information.
  • h

    haltbill

    05/14/2023, 9:46 AM
    https://log-api.newrelic.com/log/v1?Api-Key={NR_LICENSE_KEY}&format=cloudflare
  • h

    haltbill

    05/14/2023, 9:49 AM
    I'm using this https://developers.cloudflare.com/logs/get-started/enable-destinations/new-relic/
  • c

    cole | data

    05/15/2023, 4:57 PM
    Please contact Cloudflare support if you haven’t already.
  • o

    omrilotan

    05/18/2023, 11:02 AM
    Something I can't understand. Is anyone else getting this? I get a few of these each day, where the outcome is "exception", but there are no exceptions and the response code is 200. Here's an example. I have no idea what to make of this.
    Copy code
    {
      "Outcome": "exception",
      "ScriptName": "gateway-main",
      "ScriptTags": [],
      "Exceptions": [],
      "Event": {
        "RayID": "7c90bb00fb5ee338",
        "Request": {
          "Method": "GET",
          "URL": "<REDACTED>"
        },
        "Response": {
          "Status": 200
        }
      },
      "Logs": [],
      "EventTimestampMs": 1684377574619
    }
  • c

    cole | data

    05/18/2023, 3:49 PM
    You likely have a promise resolving after a response is returned that is throwing
  • o

    omrilotan

    05/18/2023, 3:56 PM
    Would that resolve in an exception without the actual exception? That's strange behaviour. Short of checking my code has no hanging promises (they all wait for resolutions or are forwarded to ctx.wailUntil) how can I check what was the exception? Would there be a separate log? This also happens very seldomly (252 / 33.3M) so I can not reproduce it really.
  • c

    cole | data

    05/18/2023, 4:08 PM
    My intuition tells me that the promise being passed to waitUntil is the one throwing the exception. You can always try wrapping those in a try/catch it they aren’t already. As for why that particular exception doesn’t show up in the log is out of my area of expertise. I’ll forward that to the workers logging team
  • o

    omrilotan

    05/18/2023, 4:25 PM
    Thank you, that's a very probable observation. My only waitUntil promise in the worker is actually a log to be sent upon errors, which I have yet to replace with the native log.
  • l

    Leesy

    05/22/2023, 12:43 PM
    I am trying to change which headers get logged in my cloudflare worker logs via the dashboard but I get a 404 when i go to /analytics/logs?fromAccount=true
  • l

    Leesy

    05/22/2023, 12:43 PM
    From here

    https://cdn.discordapp.com/attachments/1041788765783474216/1110186199865364480/image.png▾

  • l

    Leesy

    05/22/2023, 12:43 PM
    Does anyone else have this issue?
  • c

    Chaika

    05/22/2023, 1:43 PM
    Most likely, you are getting that error because you need Enterprise to access the zone-scoped Datasets. The only non-enterprise dataset is the account level Workers Logpush
  • c

    Chaika

    05/22/2023, 1:44 PM
    What do you mean "which headers get logged in your worker logs"? Doesn't sound related to Logpush
  • l

    Leesy

    05/22/2023, 7:49 PM
    In the request part below I want to log the useragent string..

    https://cdn.discordapp.com/attachments/1041788765783474216/1110293326952927393/image.png▾

  • c

    cole | data

    05/22/2023, 8:45 PM
    This is not currently possible. Adding custom fields (as they are called in
    http_requests
    dataset) is not supported for
    workers_trace_events
    .