Yeah I see this is workers not pages

Hi everyone. I will be eternally grateful if we can solve this issue. Rather than using one kv write per user, we add all user json records to kv as a JSON Array (per audience). We identify the audience from the url making the request, parse the json from KV, then request the specific json record. This works great but takes time and CPU resources. Interestingly 90% of the processing time is parsing the JSON (JSON.parse) not getting the json record in the array. So what I want to do is speed up the parsing between worker invocations. Imagine 1 0 million invocations each taking a second to parse the data. It would be great if the data was already parsed. Of course we can make auduences smaller but we would prefer not to place these limits on our customers. Any ideas? Thank you.

If the KV value is stored as JSON, you can do

await env.KV.get('key', {type: 'json'})

which will handle the JSON parsing for you. I doubt it'll be any faster though, honestly

How large is the data though? I would not expect parsing JSON to take a full second, even if you have like a megabyte of JSON

> we add all user json records to kv as a JSON Array The solution is not to do this 😅

But yeah, interested in hearing how large the value is

Another thing you could do is switch to a format that works better as being read as a stream, this should allow you to skip CPU cycles on chunks of the data you don't care about

It's not so much the size of the data (25 MB KV Limit) its the number of records. 100,000 Records takes 1s and 200ms CPU to process . Clients can have millions of records which are all updated at least once a day, sometimes more. Its not just about reading the data either. We have to get all of the data into KV in the first place and have a 3 minute window to do so. Please note time doesn't matter when we can used Bundled Workers. Which work up to 10,000 records leaving enough of the 50ms CPU resource to carry out our other tasks. Typical audiences are much bigger. We then need to use Unbundled where time costs money. Does the suggested approach still use the Worker CPU or does KV do the work. If KV does the work at no CPU cost that would be awesome. Due to the fact json data is changed every day and millions of records are changed at a time using 1 KV Write Per User is not financially viable. Hence the need to use batches. Assuming the suggested approach is processed by workers and takes just as long and consumes just as much CPU resources is there another way. I am happy the pre-process the data ready for use in each invocation, I am looking to make the process more efficient. Of course using /audience/random prefix / in the URL can make each KV JSON Array smaller (max 10k per prefix) but I am looking for a more elegant solution. Please explain how a stream would work with node.js I haven't used streams before. How would I get the same data in and out of KV? Thanks

In this case I'd look into moving away from JSON. JSON is slow here because you spend a lot of time parsing the records you don't care about As a naive and simple approach imagine each record was delimited by a newline and the first 8 bytes denoted the user id You can stream the data from KV and check the first 8 bytes of each line for the relevant user. You can discard every record that isn't the correct ID, and every record after it (stop reading) From there you could look to sort the audience batch (the value you store in KV), you could skip over whole chunks of the streamed data instead of checking each record line by line

With sorted data you can do a binary search on each chunk streamed in

Hi Guys, Need some help. Is there any option in workers to use stateful memory which can be accessed and updated externally and shared across all incoming requests ? Apart from using KV or Cache API

durable objects can do this 😄

Cloudflare seems to place a huge emphasis on workers but I could never quite get what they exactly did. Could someone explain what they do?

have you read this article? https://developers.cloudflare.com/workers/learning/how-workers-works/

Hey guys and gals, is there a way to see the incoming request count for prev. months? It show the current month in the top right corner but I would like to compare to prev months.

ah seemed to have missed this

thanks

You can check the previous month under Profile > Billing > Billable usage at least for a paid Workers plan, not sure if you can on free

I do have a paid plan yes

Then billable usage will show you nice breakdowns over time going back to the previous month

and you can also check your invoices for past months and see a count on there

It shows me zones and that is 1. so everyday in the graph is 1

can you share a screenshot?

should have a graph for workers requests:

and at the top you can change to last month

Hi 👋 , looking for clarification on costs with unbound usage. I'm planning on using a worker to call an external API, then (optionally) stream an object back to the client. I wouldn't be billed for the duration of the external API call, or for streaming the response, right? Since the worker would be considered idle in both cases?

Duration of the API call, yes

Streaming the response, no

Even if the response isn't modified? E.g. like the diagram here https://blog.cloudflare.com/workers-optimization-reduces-your-bill

Oh, I meant yes as in you will be billed for it