no worker

kk

i'll try just under 3gb then

3GB isn't a special value

At least not that I'm aware of. Just currently the R2 worker runs out of CPU time encrypting a 3GB file.

i see, so it isn't based on the content-length?

I mean I'm sure it is, but the issue you're describing isn't in the R2 code base. I would be surprised that 3GB is in any way special. You'd have to do a bisect to figure out what the actual limit is (which might also speed it up on our end to figure out which Cloudflare subsystem we're making unhappy)

k, does it matter if the test file contents are all zeros or not? (ie compressible)

🤷‍♂️

k i'll just use mkfile

I wouldn't think so

Actually, it can't matter because if there's any compression layer in front of R2 then R2 will reject the request because it requires a content-length up front.

528,482,304 bytes: fails quickly with 413 Request Entity Too Large (html body) 524,288,000 bytes: takes a while, then fails with 500 InternalError, try again (s3 xml error body) 520,093,696 bytes: succeeds

oh wow. i just learned about r2 after commiting to change s3 providers starting tomorrow 😛

Thanks for answer. I just made my compatibility_date = "2022-04-26" (today) but still getting

"The value cannot be converted because it is not an integer."

Note that this only happens when I do a R2 put with

response.body

from Worker Binding fetch response. If I do a put with other

response.body

it works fine. Sadly, the error is not informative 🤷‍♂️

curious: when a single DO instance receives requests faster than it processes them and the request queue builds up to a certain point, new requests start failing. Say a single hot R2 object is receiving a large number of simultaneous incoming GETs, will it have the same problem and start failing?

Currently yes. I'm waiting on a review that will speed things up & R2 will start to work in a special DO mode. To actually make the release I'll be taking down R2 for the cutoff (I'll give a shout here when that happens)

Do you have a gist of an example that's failing?

I too don't understand where

The value cannot be converted because it is not an integer

could be coming from. There's nothing in the runtime for

put

public beta bindings that takes in a WebIDL integer which is where that error would come from. The internal beta bindings would generate that error but it sounds like you're saying it's conditional on the type of

ReadableStream

you're putting in there.

excuse the ignorance, but “DO”?

Durable Objects

ah!

I don't have a gist, but here is the code path: https://github.com/nftstorage/nftstorage.link/blob/feat/api-with-worker-bindings/packages/api/src/perma-cache/put.js#L45 that throws in R2.put The response is simply a worker bindings fetch response https://github.com/nftstorage/nftstorage.link/blob/feat/api-with-worker-bindings/packages/api/src/perma-cache/put.js#L83

I can definitely get a simpler test scenario with two workers if there is no clear idea on what it might be. Certainly some difference on Readable stream from worker binding fetch

But you're saying other `put`s work fine if it's a different kind of body?

yes, if I import code that I have in the other Worker handler and run it within the worker without worker binding it works

so it just errors when worker bindings in the middle 🤷‍♂️

Any chance you can point me to a branch where you're saying the put works?

https://github.com/nftstorage/nftstorage.link/blob/fix/integrate-edge-gateway-in-api-until-workers-bindings-available/packages/api/src/perma-cache/put.js#L48 is the put, with tweaked import of code from other worker: https://github.com/nftstorage/nftstorage.link/blob/fix/integrate-edge-gateway-in-api-until-workers-bindings-available/packages/api/src/perma-cache/put.js#L107

For the worker subrequest - is it returning a content-length?