Your multipart upload can remain unpublished however long you want. You'll be paying for its storage but otherwise there's no time at which we go and cancel your multipart uploads.

Oh, awesome! Thanks

So would that be a realistic use-case? Can you think of anything I'm not thinking of in there?

I'm trying to keep my DB as in-sync to my uploads as I can. Previously I just uploaded the object but found sometimes the DB would fail for one reason or another and I'd have out-of-sync uploads unless I manually deleted them.

One problem you'll want to consider is whether there can be concurrent uploads to the same file. So for example, DB query succeeds for upload 1 to file X Publish upload 1 for file X DB query succeeds for upload 2 to file X Publish upload 2 for file X Conceivably, without knowing more about your solution you might have DB query succeeds for upload 1 to file X DB query succeeds for upload 2 to file X Publish upload 2 for file X Publish upload 1 for file X If the uploads are single-threaded, then that's not really possible. However, if this is concurrent it seems conceivably possible because all R2 promises is that once a request succeeds, future requests see that state. Two concurrent state modifications are racy and there's no way R2 can solve this for you. It sounds like you actually need 2PC (two-phase commit) where you do: * DB query commits "publish in progress" for file X state if it's current state is "published" or file X doesn't exist * Publish upload for file X * Db query commits "published" for file X state Then you have a guarantee that things are also strongly consistent in your application. There's a lot of error handling you need to handle to actually make this work in practice, keeping in mind you probably want to handle R2 abort upload however makes sense in your app to avoid leaks, handling retries of the "publish" step correctly if the "published" commit to the DB fails (i.e. "no such upload" = published).

Oh wow, lots of detail! Yeah I definitely didn't think of that and will have to consider it... thanks for your input!

Np. Good luck!

(2PC comes up a lot in R2 as you can imagine as we have the similar problem between our metadata layer and storage nodes)

Oh yeah, interesting.

I do multipart upload that i only confirm several seconds after the last part and work all the time. (using S3 over B2)

any news on when r2 will possibly launch? 😄

open beta in q2, and ga after that ( i think, not very sure)

ga is second half of this year, so before 2022 ends

The 2nd quarter of the year is open beta, the 2nd half of the year is general availability.

Let's guess, May 5 are financial results for the quarter, May 12 is a Cloudflare event (https://www.cloudflare.com/fr-fr/connect2022/) so my bet is to see a beta launch for R2 around that time. Really hoping I am true 😄

So... Maybe I can attend this?

are you french?

(assuming fr is for france/french)

https://www.cloudflare.com/connect2022/ is the same link, so I don't think it's only for french people.

Is it possible to access (read/write) R2 object directly over the Internet via rest API like S3 (ie: not from Worker)? Thanks!

:cough: the channel topic

🙂

I'm sure they don't really mind 😅

Right now it's fairly restricted... they announce giveaways in #909458221419356210 but only a few (like 20 last time) at a time, since it's not yet open-beta.

You should be fairly safe transitioning from S3, there's a feature that will lazily pull from AWS to R2 on first access and then pull from R2 on subsequent accesses.

https://developers.cloudflare.com/r2/platform/s3-compatibility/s3-compatibility/ also documents which S3 APIs aren’t currently supported

so stay clear of those and the migration should be seamless

Yes. We have a fairly flushed out S3 compat (see https://developers.cloudflare.com/r2/platform/s3-compatibility/s3-compatibility/). Like all Cloudflare APIs we also have an APIv4 endpoint which just isn't documented at the moment (& has other restrictions limitations that we'll be lifting by GA)

I'm not very familiar with the s3 api, so this may be a stupid question. From my understanding s3 objects can be json documents, can we get a specific field inside it to save bandwidth and response time? like, the object:

json
{
  "owner": "user::id",
  "data": "base64(...)",
  "meta": {
    "something": "other",
    "hello": "world"
  }
}

is it possible to get like

GET /file::id?path=meta

json
{
   "something": "other",
   "hello": "world"
}

or

GET /file::id?path=meta.hello

json
"world"