https://discord.cloudflare.com logo
Join Discord
Powered by
# r2
  • j

    James

    04/02/2022, 12:05 AM
    Yeah makes sense, we were warned the bindings would almost certainly change before launch 🙂
  • w

    Walshy | Pages

    04/02/2022, 12:05 AM
    hahaha ty for the detailed answer
  • v

    Vitali

    04/02/2022, 12:06 AM
    At some point we'll make the old bindings start warning you in the logs. Any users still using them we'll try to make proactive contact with. At some point we may make the decision to remove them anyway to cleanup the codebase.
  • w

    Walshy | Pages

    04/02/2022, 12:11 AM
    expirationTtl is removed? Are there plans to have temp objects?
  • v

    Vitali

    04/02/2022, 12:19 AM
    expirationTtl
    was never a thing and was never hooked up to anything IIRC (or maybe it was actually setting the cache expiry header? can't recall). There is a plan around it and we will reintroduce but we probably need to have object lifecycles comprehensively solved (although maybe we put in a hack before then - hard to say. we'll see how things develop). Talk to us internally if you absolutely need it but most likely our answer will be no.
  • v

    Vitali

    04/02/2022, 12:44 AM
    Thanks to @User for pointing out a couple of things I missed. * Blob was missing from the accepted types. This was an oversight in the runtime code I wrote so the generated types are correct (if it's fixed next week in workers-types then it means the runtime also has it fixed) * ListOptions has a poorly documented and poorly typed
    include
    field which is strange since it's R2 specific. This is what is should be instead:
    Copy code
    /** 
       * If you populate this array, then items returned will include this metadata.
       * A tradeoff is that fewer results may be returned depending on how big this
       * data is. For now the caps are TBD but expect the total memory usage for a list
       * operation may need to be <1MB or even <128kb depending on how many list operations
       * you are sending into one bucket. Make sure to look at `truncated` for the result
       * rather than having logic like
       *
    * while (listed.length < limit) { * listed = myBucket.list({ limit, include: ['customMetadata'] }) * } *
    Copy code
    */
      include: ("httpMetadata" | "customMetadata")[]
    Basically it controls if the returned
    R2Object
    list will have a `httpMetadata`/`customMetadata` populated. The performance notes are very preliminary and I wouldn't take them too seriously. Design around what's easiest for now. The performance of the system will be changing drastically in the coming weeks.
  • a

    andrew

    04/02/2022, 12:54 AM
    wow really exciting to see things moving along. looking forward to trying out the beta once it's available
  • v

    Vitali

    04/02/2022, 12:54 AM
    How much load can you throw at us?
  • v

    Vitali

    04/02/2022, 12:55 AM
    and are you looking for accessing it via Workers or S3
  • a

    andrew

    04/02/2022, 12:58 AM
    one bit to pay attention to -- if a content-encoding header is specified during PUT, it should also be present for a GET. this is important if you're using deflate/brotli/etc and want browsers to auto-decompress content when fetching
  • v

    Vitali

    04/02/2022, 12:58 AM
    Yup. That's exactly how R2 works
  • v

    Vitali

    04/02/2022, 12:58 AM
    We also support overriding those headers during S3 GET (doesn't make sense for a worker)
  • a

    andrew

    04/02/2022, 12:59 AM
    cool 😄
  • a

    andrew

    04/02/2022, 12:59 AM
    can metadata / http headers of an object be changed after upload, or does that need a reupload?
  • v

    Vitali

    04/02/2022, 12:59 AM
    And I wanted to keep it a surprise but @User's request here will hopefully make open beta because it's a semi breaking change so we want to level set expectations at launch
  • v

    Vitali

    04/02/2022, 1:01 AM
    You can change the metadata/http headers in the S3 API but issuing a CopyObject with the same source and destination (sorry folks, that's how the S3 API works). I plan on plumbing through a more sane convenience name for Workers bindings eventually (which internally just aliases to the same operation internally that copy with identical source/dest bucket + name maps to).
  • v

    Vitali

    04/02/2022, 1:02 AM
    We also have an extension to CopyObject. You can use "MERGE" as the
    x-amz-meta-directive
    which lets you update key/values without having to know the entire set ahead of time
  • v

    Vitali

    04/02/2022, 1:03 AM
    I don't know how good tooling support on that is (so please ask your favorite libraries to add support) but we support it
  • a

    andrew

    04/02/2022, 1:04 AM
    sure makes sense, great stuff once testing is available, i would be looking at high throughput GET usecases... and testing questions like "Can I achieve 1GB/s+ with multiple simultaneous GET requests?" most likely via the S3 endpoint and fetching multiple objects simultaneously across multiple threads to VM destinations (would consider using workers, if that turns out better for performance than S3 GET endpoint)
  • v

    Vitali

    04/02/2022, 1:05 AM
    Is that manual load to play around with or continuous sustained load?
  • a

    andrew

    04/02/2022, 1:05 AM
    would consider range requests as well
  • v

    Vitali

    04/02/2022, 1:05 AM
    range requests are a thing
  • v

    Vitali

    04/02/2022, 1:06 AM
    Our S3 API implementation is fairly complete
  • a

    andrew

    04/02/2022, 1:06 AM
    for the real world use case, it wouldn't be continuous load like that (1GB/s), but ability to burst to that level is very desirable/important
  • j

    john.spurlock

    04/02/2022, 1:06 AM
    looks like single range only? or will it support multiple ranges like the http range header?
  • v

    Vitali

    04/02/2022, 1:06 AM
    Single range for now unfortunately
  • v

    Vitali

    04/02/2022, 1:06 AM
    It might be a couple of years before we get to multi range
  • j

    john.spurlock

    04/02/2022, 1:07 AM
    k, and sorry which request of mine were you referring to? conditional puts?
  • v

    Vitali

    04/02/2022, 1:07 AM
    I'll leave you to guess at based on the hint already provided 😛
  • v

    Vitali

    04/02/2022, 1:07 AM
    Which one might be a breaking change
1...151617...1050Latest