https://discord.cloudflare.com logo
Join Discord
Powered by
# r2
  • v
    Unrelated 
    nit
    -
    Copy code
    customMetadata: {
        'content-length': response.headers?.get('content-length'),
      },
    is redundant. The size of the file is already stored by us for free (see the 
    .size
    member variable beside 
    .customMetadata
    in 
    R2Object
    ).
  • v
    yes, just tried it
    Copy code
    Connected! Streaming logs from nftstorage-link-api-staging... (ctrl-c to quit)
{"outcome":"ok","scriptName":null,"exceptions":[],"logs":[{"message":[{}],"level":"log","timestamp":1650982992419},{"message":["header","23"],"level":"log","timestamp":1650982992675},{"message":["[2022-04-26T14:23:12.675Z] ","name: 1307 ms"],"level":"log","timestamp":1650982992675},{"message":["TypeError: The value cannot be converted because it is not an integer.\n
    header","23" per 
    console.log('header', response.headers.get('content-length'))
  • v
    oh nice, good catch! I am going to use it
  • v
    Just sanity check. Are you sure it's 
    R2.put
    that's throwing the exception and not something else?
  • v
    Yes, so changed code to:
    Copy code
    const response = await getResponse(request, env, normalizedUrl)
if (!response.ok) {
    throw new HTTPError(await response.text(), 400)
}

console.log('pre put log', normalizedUrl.toString())
r2Object = await env.SUPERHOT.put(normalizedUrl.toString(), response.body)
console.log('post put log')
    And called it with tail on
    Copy code
    Connected! Streaming logs from nftstorage-link-api-staging... (ctrl-c to quit)
{"outcome":"ok","scriptName":null,"exceptions":[],"logs":[{"message":[{}],"level":"log","timestamp":1650983696930},{"message":["pre put log","https://bafkreidyeivj7adnnac6ljvzj2e3rd5xdw3revw4da7mx2ckrstapoupoq.ipfs.nftstorage.link/"],"level":"log","timestamp":1650983697103},{"message":["[2022-04-26T14:34:57.103Z] ","name: 1177 ms"],"level":"log","timestamp":1650983697103},{"message":["TypeError: The value cannot be converted because it is not an integer.\n    at permaCachePut (worker.js:3883:35)\n
    worker.js line is the put line + no post put log
  • v
    Just sanity check - can you put a 
    try/catch
    around the 
    SUPERHOT.put
    and log from within the catch just to be extra sure?
  • v
    Yeah, same
    Copy code
    `
console.log('pre put log', normalizedUrl.toString())
    try {
      r2Object = await env.SUPERHOT.put(normalizedUrl.toString(), response.body)
    } catch (err) {
      console.log('err', String(err))
      throw err
    }
    console.log('post put log')
`
    Copy code
    `
Connected! Streaming logs from nftstorage-link-api-staging... (ctrl-c to quit)
{"outcome":"ok","scriptName":null,"exceptions":[],"logs":[{"message":[{}],"level":"log","timestamp":1650984564376},{"message":["pre put log","https://bafkreidyeivj7adnnac6ljvzj2e3rd5xdw3revw4da7mx2ckrstapoupoq.ipfs.nftstorage.link/"],"level":"log","timestamp":1650984564449},{"message":["err","TypeError: The value cannot be converted because it is not an integer."],"level":"log","timestamp":1650984564449},
  • v
    Thanks for double-checking
  • j
    does 
    CopyObject
    work yet? I'm sending the 
    x-amz-copy-source: /<bucket>/<key>
    header, but getting a 400 Invalid Argument: copy source bucket name error back from r2. The source bucket is the same as the target bucket (same error when target bucket is different than source)
  • j
    aha! it works without the leading slash. This seems like an R2 bug in 
    CopyObject
    . Also the success response content-type should be 
    application/xml
    , not 
    text/plain;charset=UTF-8
    Also copies to a bucket that does not exist fail with 500 internal error, not some 4xx error.
  • v
    Appreciate your help so far. Just created a repo with two bare minimal workers, with API worker bind to Gateway Worker. API Worker does request to Gateway Worker and adds its response body to R2 https://github.com/vasco-santos/workers-bindings-r2 Get exactly same error 
    ypeError: The value cannot be converted because it is not an integer.
    and should be easier on your side to test with README instructions
  • v
    Wow. That's much appreciated
  • v
    What's your account ID?
  • v
    Hmm... we do have e2e tests to do cross-bucket copies using the JS SDK
  • j
    ah sorry i was copying to a bucket that did not exist (for the 500 error), updated my prev comment - still should probably fail with an error in the 400s with a better message : )
  • v
    Yup - that's super helpful. We don't have a test for that. Thanks!
  • j
    noticed yesterday that the 
    Quiet
    parameter to 
    DeleteObjects
    is ignored/not-implemented, deleted keys are included in the response no matter the value
  • v
    Looks implemented from my end and we have e2e test coverage
  • v
    What's the XML you're sending?
  • j
    Copy code
    <Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">

  <Object>
    <Key>a</Key>

  </Object>
  <Object>
    <Key>b</Key>

  </Object>
  <Object>
    <Key>c</Key>

  </Object>
  <Quiet>true</Quiet>
</Delete>
  • v
    Hmm.... that's very strange
  • j
    haha wait those commas did not make a difference
  • v
    Just ran a unit test to double-check:
    Copy code
    const xml = '<Delete><Quiet>true</Quiet></Delete>'
    expect(
      fromXML(
        xml,
        z.object({ Delete: z.object({ Quiet: z.boolean().optional() }) }),
      ).val,
    ).toEqual({
      Delete: {
        Quiet: true,
      },
    })
    That passes and that's roughly the schema we use. Our e2e tests use the official amazon JS SDK (v2 though). What are you using?
  • j
    it's definitely getting parsed on your side, i tried passing True instead, and got a failed schema error, it's just doesn't seem to have the desired effect (no keys included in the response)
  • v
    Copy code
    const deletedKeys: DeleteObjectsResultItem[] = quietMode
    ? []
    : resp.val.toDelete.map((d) => {
        return { key: d.object }
      })
  <snip error mapping>
  return ok([...deletedKeys, ...errors])
    We definitely have this logic....
  • v
    Copy code
    const res = await sync(
      s3.deleteObjects({
        Bucket: bucketName,
        Delete: {
          Objects: [{ Key: 'fish1.jpeg' }, { Key: 'fish2.jpeg' }],
          Quiet: true,
        },
      }),
    )

    expect(res.Deleted?.length).toBe(0)
    expect(res.Errors?.length).toBe(0)
  • v
    That's the e2e test
  • s
    Can we make files in R2 buckets public to download with workers?
  • i
    Yup, that's how you need to do it so far.
  • i
    Just get from R2 in the Worker, then publish the Worker.
1...585960...1050Latest