This has changed since then too iirc

I have R2 access and I've also fixed both the issue in the wiki 🙂 The CompleteMultipartUpload is failing like this

2022/05/04 16:34:29 DEBUG : POST /rclone/file.txt?uploadId=AAJcNgopj%2BX%2BQOnZblAhgHFXYt5GzN7MBylPyGthxqrrJ4TDfGABNwf8adg7SLLS%2B7eRIp6OaQ%2FWvjR6lzrKUX0in%2Fwri5l9q0pZ1WISyG%2F2JPQLUQa%2B2hbBdKzi36OJse4wPxao2WSaEjMhYtFduH%2F3lmtjdWvfevm8hHTs9Wlm8hhJZsHTDrtEhUDiZx0M9crj6GMrcvuSppAaEl95BpSbfrZqlxYsYCflBS%2Fl3TVmpl93wALwFaIWgbMrnRKi%2BTlq8SP5PeN4Z41f2tvI2g1DXOkqXML2DbG5cgp0suzvVZQfjwIcL4EyqGE5iDqXVw%3D%3D HTTP/1.1
Host: 14aad7c9ed489151b51557e321b246cf.r2.cloudflarestorage.com
User-Agent: rclone/v1.59.0-beta.6116.781bff280.fix-5422-s3-putobject
Content-Length: 421
Authorization: XXXX
X-Amz-Content-Sha256: d60d5c7de7cb2c0ac85f252b556a6e6a4696b6181d7a9e80b1d0633139ac2dda
X-Amz-Date: 20220504T153429Z
Accept-Encoding: gzip

<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Part><ETag>"AEXG0nZNf6jx5XD+rl9ikACIDYq5BRhe1B+Q1nn6bEU76waJFoM1Cv9K/YsG0I76U/7NZtMmge8GO/gTZtqMlzxKXFB+gH7npfE52MmvIa4jLboaljxzcfpYrb0dXSRBkBNHbfHHWKKCj1co1sbpnhgyk9ditdnt1mtT1gzzakR0oGF+sMtLlWB3EbgFn+srb6AVIgQJ043OQ79UTrDANllWSGGwiGgTUBNOhc2RRVDwiGhjLlBkZiOoUIsRbknybQ=="</ETag><PartNumber>1</PartNumber></Part></CompleteMultipartUpload>

With the response

2022/05/04 16:34:30 DEBUG : HTTP RESPONSE (req 0xc00060f500)
2022/05/04 16:34:30 DEBUG : HTTP/2.0 400 Bad Request
Content-Length: 149
Cf-Ray: 70624d04ab768898-LHR
Content-Type: text/plain;charset=UTF-8
Date: Wed, 04 May 2022 15:34:29 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
Vary: Accept-Encoding

<Error><Code>MalformedXML</Code><Message>The XML you provided was not well formed or did not validate against our published schema.</Message></Error>

I can't see anything obviously wrong with the XML and that code works with dozens of other providers so I suspect R2 but I might be wrong 🙂

If you want to try that yourself, you'll need the rclone version from here: https://github.com/rclone/rclone/issues/5422#issuecomment-1116330745 And something like this to test it and show the HTTP traffic

date -Is > /tmp/file.txt ; rclone copy /tmp/file.txt r2:rclone -vv --dump bodies --retries 1 --low-level-retries 1 --s3-upload-cutoff 0

I posted an updated version of rclone to that issue which works for non multipart uploads now.

Rclone:

<ETag>"AIwC3M+GkIGiAk3s2dnvPojGecNarOiCa0Zn/ZTMNPumf0PACnuyma0PrwtKGmOZ6kKlxYgI+id4WDWdRcpPWXeyEoZ65LpYnOsDo1kA2M53IOCL3rwbRkWcQXT6vLqDRiYbAcgo6Ku4QX2SxG5nByXNZDtGzboz32qklCoRK5C0NF2LG/MVw2YQMn3g7LA+9vEusp5JeJHj7cuh+ZUcvYqTvEw3nE9SgSs3biSE4BbE21DrZC8FKzydRE0Cw1lVmQ=="</ETag>

AWS CLI:

<ETag>"ADvns+vv3z7rg8c3pLm8tbkPGVtTkb6U2Y0mE5F7thgl9NUVdJ7QCLAAOHS9UJzFV3PEB4yQ7yThOKavy+lcpraxiaYkU/M47KqSzSRD+htYxN9gOb11SEFGCAY+W50FIiS0EkHw7j9jswyk0+6kqqg70Pf6oe6N35yV3IJBx22+3ZWmS71AaIeLhGcNJZqZytWbEkcF+UFjP1n4pIRoPmDClC501TcmRLJhVG7nTZfyvS6IOuoqqtZ0vjPn5H2oCg=="</ETag>

Rclone XML-escapes the quotation marks - perhaps R2 does not like this? Also seems to include a

;

at the end 🤔

Well spotted @albert . I don't think I have control over the XML escaping - it is the AWS Go SDK doing the creation of the XML here. I think the

;

is just the end of the entity

"

Ahh, of course it is, my bad 😅

@Vitali I just tried manually completing an Rclone-created multipart upload (after replacing the XML entities with proper quotation marks) and it works great. I think this is something R2 should fix rather than Rclone - escaping quotation marks inside XML tags is standard behaviour afaik.

I need to double-check if the version of the XML parser we use doesn't support doing that. We're still on fastxml 3

I'll make a change today to allow

empty

and

us-east-1

regions to be identical to

auto

. Sorry about the churn.

I can resume the backups ahah

you aren't a terraner?

R2 isn't, apparently! 😄

To the moon!

so I'm not the only one that saw the change and thought > oh, so they're expanding already...

haven't you heard about the

ISS

colo?

stlink wen

I'd like that 😆

don't you mean the slink colo

unless there's another colo with the code

ISS

that I haven't heard of

"international Space Station" /s

I wouldn't be too surprised if it actually existed

Anyone have a good tutorial on how to handle file chunking with Workers?

What does this mean?

HTTP chunked encoding? Or something else?

It actually doesn't they supposedly have internet from Houston.

Yeah, that sounds right. To be honest, I don’t know a whole lot about it, but I have just noticed that some people have talked about using chunking to improve uploading large files, so I wanted to learn some more about it.

Well HTTP chunked encoding wouldn't be relevant here. Chunked encoding can't be used with R2 because R2 needs to know the object size upfront (& chunked encoding doesn't improve performance). I'm not sure what you've read - can you provide a reference? That might make it easier for me to explain the discussion.