Unable to perform multi-part upload for files
# workers-help
i
Hi all, I'm currently trying to use a worker to perform a multipart upload of a file on my front-end. Since my website is hosted on vercel, I have no choice but to tunnel my request through their APIs in order to reach my worker. As a result, I have a hard 8mb limit on the size of my function body. However, when I try to perform a multi-part upload, I get the following error
Copy code
Error Encountered!
Error: completeMultipartUpload: Your proposed upload is smaller than the minimum allowed object size.
    at R2MultipartUpload.complete (/Users/ivanleo/Documents/Coding/schulz/tut/node_modules/@miniflare/r2/src/multipart.ts:490:15)
    at Object.fetch (/Users/ivanleo/Documents/Coding/schulz/tut/src/index.ts:99:30)
    at ServiceWorkerGlobalScope.[kDispatchFetch] (/Users/ivanleo/Documents/Coding/schulz/tut/node_modules/@miniflare/core/src/standards/event.ts:385:13)
    at Server.<anonymous> (/Users/ivanleo/Documents/Coding/schulz/tut/node_modules/@miniflare/http-server/src/index.ts:291:20)
POST /54cbb39f-863d-4365-bc33-d2c31f4a523c.mp4?action=mpu-complete&uploadId=LR5UYQuJToSuQ26G5v_pAYg-bfcsSA_kPKmjbGvhAHy2lqX0n9dFXaUnjZYKI3G-SIdYsBjCTg_3I1srvZqh3GYwK9Oi2GKtxEdhm5jZeV-4kc8hs6c8KaDEHV7Mod32Sa-WiThJoFe1IaWTv0cOF0wwebHBVsOooBky-edIwN8 400 Bad Request (42.35ms)
It seems to me like the individual upload portions succeed but then when it comes to the final step of completing the multipart upload, I get a 400 error because of the error that
completeMultiPartUpload
throws. I'm a bit confused as to how I might be able to fix this. I've tried to modify my partition function so that the last element is always at least 5MB. I originally started slicing from the front (Eg. 12mb file becomes 5 5 2 and the new algorithm would slice it as 2 5 5 ). Any help would be much appreciated. I am using NextJS with typescript and am happy to share source code if needed.
For context, the object I am testing this on is ~60mb. I generated chunks manually using the
.slice
function on the File object using the following code
Copy code
js
export const createChunks = (
  file: File,
  chunkSize = 15 * 1024 * 1024 /* cSize should be byte 1024*1 = 1KB */
) => {
  let offset = 0;
  const originalFileExtension = getFileExtension(file);
  console.log(`Identified Extension as ${originalFileExtension}`);
  const chunks = [];
  while (offset < file.size) {
    const chunkEnd = Math.min(file.size, offset + chunkSize);
    const truncatedBlob = file.slice(offset, chunkEnd);
    const truncatedFile = new File(
      [truncatedBlob],
      `audio-${offset}-${chunkEnd}.${originalFileExtension}`,
      {
        type: file.type,
      }
    );
    chunks.push(truncatedFile);
    offset += chunkSize;
  }
  return chunks;
};
This yields around 13 chunks which each are
5242880
bytes in size and a last one which is
101232
in size.
k
> I originally started slicing from the front (Eg. 12mb file becomes 5 5 2 and the new algorithm would slice it as 2 5 5 ). You'd want the original slicing
i
So I reverted back to the original algorithm I used
Copy code
xport const createChunks = (
  file: File,
  chunkSize = 15 * 1024 * 1024 /* cSize should be byte 1024*1 = 1KB */
) => {
  let offset = 0;
  const originalFileExtension = getFileExtension(file);
  console.log(`Identified Extension as ${originalFileExtension}`);
  const chunks = [];
  while (offset < file.size) {
    const chunkEnd = Math.min(file.size, offset + chunkSize);
    const truncatedBlob = file.slice(offset, chunkEnd);
    const truncatedFile = new File(
      [truncatedBlob],
      `audio-${offset}-${chunkEnd}.${originalFileExtension}`,
      {
        type: file.type,
      }
    );
    chunks.push(truncatedFile);
    offset += chunkSize;
  }
  return chunks;
};
But it still throws the same error that
Copy code
Error: completeMultipartUpload: Your proposed upload is smaller than the minimum allowed object size. (10011)
    at async Object.fetch (index.js:52:31) {
  stack: Error: completeMultipartUpload: Your proposed uplo…10011)
    at async Object.fetch (index.js:52:31),
  message: completeMultipartUpload: Your proposed upload is s…ler than the minimum allowed object size. (10011)
}
I'm using the same worker provided in the docs too haha
k
How big was that file overall?
i
It's around 60mb
Have been using the same file to test
But i'm getting the same error on my local wrangler server and my deployed worker
File gets broken up into 13 5.24 mb chunks and 1.01 mb chunk.
Seems like its around 68.3 mb
k
No idea - my only possible guess would be there's an issue with your actual UploadPart operations and the multipart upload on R2 isn't right
i
Hmm, ok nvm I will try working with the s3 api first and see if that works out
Thanks so much @kian 🙂
2 Views