Hello everyone i'm using multer and gcp buckets fo...
# orm-help
m
Hello everyone i'm using multer and gcp buckets for uploading images but as we knew that normally for uploading large video i have seen a practice that first via multer we parse and store big file in a temporary folder and than stream over this temporary file and upload to bucket is there is any other way to skip this storing file in temporary folders of os and than streaming to bucket?Plz let me know thanks in advance.
1
n
Hey Moin 👋 You could use MemoryStorage option to store images in memory as Buffer objects, this won’t store the image in the disk.
Copy code
const storage = multer.memoryStorage()
const upload = multer({ storage: storage })
MemoryStorage Reference
m
@Nurul But i want to upload large files like videos putting into memory storage can cause leaked or out or memory issue
Such as GB's filethan what we can do?
n
In that case, can you store the video in a temporary folder and once it’s uploaded you delete the video from the temporary folder, does that work?
m
I have done this before using busboy i'm asking for any good approach which big organizations uses.
What they does for uploading large files i'm not able to figure iut this problem
n
Okay, one other solution I would have recommended is streaming your video files, I’ll search for any other alternatives.
m
If you can provide me any example that would be mush apprciable please
@Nurul But for streaming also doesn't we have to store file first in a temporary folder than stream from that folder after streaming delete file from folder?
n
Example of streaming?
m
Yes actually i'm using nestjs
n
But for streaming also doesn’t we have to store file first in a temporary folder
No, Cloud Storage supports streaming transfers, which allow you to stream data to and from your Cloud Storage account without requiring that the data first be saved to a file. From their docs
m
But using multer we have to specify storage thing which will store file somewhere first?
Than how's this possible that we can stream file directly?
n
You won’t need to use multer if you decide to go with streaming. Is it necessary for you to use multer?
m
Not necessary but without multer how we going to parse multipart-form data?
In example we have
Copy code
const file = myBucket.file(destFileName);
what this destName thing?
Also not able to understanfd this line as well
Copy code
passthroughStream.write('input text');
t
You can use signed-url to directly upload file from frontend.
👍 1
m
@Taras Protchenko What's the purpose of signed-url?
n
destFileName is the name of the file you would want to store in the bucket.
And in that write function, you would need to pass the blob of your video
m
@Nurul ok destFileName cleared but not cleared about blob thing yet .
t
As I understand your problem is that the server processes large files. Using signed-url, you can not upload files to the server, but directly upload them to the bucket from frontend.
m
Do we have any library with the help of which i can upload file fromt front-end or client side instead of server/backend
??
n
Can you have a look at this cloudinary widget? https://cloudinary.com/documentation/upload_widget