Ok, let me tell the truth. OpenDAL is fully community supported and not backed by a business company. So we are not hiring any people, instead, we are open to all kinds of contributions.

That’s said that, we don’t have a dev team, we only have a dev community.

Hello, has anyone here dealt with a problem where OpenDAL apparently does NOT set headers for a file in GCS? It works for most files, but for some reason when I upload this one file, Content-Type, Content-Encoding and publicRead ACL is not set. I get no errors, these headers are just silently left unwritten, and it happens every time

What's the filename for that file?

model_opt.glb

Is the only difference that this file can't be written? How large is the file?

I have no trouble with other model_opt.glb files though, just for this one (so far)

The final compressed size is about 9.3 MiB

I see.

I think it's a bug that we didn't carry the

OpWrite

in

gcs_initiate_multipart_upload

We should set them while starting the upload process.

It's worth to add a test to set content-type for a large file, like 16MiB

That makes sense, another file that is 27.6 MiB large does not have headers either.. I just didn't notice it at first, because it was supposed to be private anyway

Maybe a workable stopgap solution would be to upload the whole file in one... only, I am afraid to compress whole files instead of streaming compression

Or is there an actual good and quick fix here?

You can try op.write_with(path).chunk(128MiB).await;

This can ensure that all files under 128MiB will upload in oneshot.

I meant a bug fix, not a workaround

o, bug fix is easy.

Like what we do in

gcs_insert_object_request

, add

OpWrite

for this function and set the correct header.

Okay, thanks. Will probably try that next week

Thank you in advance!

I submitted a PR. I felt lazy, so I packaged a fix for another breaking but small-scale bug. I hope you don't mind https://github.com/apache/opendal/pull/6275

Great work!

I tested this on our project, and the headers now work on multipart upload with these changes, as does the presign feature

That's nice.

My PR also adds custom metadata in multipart upload, but I didn't test that part since I have never used them

hi, I am trying to use the http operator in the python bindings to access a url with a custom certificate, but i get:

error sending request for url (https://internal/index.html): client error (Connect): invalid peer certificate: UnknownIssuer

with:

python3 -c "from opendal import Operator; print(Operator('http', endpoint='https://internal').read('index.html').decode().splitlines()[0])"

I have added the certificate to the system, and it works with curl, but I can't get it to work. It seems to be related to the features enabled by reqwest, but I guess I can't control from the python library? Is there any way around this?