I wonder it's possible to use OpenDAL with NodeJS due to this issue: https://github.com/apache/opendal/issues/5070

Hi, would it be possible to approve https://github.com/apache/opendal/pull/7199 It's been reviewed a while ago ( https://github.com/apache/opendal/pull/7199#issuecomment-4249622325 ) but the CICD tests are flaky (failing randomly, nothing to do with the code change), I cannot force rerun the tests and I didn't want to push changes just to trigger the CICD pipeline. So I had to wait for main to changes and merge it to trigger the tests. They all pass - the OPFS change is isolated. It works fine - I use it in my WASM project. Much appreciated! Thanks!

Thanks for merging the PR @Xuanwo ! If you are curious to see where I am using OpenDAL+OPFS in WASM, here it is: https://dagraph.com cheers.

I will merge 10x faster if I have known you are building such cool stuff

Hi, I'm new here and a student looking to contribute to open source projects. How to proceed ? The website is showing 2025 projects and they are all closed. Please I need your help. Thanks

@Xuanwo can you update GitHub Action's allowed action? ~ -

rubygems/release-gem@f0d7faff26625599a847d40d9fa28ace24c2aacc

-

rubygems/configure-rubygems-credentials@762a4b77c3300434bb57c7ce80b20e36231927aa

Also listed [here](https://github.com/apache/opendal/pull/7672).~ I found what to do. Please ignore this message.

@Xuanwo Hi raised [#7629](https://github.com/apache/opendal/pull/7629) for initial globbing support for your review whenever you get a chance.

Just noticed that the memory service doesn't seem to support the rename capability, may I ask why?

No native support, yes. Implemented as simulated, aka, copy and remove

I ran a quick benchmark and noticed a 5x performance difference between OpenDAL and standard tokio fs, is this to be expected or is it due to an issue on my end? I expected it to be similar and the syscalls themselves taking up most of the time. Total files : 1000 Throughput OpenDAL : 5582.291712217004 Duration OpenDAL : 1.791378974s Throughput Tokio : 23291.168105044584 Duration Tokio : 429.347294ms

Test code:

rs
#[tokio::test]
async fn opendal_vs_tokio_write() {
    // Payload of 1024 bytes
    let mut payload = String::with_capacity(1024);
    for _ in 0..payload.capacity() {
        payload.push('A');
    }
    let payload: &'static str = payload.leak();
    let total_files = 1000;
    let runs = 10;
    let root = "/tmp/ramdisk/files";

    // OpenDAL
    let op = Operator::new(services::Fs::default().root(root))
        .expect("Failed to create operator")
        .finish();

    let mut opendal_elapsed = Duration::ZERO;
    let mut tokio_elapsed = Duration::ZERO;
    
    for _ in 0..runs {
        let start_opendal = tokio::time::Instant::now();

        for i in 0..total_files {
            op.write(&i.to_string(), payload).await.unwrap();
        }
        for i in 0..total_files {
            op.delete(&i.to_string()).await.unwrap();
        }

        let end_opendal = tokio::time::Instant::now();

        // Tokio
        for i in 0..total_files {
            fs::write(&format!("{root}/{i}"), payload).await.unwrap();
        }
        for i in 0..total_files {
            fs::remove_file(&format!("{root}/{i}")).await.unwrap();
        }

        let end_tokio = tokio::time::Instant::now();

        opendal_elapsed += end_opendal - start_opendal;
        tokio_elapsed += end_tokio - end_opendal;
    }

    let opendal_throughput = (total_files * runs) as f64 / opendal_elapsed.as_secs_f64();
    let tokio_throughput = (total_files * runs) as f64 / tokio_elapsed.as_secs_f64();

    println!("  Total files             : {}", total_files);
    println!("  Throughput OpenDAL      : {}", opendal_throughput); // 5505.096180388658
    println!("  Duration OpenDAL        : {:?}", opendal_elapsed); // 181.649869ms
    println!("  Throughput Tokio        : {}", tokio_throughput); // 24910.126753685377
    println!("  Duration Tokio          : {:?}", tokio_elapsed); // 40.144316ms
}

https://github.com/dyrnq/mntrs found a community fuse project built on top of OpenDAL

I noticed someone implemented conditional reads for WebDAV (https://github.com/apache/opendal/pull/7637) and was wondering about conditional writes. There seemed to be several tracking issues for conditional writes but they're all closed (https://github.com/apache/opendal/issues/4074, https://github.com/apache/opendal/issues/5218). Is there an existing tracking issue I'm missing or should I create one? Another related question. I wasn't able to find a documentation page with which specific native capabilities are supported by each service (there is of course the capabilities list on the service struct documentation pages, but it doesn't mention all capabilities). I can make it if it doesn't exist.

https://github.com/apache/opendal/pull/7815 Hi, could you help me merge this pr and backport it or release a new opendal version such as v0.57.1 ? This pr blocks me when i try to support hdfs in lance. https://github.com/lance-format/lance/pull/5472/

@Xuanwo @erickg please cc. Thanks!

@Qaz The tracking issue was closed for conditional write. Are there services missing implementations? If so, you can open a new issue. OpenDAL defines native capability in the code. Because of SimulateService, an operator could also have more capabilities. And yes, feel free to add an issue and make a design for documentation. Some language bindings also need a way to get to know what capabilities are available.

Thanks for the response. WebDAV is missing it, I'll take a look at making an issue after work

If WebDAV is the only service missing support, you could write a short issue instead of a tracking issue.

I suspect others may also be missing it, but I'll check later

It seems there are 62 backends that don't support

write_with_if_match

@erickg @Xuanwo Hi, master. can we merge this into main branch? https://github.com/apache/opendal/pull/7815 I am blocking by this PR.😂

Hey! You can always build on your branch. I see at least 5 issues before next release unfortunately. And I believe Xuanwo is a bit occupied at the moment.

👌 Thanks for replying~

https://github.com/apache/opendal/discussions/7860 We are setting up access to the NuGet organization (C# package) for OpenDAL PMC members. If you would like to join, please leave a comment here. Thanks!

Hey, I was looking at the capabilities of the memory service after trying to use it for unit tests. It doesn't seem to support rename and copy, but I found an issue about adding those capabilities and a PR to match. Were these capabilities removed or were they implemented through an intermediary layer?

memory/backend.rs

rs
fn copy(
    &self,
    _: &OperationContext,
    _: &str,
    _: &str,
    _: OpCopy,
    _: OpCopier,
) -> Result<Self::Copier> {
    Err(Error::new(
        ErrorKind::Unsupported,
        "operation is not supported",
    ))
}

async fn rename(
    &self,
    _: &OperationContext,
    _: &str,
    _: &str,
    _: OpRename,
) -> Result<RpRename> {
    Err(Error::new(
        ErrorKind::Unsupported,
        "operation is not supported",
    ))
}

The service's native capabilities also don't include

rename

or

copy

rs
let capability = Capability {
    read: true,
    read_with_suffix: true,
    write: true,
    write_can_empty: true,
    write_with_cache_control: true,
    write_with_content_type: true,
    write_with_content_disposition: true,
    write_with_content_encoding: true,
    write_with_if_not_exists: true,
    delete: true,
    stat: true,
    list: true,
    list_with_recursive: true,
    shared: false,
    ..Default::default()
};

https://github.com/apache/opendal/issues/2483 "feature request: support copy and rename for memory services" https://github.com/apache/opendal/pull/2513 "feat(adapters-kv): add rename and copy support to kv adapters"

I hope @Xuanwo remembers. Otherwise, you can always do git bisect until you find relevant commits

The current capabilities were set by Xuanwo during refactoring https://github.com/apache/opendal/commit/90810c69c0897c9d7e53eac04eec9a8d679e48a2#diff-4ecaa410a4d71aca0774a82873bf4a8e8d3af0878c938831c340827df523b4a4R78-R96 It previously used the typed_kv::Adapter trait, which supports rename and copy based on PR 2513. I suspect the capabilities were lost during the refactoring

any chance to get a review? https://github.com/apache/opendal/pull/8010

Hi @Xuanwo opendal 0.56-0.59 are broken on crates.io because of

include_str!

a CHANGELOG

error: couldn't read …/opendal-core-0.59.0/src/docs/../../../CHANGELOG.md: No such file or directory
    --> opendal-core-0.59.0/src/docs/mod.rs:34
     |  #[doc = include_str!("../../../CHANGELOG.md")]

I have to use the git dependency to use opendal

on it