Here is something for multiplatform files. Of course, no API stability, no binary compatibility, nothing. We even didn’t have a proper API design session 🙂 Binaries: https://bintray.com/beta/#/orangy/maven/kotlinx-files/0.1.0-dev-26 GitHub: https://github.com/orangy/kotlinx-files

pushed dumb implementation of memory fs

Is passing a FileSystem reference around a deal breaker? I think it's actually quite powerful not just for testing but also metrics and enforcing certain performance requirements

You could have a LoggingFileSystem that records how many bytes were written where and how long that took

You could have a StrictFileSystem that refuses access from the UI thread

But such indirection is only possible if you separate getting the FileSystem from using it

Imagine how baller it would be if IntelliJ’s sprawling VFS could participate here

It’s not a deal breaker, no, but it’s quite a problem for usability. There are other issues with a concept of universally applicable

Path

, e.g. in Windows

C:foo.txt

is a “path to foo.txt in a current directory for drive C:“, and it’s different from

C:\foo.txt

which is an absolute path. It is kinda bizzare to apply these Windows concepts to any other file systems. I’m not saying it breaks it, just pointing out there are issues in FS-independent paths. Also I agree that having a data-only path brings a lot of benefits. Just need to have a nice idea/design here.

Hi. I've promised my use case for file references. I have in fact two. First is rather simple, and, I believe already covered in the library. I want an external resource management, specifically data files. What is important is that I want to be able to evaluate file by parts without loading it to memory. For example, I need to read files preambles, store them and then access full files if necessary like it is done here: https://bitbucket.org/Altavir/dataforge/src/default/dataforge-core/src/main/kotlin/hep/dataforge/io/envelopes/DefaultEnvelopeReader.kt.

The second one is more esoteric. I need file references as a resource handlers to pass them to external programs. Something like this: https://bitbucket.org/Altavir/dataforge/src/default/dataforge-core/src/main/kotlin/hep/dataforge/workspace/FileReference.kt. I will write specifics later.

Launched it as a js dependency and immediately got "Cannot find module 'fs'". Added the module manually into frontend config.

The same for

text-encoding

Fixed it and now I get

"IOException: Failed to create an input stream for geofile_full.json: TypeError: JsFileSystem$Companion_getInstance(...).fs_8be2vx$.openSync is not a function
    at IOException_init (webpack-internal:///./kotlinx-io.js:4755:17)
    at JsFileSystem.openInput_pizevn$ (webpack-internal:///./kotlinx-files.js:740:13)
    at jsRoot_0 (webpack-internal:///./dataforge-vis-spatial-js.js:321:43)
    at JSRootDemoApp$start$lambda (webpack-internal:///./dataforge-vis-spatial-js.js:166:5)
    at render (webpack-internal:///./dataforge-vis-spatial.js:133:5)
    at JSRootDemoApp.start_x7u0o8$ (webpack-internal:///./dataforge-vis-spatial-js.js:175:5)
    at start (webpack-internal:///./dataforge-vis-spatial-js.js:150:19)
    at HTMLDocument.eval (webpack-internal:///./dataforge-vis-spatial-js.js:122:31)"

Maybe I need to use different file name...

No, it seem to expect the node back, not the browser. A pity. I will look into it later, maybe create additional back for browser cache/local files.

Indeed, browser configuration is not there yet.

Is there a simple way to convert

ByteBuffer

to

Input

on JVM?

Note, that we are rewriting kotlinx.io right now, so it all will change.

Another use-case for io: I need to redirect input to output. The best solution without creating intermediate buffer I found is to write:

while (!endOfInput){
    out.writeByte(readByte())
}

If I understand current IO logic correctly, the input in general has a sequence of pooled buffers, so it would be simple to redirect those buffers directly to the output, but I can't find a way to do it with current API.

I am but a tumbleweed happening into here by idle weekend projects... but I do have a a breadth of real-world experience with formats that would be well accomodated square in a vfs api layer.

i work daily with a CMS of loosely peiced together media formats of json, iso mp4 and webm box formats, and xml dash streaming media manifests. it would be nice to spiral outward from something simple like a plan9 vffs module based plugin system to accomodate the breadth of translations from windows C drives all the way to xml-indexed byteranges of http buffers across a unified filesystem of buffer access

having kotlin as the unifying peice of dozens upon dozens of nodejs workarounds and avoiding jvm or python assumptions would be a win here imho since kotlin-io is still apparently very very late bound to the kotlin-common spec at this time

there was also an OS, VSTA, Andrew Valencia Simple Tasker System, where the mach kernel / 9p module was the ~2k boot kernel enabling "everything is a vfs interface" hosting a complete gnu OS about on par with linux at the time.

recursive chunk/box/inode trees as a first class abstraction is underwhelming in the dominant languages to date.

it doesn't take much to create such abstractions, but these abstractions never seem to creep into mainstream

Hey, guys. Any suggestions on how to consume kotlinx-files in my app (in tests)? Should https://bintray.com/orangy/maven/kotlinx-files-dev#files/org/jetbrains/kotlinx work?

^ I was able to add maven repo and gradle download artifacts. However build for native failed:

> Task :desktop:performance_app:compileKotlinNative FAILED
w: skipping /Users/asmirnov/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-files-macosx64/0.1.0-dev-31/77854b4718090e294deda0273771c40396a6e381/kotlinx-files.klib. The abi versions don't match. Expected '[9]', found '5'
w: The compiler versions don't match either. Expected '[1.3.1]', found '1.1.2-release-6625'
e: Could not find "/Users/asmirnov/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-files-macosx64/0.1.0-dev-31/77854b4718090e294deda0273771c40396a6e381/kotlinx-files.klib" in [/Users/asmirnov/Documents/dev/src/kotlin-research/project-kotlin/desktop, /Users/asmirnov/.konan/klib, /Users/asmirnov/.konan/kotlin-native-macos-1.3.1/klib/common, /Users/asmirnov/.konan/kotlin-native-macos-1.3.1/klib/platform/macos_x64].

FAILURE: Build failed with an exception.

Is it correct that exactly the same k/n should be used (that was used to compile native artifact)?

Hey, guys. Any updates on this? What's the suggested approach for multiplatform files?

If anyone’s looking for an alternative to

orangy/kotlinx-files

,

<http://pw.binom.io|pw.binom.io>

was just updated with macOS support: https://github.com/caffeine-mgn/pw.binom.io/commit/1ee821058ccce7309fa7694e7b3a8142081092ef#diff-783b540faf0155be89004ca011b2c8fc