Hi, is it possible to get a `Sink`/`Source` that corresponds to stdout/stdin using kotlinx.io on Native and WasmWasi?

With

kotlinx-io

is there any easy way to read a file from the

resources

directory?

does kotlinx-io allow to manipulate byte buffer outside JVM heap ? for example to send it to a C library ?

I have an issue using kotlinx.io in a kotlin mpp. The situation is that I use FileKit (https://github.com/vinceglb/FileKit) to select files on the different platforms (android, ios, jvm, etc.). FileKit returns a PlatformFile which can return a PlatformInputStream. This stream has a

readInto

method which is a suspend method (which makes sense with io operations). I want convert this FileKit specific PlatformInputStream into a generic kotlinx.io (Raw)Source to further use in my application. However, the kotlinx.io RawSource has a method, override fun

readAtMostTo

, which I need to implement for this, but this method is not a suspend function. Because this is a MPP and both FileKit and kotlinx.io are also MPP I think I should be able to make this work in common code, but I fail because the readInto method is not a suspend function but I need a coroutine context to call the readAtMostTo and wait for the result to return. In jvm I would be able to use runBlocking to call and wait for the result, but this is not possible in MPP common code. I really thought about different solutions but I cannot find a good solution for this. Does anybody have an idea? For simplicity here is some pseudo code showing what I try to achieve:

class PlatformInputStreamSource(
    private val platformInputStream: PlatformInputStream
) : RawSource {
    override fun readAtMostTo(sink: Buffer, byteCount: Long): Long {
        ...
        
        << Here i need to start a coroutine context and wait for the numberOfBytesCopied
        val numberOfBytesCopied = platformInputStream.readInto(buffer, byteCount, bufferSize))
        
        ...
        
        << This non suspend function needs to return the number of bytes copied and therefor has to wait for the result from the readInto function.
        return numberOfBytesCopied.toLong()
    }
}

Hi, apart from file I/O, what are the future plans for kotlinx-io APIs? Can we expect some kind of Process (like ProcessHandle) or Signal Handler API to be added to kotlinx-io that will work on all native platforms?

Is this expected that calling

myRawSource.buffered().buffered().buffered()

creates a new RealSource each time? Shouldn't be there an extension on

Source.buffered() = this

?

On JVM only, I need to create an OutputStream to write 10 MB (data comes from memory, json encoding) over the wire. Should I use ByteArrayOutputStream or kotlinx Buffer and asOutputSource? Are there any performance differences?

Haven’t been able to find a good example of using kotlinx.io (or okio) to parse data. As a trivial/contrived example (real world scenarious have more layers/complexity), if you have a stream of records that are line-delimited: a) you need to demarcate a record - find the end of line; b) you need to parse within the line, where various pieces may or may not be required, may be further transformed, etc Yes, you could use readLine() - but that incurs copies into Strings that need to be further parsed. How to properly do the framing / parsing?

I have a kotlinx io Source. I would like to read chunks of size N and only get a smaller chunk if it’s the end of the file. readAtMostTo looks like what I want, but it’s not guaranteed to read the full chunk size

Looking to migrate (with KMP in mind) from using java.io.* in my Android app to using either kotlinx.io or okio. I noticed kotlinx.io doesn’t support much in the way of system file operations (like delete). However, with ktor recently switching from okio to kotlinx.io should I consider okio as a no-go and instead place my bets on kotlinx.io with anticipation of upcoming expanded functionality. I’m not in any huge rush.

What would be the right way to copy a whole file with kotlinx.io? I wrote this but I'm really not sure it's OK. What does this do with large files?

fun Path.copy(target: Path, override: Boolean = false) {
    check(target.exists() && ! override) { "file $target already exists" }
    SystemFileSystem.sink(target, append = false).buffered().use { sink ->
        SystemFileSystem.source(this).buffered().use { source ->
            source.transferTo(sink)
        }
    }
}

First time using kotlinx.io today and I was a bit thrown that

kotlinx.io.asSource

returns a

RawSource

, which is not a

Source

. Have you considered renaming it to

kotlinx.io.asRawSource

, and making

kotlinx.io.asSource

syntactic sugar for

.asRawSource().buffered()

?

When writing files I sometimes like to write into a temporary file first and then rename it to ensure that the file content is always valid. However, as I understand

atomicMove

in

<http://kotlinx.io|kotlinx.io>

does not support overwriting existing files. Any advices how to achieve this? The problem with

delete

+

atomicMove

is that there exists a moment where the original file is deleted but the move did not happen yet.

anybody know if there's updates on the Async proposal? trying to implement OPFS storage for web but everything is async and

Source

is all sync

How do I append a line into a file (kotlinx-io Path)

What's the best way to read a local file with random access? Use-case, I have a parquet file that contains the metadata at the end of the file... So I need to read the last 8 bytes to get the start offset of the metadata footer, and then read the file starting from that offset to parse the metadata. I currently have a RawSource, so to "start" reading at a specific offset, I want to use

rawSource.readAtMostTo()

, but this api requires a

Buffer

, not a

RawSink

that can be ignored by an

discardingSink

. (I can use

discardingSink().buffered().buffer

though, but it is an internal api.)

Hi, how to I get a file modification time or timestamp with kotlin native on linux

Why does ByteString need an unused

dummy: Any?

parameter which is always null? https://github.com/Kotlin/kotlinx-io/blob/master/bytestring/common/src/ByteString.kt#L81

And do you have any plans to support "stream" converters like Base64, or compression like gzip? And what about in memory zip streams (and zip file systems at all)? I currently use the Java api but would like to migrate my code to Kotlin Multiplatform.

What does

FileSystem

being

experimental

mean in practise say for ios build? Is it runtime safe for prod? or is it more of api stability issue

Is there any special attention that should be paid to loading large files into the browser with

Buffer

? (200MB+)

kmp-file

0.3.0

is out with support for Android Native, including a proper system temp directory for Android API 31 and below 🎉

Where are API changes proposed for KotlinX IO? Is there a roadmap somewhere? I want to know what parts of the API are considered more stable and which ones aren't, and what major plans exist to overhaul any of the systems.

I use kotlinx.io and after writing a file I call flush(). However, if the computer the code runs on switches off without proper shutdown, the file contains zeroes after the reboot. According to JVM OutputStream docs flush() guarantees that the data is passed to the operating system but it doesn't guarantee that it is actually written out. Are there any good solutions for this apart implementing my own functions to write files?

On this note, it would worth to mention this behavior in the docs, so it is clear what to expect.

It would be nice if KotlinX IO had something like this so I don't have to pass FileSystem and Path around together everywhere.

And, when will we define some specified FileSystem exception types? It's important to who want to know what's happened in the underlying OS. And some internal operation also need it. Some methods like

deleteIfExists

shouldn't be implemented as check existent and then do deletion. Try to delete it and catch the

NoSuchFileException

is a better choice. Even the FileSystemException tree definition designed by the Java is not clearly enough(At least to me that's true.), a new design is better than no. I create an issue on the GitHub, shall we talk it on the GitHub? I saw currently there's on place to discuss the API design. https://github.com/Kotlin/kotlinx-io/issues/472 At least there's no link to the slack channel so I don't know we have a heated discussion here.

Is there a straightforward way to read all bytes from a source in some reasonable chunks, ideally in a way that can handle the consumer suspending? My particular need is to send it to

MessageDigest

but it seems like a pretty straightforward and common operation

My own WindowsPath.kt. Windows has many special cases, MinGW POSIX API is not so useful, so it's better use Win32 API directly. https://gist.github.com/iseki0/eb4495e72d65fd10c0dce408241d0fae

I don't understand, why you don't care the memory pressure, and what will happened once the

byteCount

large than

Int.MAX_VALUE

? The code from Kotlinx.io The second important, how to do zero-copy? Of course I can't access the internal data structure of the Buffer.

override fun write(
        source: Buffer,
        byteCount: Long
    ) {
        require(byteCount >= 0L) { "byteCount: $byteCount" }
        require(source.size >= byteCount) { "source.size=${source.size} < byteCount=$byteCount" }
        check(!closed) { "closed" }

        val allContent = source.readByteArray(byteCount.toInt())
        // Copy bytes from that segment into the file.
        val bytesWritten = allContent.usePinned { pinned ->
            variantFwrite(pinned.addressOf(0), byteCount.toUInt(), file).toLong()
        }
        if (bytesWritten < byteCount) {
            throwIOExceptionForErrno("file write")
        }
    }