Hi everyone! Hope it'll be a fun project 😁

if (declaration is IrConstructor && backendContext.inlineClassesUtils.isClassInlineLike(declaration.parentAsClass))
	return

val isIntrinsic = declaration.hasWasmNoOpCastAnnotation() || declaration.getWasmOpAnnotation() != null
if (isIntrinsic) {
	return
}

val wasmImportModule = declaration.getWasmImportDescriptor()
val jsCode = declaration.getJsFunAnnotation()
val importedName = when {
	wasmImportModule != null -> {
		check(declaration.isExternal) { "Non-external fun with @WasmImport ${declaration.fqNameWhenAvailable}"}
		context.addJsModuleImport(wasmImportModule.moduleName)
		wasmImportModule
	}
	jsCode != null -> {
		// check(declaration.isExternal) { "Non-external fun with @JsFun ${declaration.fqNameWhenAvailable}"}
		val jsCodeName = jsCodeName(declaration)
		context.addJsFun(jsCodeName, jsCode)
		WasmImportDescriptor("js_code", jsCodeName)
	}
	else -> {
		null
	}
}

if (declaration.isFakeOverride)
	return

In this snippet from

DeclarationGenerator.kt::DeclarationGenerator::visitFunction

, is there a reason for moving the check for fake overrides specifically after the check for imports, or is it just a coincidence? I believe fake overrides aren't possible to get imported anyways, but the location of the condition makes me think that it's put on purpose so that one of the calls to the

context

happen before returning.

Also, is it the case that a single ITable type is defined for each disjoint set of interfaces, and for each class that implements any of the interfaces, an ITable representing the whole disjoint set is allocated, with the slots of interfaces it doesn't implement set to null?

It seems like I sent the messages during a weekend. My bad! 😅 I would appreciate telling me the usual working hours, so that I get a better idea of when you’re typically at work. Of course, I understand that doesn’t mean you’re free all the time during those hours. 🙂

This is BTW not the case in Egypt, in which weekend is Friday and Saturday. It slipped my mind that today might be a weekend in Netherlands.

Inspired by the discussion about itables, I wonder, is flattening all sub-tables of the itable into the same table a bad idea? In this case, the itable representing the whole disjoint union will have all function references directly, instead of having to reference the struct of a particular interface first. In this case, instead of storing the

slot

of an interface, we'll store the

offset

that the functions of a given interface start at. This way, we save one level of indirection. The downside is that the size of the itable will be bigger, and thus allocating bigger itables for each class type, because instead of having a single null reference for interfaces that the class doesn't implement, we'll have a null reference for every function in every interface the class doesn't implement. I checked out if Wasm provide a way to compress contiguous bytes with the same value, thus not having a big binary size, but I couldn't find any. Not sure if the tradeoff is worth it.

In

DeclarationGenerator::visitField

:

override fun visitField(declaration: IrField) {
	// Member fields are generated as part of struct type
	if (!declaration.isStatic) return
	
	...
	
	val initValue: IrExpression? = declaration.initializer?.expression
	if (initValue != null) {
		check(initValue is IrConst<*> && initValue.kind !is IrConstKind.String) {
			"Static field initializer should be string or const"
		}
		...
	} else { ... }
	...
}

Why is the string type excluded from being an initializer to a static field?

In

WasmCompiledModuleFragment::linkWasmCompiledFragments

:

fun wasmTypeDeclarationOrderKey(declaration: WasmTypeDeclaration): Int {
    return when (declaration) {
        is WasmArrayDeclaration -> 0
        is WasmFunctionType -> 0
        is WasmStructDeclaration ->
            // Subtype depth
            declaration.superType?.let { wasmTypeDeclarationOrderKey(it.owner) + 1 } ?: 0
    }
}

val recGroupTypes = mutableListOf<WasmTypeDeclaration>()
recGroupTypes.addAll(vTableGcTypes.elements)
recGroupTypes.addAll(this.gcTypes.elements)
recGroupTypes.addAll(classITableGcType.elements.distinct())
recGroupTypes.sortBy(::wasmTypeDeclarationOrderKey)

...

val allFunctionTypes = canonicalFunctionTypes.values.toList() + tagFuncType + masterInitFunctionType

// Partition out function types that can't be recursive,
// we don't need to put them into a rec group
// so that they can be matched with function types from other Wasm modules.
val (potentiallyRecursiveFunctionTypes, nonRecursiveFunctionTypes) =
    allFunctionTypes.partition { it.referencesTypeDeclarations() }
recGroupTypes.addAll(potentiallyRecursiveFunctionTypes)

Why add the function types at the end, letting them pass the sorting function? It's a bit confusing that there is a case specifically for functions in

wasmTypeDeclarationOrderKey

, but not used since there exists no function type in the list at the time of its usage, and then a list functions is appended at the end, contradicting its logic which puts functions and arrays at the beginning of the list in a stable order.

Update: I'm now familiar with the structure of Wasm-related code. I focused the most on

ir2wasm/{DeclarationGenerator.kt, WasmCompiledModuleFragment.kt}

, going through them line-by-line. I also went through the rest of the files related to code generation such as

WasmModuleCodegenContext.kt

and

WasmModuleFragmentGenerator.kt

, stuff in

utils

,

wasmCompiler.kt

, and stuff in

<http://wasm.ir|wasm.ir>

(except convertors). I didn't go through DCE-related stuff and

BodyGenerator.kt

because they're probably irrelevant at the moment. I'm slightly familiar with IrElements, but I'm not aware of all the mappings between Kotlin constructs and the corresponding IrElements yet. I'm going through the list of lowerings one by one, which will take some time since there are almost 100 of them. Any comments?

This is the initial design of the serializer for Wasm structs. I assume that the serialized files are used only as temporary files, and thus no need to account for backward compatibility, right?

As for

IdSignature

and its decedents, there is already a serializer (

compiler/ir/serialization.common/src/org/jetbrains/kotlin/backend/common/serialization/IdSignatureSerializer.kt

) and a deserializer (

compiler/ir/serialization.common/src/org/jetbrains/kotlin/backend/common/serialization/IdSignatureDeserializer.kt

). I don't think we need to do further work there. 🤔

I suggest serializing Wasm structs using Protobuf. It's already being used in the codebase, and no extra dependency is needed. Plus, it's what is used for serializing

IrSignature

structures. Using Protobuf will achieve consistency with the rest of the code base, compactness, and quick serialization/deserialization. What do you think?

Hi, Osama, I made special branch can now use for very first experimental test of your serialization/deserialization. https://github.com/JetBrains/kotlin/tree/yakovlev/incremental_base You may to try serialize and deserialize

org.jetbrains.kotlin.backend.wasm.ir2wasm.WasmCompiledFileFragment

instances which is created here:

org/jetbrains/kotlin/backend/wasm/wasmCompiler.kt:158

You can try to run

org.jetbrains.kotlin.wasm.test.FirWasmWasiCodegenBoxTestGenerated.testSimpleWasi

which is pass now.

Hi Osama, I have updated the branch

yakovlev/incremental_base

and now most of the tests passed (like tests in

FirWasmJsCodegenBoxTestGenerated

). The

WasmCompiledFileFragment

slightly changed.

I'm done with serializing and deserializing

WasmCompiledFileFragment

but without deserializing same symbols to the same instance. This is what I'm going to do next. The only problem here is the presence of

FileSignature

. This is my test:

fun linkWasmCompiledFragments(): WasmModule {

    wasmCompiledFileFragments.forEach {
        try {
            val outputStream = ByteArrayOutputStream()
            WasmSerializer(outputStream).serialize(it)
            WasmDeserializer(ByteArrayInputStream(outputStream.toByteArray())).deserializeWasmCompiledFileFragment()
        } catch (e: FileSignatureEncountered) {
            println("Oops...")
        }
    }

    bindFileFragments(...

Output:

Oops...
Oops...
Oops...
Oops...
Oops...
Oops...
...

As a side note, I changed the members of

WasmCompiledFileFragment

to

var

so that I can deserialize and assign each one easily. Do you have a problem with that?

I thought I was almost done, but the deserialized

WasmCompiledFileFragment

was still not substitutable in place of the original one. After some debugging, I found that this is because `WasmNamedModuleField`s were deserialized by value, and not by reference like `WasmSymbol`s. I'm now changing what's needed to be serialized by reference instead of by value. BTW, here is where I push the code: https://github.com/OsamaAhmad00/kotlin/tree/serialize-file-fragment/compiler/ir/backend.wasm/src/org/jetbrains/kotlin/backend/wasm/serialization hopefully I get the serializer done soon, merge it to the IC branch, then move to the next step

The serializer is almost working. https://github.com/JetBrains/kotlin/pull/5301 I ran all tests on serialized then deserialized

WasmCompiledFileFragment

, and 15291 out of 15685 tests have passed (397 failures). This is more than the original branch (which has 197 failing test) by 200 failing tests. The additional failures are because of this line in the deserializer:

8 -> IdSignature.FileSignature(0, FqName(""), "")

where a "default"

FileSignature

is returned. When I ignore serializing the ones which contains a

FileSignature

, I get the same number of failing tests as the original branch. I'm going to make use of the following days in understanding the changes introduced in this branch, as I had no chance of actually reading the code up until now, and also investigate a bit more for how to serialize `FileSignature`s.