Are there any plans regarding mixed kotlin->python and kotlin->python C API projects?

happy new year! some cleanup during Christmas break 😄 • [#94] Remove unreferenced code • [#103] Clean up CLI My plan was also to move cleanup in lowerings, but it's not that easy to find unneeded ones. Some work in progress in PR #94, not yet ready to review. Instead, some eye-candy: auto-generated graph of dependencies between lowerings:

I once mentioned the problem of duplication in GitHub Actions workflows (YAML doesn't let us do much about it, maybe apart from extracting simple stuff to env vars). Well, I decided to tackle it with a Kotlin DSL which is now published to Maven Central, and I propose to adopt it in kotlin-python: https://github.com/krzema12/kotlin-python/pull/106

Trying to set up kotlin-python on a new computer but having the following error:

> Task :compiler:resolution:compileKotlin
e: error while writing /home/servb/IdeaProjects/kotlin-python/compiler/resolution/build/classes/kotlin/main/org/jetbrains/kotlin/resolve/calls/components/CollectionTypeVariableUsagesInfo$isContainedInInvariantOrContravariantPositionsWithDependencies$isContainedAnyDependentTypeInReturnType$1$1.class (Permission denied)

Any ideas what the problem is? Probably too long path but then I'm wondering why it's worked before...

@Piotr Krzemiński, could you please clarify the status of https://github.com/krzema12/kotlin-python/pull/97? I'm going to continue researching the codebase and removing unneeded stuff. Should I make my branch out of yours?

Today: continued researching the codebase, removed and renamed some stuff (PR). Next time: want to continue for one or two weeks, this research still looks useful to me.

@Piotr Krzemiński, do you have any news regarding stdlib? I want to research here too but not sure where to start if not from scratch

Today: • Looked inside the stdlib branch but after a few tryouts decided to postpone it: not much clear was there. • Continued with researching and cleaning our codebase (PR). Looks like now I finally understand our current testing workflow pretty good (for each test: take the test file, split it to multiple files in a temp dir but looks like most tests have a single file after split too, compile them including libraries to py in our testData dir, run it and check the result). Next time: • More researching, starting from the CLI tool. Also would be good to understand how libraries are taken into the compilation.

Hey Piotr, I have some thoughts regarding Python AST code that we now have. It's auto-generated now. However, using that auto-generated code is quite wordy because all the parameters need to be specified and also some names are long. Don't you mind if I create wrappers for some of that stuff, in a new package (not inside

generated

)? Also, would be cool to have constants for

True

and so on. Maybe in the future we can even extract it as a Python AST library for JVM and Kotlin Multiplatform! By the way, I don't want to get rid of auto-generated code: it seems useful because it's actually a base that ensures that we have every AST element covered correctly

Hi Sergei, makes sense to me :) this generator was written pretty quickly, I didn't put too much thought into it, so any optimization opportunities you see are appreciated

Today: I wanted to go with something self-contained as a warmup after the pause: https://github.com/krzema12/kotlin-python/pull/113 - hopefully it will help allow creating simple CLI tools soon

Today: • More cleaning (PR). • Still don't understand compilation of stdlib. I see on the IR->PYAST stage, there are many stdlib files, but it's better to understand what's going on before this stage. • To do something more handy, found that function references are unsupported at all. Added a couple of lines and +36 box tests pass (PR). Next time: • Continuing research of how stdlib files are processed. • Will also research something else, for example, unsigned numbers. Probably they can be implemented via intrinsics like we do with signed numbers, without touching stdlib.

Asking to vote for https://youtrack.jetbrains.com/issue/IDEA-169282 because now it's inconvenient to track elapsed time when running box tests: now it sums all the threads time together, so it doesn't show the time from start but a much bigger number

Today: • Played with a new remote server that JB temporary gave to us (many thanks), it's fast. However, Gateway is difficult to use in our workflow. The current best way for me is to code and debug locally, but when need to run box tests fast, checkout the changes on the server and run them there. • Read a bit about GH self-hosted runner, looked not simple, postponed. • Inspected sources regarding usage of stdlib in compilation. Still don't find the exact place where jar is taken and translated to IR, but more is clear now. Also, made some minor changes on the way, but they aren't worth a new PR yet -- will shape them later. Next time: • Dive one more time into GH self-hosted runner topic, probably set it up. • Continue research around the stdlib.

Piotr, could you please write some stats of how much time it takes to run box tests on the server when you play with it? From my measurements, it's around 6 minutes now, but maybe you have another result 😂 Our admins suggested to also try with disabled HyperThreading and compare, maybe it will be faster (they will disable HT after I ask them)

Today: • I added missing tests for recently added support for

main

function: [#74] Add end-to-end tests for main function, and actually I was surprised that the

external

mechanism works as expected. It means we can use Python's standard library this way 🎉 For example, for such basic things as printing out, which still doesn't work through the Kotlin's stdlib • I really want Kotlin's

println

to work, and I decided I'll focus on it now. Here's are my work-in-progress notes: https://github.com/krzema12/kotlin-python/wiki/Notes-on-making-%60println%60-work I found some concrete missing stuff and things from JS stdlib that we don't need, and the bottom line is that I have a concrete plan for several first steps

@Piotr Krzemiński, could you please give me access to the settings of the repo, so I can experiment with self-hosted runners right there? For now, I don't have access to pages like https://github.com/krzema12/kotlin-python/settings/actions/runners/new

@Piotr Krzemiński, what do you think about removing old branches (like from 2020 and older) from our fork? For now we have around 3K branches and it makes them harder to use

Today: • Removed around 3K unneeded and old branches from the repo, only about 10-20 are left. Probably some of them are unneeded too, but it's not easy to decide. • Some progress with stdlib (PR). Now testing uses newly copied stdlib-py. I've checked, indeed when stdlib-py is edited, we have the changes applied to the files compiled under testing. However, compilation of a separate jar isn't implemented yet, so these changes can't be used with kotlinc. Next time: • Want to support jar too. After it, I believe we can close the ticket about stdlib-py.

Today (PR: https://github.com/krzema12/kotlin-python/pull/123): • found out that do-while is implemented with a bug that made it practically nonworking, partially fixed it (+19 box tests pass). However, it can improved further: https://github.com/krzema12/kotlin-python/issues/122; • supported same-named local variables (+12); • supported more reinterpret casts (+47). Bumped into those problems while researching what can be adjusted in the stdlib (started with "name 'Array' is not defined"). And was able to fix those without changes to the stdlib. Next time: I think I will delay changes to the stdlib because now we can apply it only to tests. Current three topics are worth further research, so will continue with them.

By the way, I have a fix for

name 'Array' is not defined

in the local shelf, it's quite simple. Should I push it before we support kotlinc with stdlib-py?

while working on copying Python's stdlib to

dist

and using it everywhere where necessary (PR #124), I concluded the next step about Python-specific stdlib would be to get independent from the JS stdlib. Currently some code is copied from it before the Python one is built, like in case of JS-IR, which is expected because we copied JS-IR's build logic. E.g. libraries/stdlib/js/src/kotlin/console.kt is just in the JS stdlib and now both JS-IR and Python stdlib copy it over to their source code before building. As a concrete example, it means that without getting independent from JS stdlib we won't be able to e.g. adjust the implementation of Kotlin's `print`/`println`

Today I made some changes to the stdlib (PR: https://github.com/krzema12/kotlin-python/pull/125): • Pushed a fix for "name 'Array' is not defined" (+29 box tests pass); • Supported integer multiplication (+23). Next time: research those three things that I wanted to do the previous time (do-while, variables namer, casts). Before it, I want to take a look at global variables because today I noticed in some tests their initializer doesn't work (see kt1291 test, for example).

Do you know why the box tests need the "minimal-for-test" variant of the stdlib? https://github.com/krzema12/kotlin-python/blob/ab198949a5378bbdd62c905ac30e2b044f6f8d17/python/box.tests/build.gradle.kts#L72 I'm asking because when decoupling Python's stdlib from JS's I'm having some trouble with the minimal stdlib. Would be cool to understand and document the purpose of it 🙂

Wow, after some adjustments to global variables support, the following code now compiles and works successfully! Beautiful 😂

Today: • Supported static fields initialization (PR: https://github.com/krzema12/kotlin-python/pull/128) – +102 passed, +4 failed because no support for unsigned numbers, chars, property delegates, any-to-string conversions. • Prettified output a bit by removing some Unit_getInstance calls (PR: https://github.com/krzema12/kotlin-python/pull/129). • Started supporting do-while completely. Looks like a lowering is needed but I haven't created any yet, so probably it will be my first. In progress. Next time (likely in two weeks because we have a public holiday next Friday): further progress with do-while; if have time: two remaining topics (variables namer, casts).

Had some time today to continue with do-while, implemented a lowering, but it gave only +5 passing tests. Turned out that other lowerings create their own do-while loops and I guess my lowering doesn't process them, even if I declare some of the lowerings as prerequisites. Probably I don't specify all the prereqs, will try to understand it the next time (but I don't now when it happens). Anyway, created a draft PR. The two remaining topics mentioned previously are still to be researched.

Hi folks, I discover this fork today and it is really really awesome, congrats to all working on the fork, this give me hope that in the future we could see more target being available with good tooling and interop, for example Lua and CLR would be awesome. I was wondering how better the new Compilers in their internal API by comparing to the old one when thinking in add a new target for Kotlin?

if you want to develop an application that when input images and videos, all similar images and videos are output, how would you propose to go about it?