Hey, could I please be added to a group that has write/merge access to the flyteorg/flytekit-java repo?

hi there! I'm trying to fill some gaps in the java version of flytekit. I'd like to update the local-executor so more test can be written. I've opened this PR to address sub-workflows. It would be nice if someone can have a look and maybe even try it. If this is going the right way, I think mocking sub-workflow in tests shouldn't be too hard

I’m currently writing a few example workflows in Scala to get a better understanding of flytekit-java/scala. I’ve only created a few simple workflows for now but it looks quite solid. After some fiddling and realizing that we already push flytekit-* to maven central, I managed to create a standalone Scala/SBT project and sucessfully register and run workflows locally as well as in a freshly created sandbox cluster. The biggest hurdle right now clearly seems to be a lack of documentation. There’s the readme, a few examples in the flytekit-java repo as well as some community sync presentations by @red-boots-99987. But you have to do some digging... I think adding more documentation here would have great potential to expand the user-base of Flyte to JVM folks and could also bring in new contributors. Here’s what I think could be most helpful: • Getting started docs similar to the Python one • Basic documented examples integrated in the docs (aka flytesnacks) • Standalone starter projects or templates like https://github.com/flyteorg/flytekit-python-template but for Java/Scala using common JVM build tools (Maven/Gradle/sbt...) I’m thinking about working on one of these but I’d like to sync first with folks here.

@boundless-pizza-95864 @better-air-29727 @boundless-twilight-37555 How do we go about updating flyteidl version (and generating grpc client) flytekit-java?

Hey, does anyone know if jflyte supports authentication? I'm trying to register a workflow against a flyteadmin instance secured with Google OpenID Connect, but I'm getting auth errors:

io.grpc.StatusRuntimeException: UNAUTHENTICATED: token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken

There's an

--auth-mode

parameter in jflyte but I have no idea what to set here. Application default credentials are set via

gcloud auth application-default login

Jar staging/upload works fine but flyteadmin auth then fails.

Does the docker build/push for flytekit-java happen in CI? It seems we’re missing the image for 0.3.16. Perhaps due to moving to jib?

unarchived the channel

@bumpy-match-83743 @brief-window-55364 et al. Feel free to add anyone else who might help here

👋

🦜 🙏

I'm having trouble registering a workflow. I created a separate Scala sbt project and copied the simple Fibonacci workflow. It complies just fine. Then I try to use

jflyte

to register.

I'm trying to import the

FlyteAdminClient

into a Scala project, but it doesn't seem to live in the

import org.flyte.jflyte.FlyteAdminClient

as per the examples. I'm likely missing a dependency somewhere?

So I've abandoned the

flytkit-java

SDK as it really doesn't provide much in terms of the FlyteAdmin service. I am wondering if there is a way to co-mingle the Scala type/binding support of the SDK with my gRPC interface. I'd have to provide some kind of shim there. Thoughts?

@brief-window-55364 shall we drop the flytekit-java support in the docs. there are anyways no docs, examples, its not easy to get running etc?

Once I get things running on my end-to-end, I intend to contribute some docs around Java/Scala support via gRPC. I don’t yet know if it’ll be a hybrid of the current SDK or not.

I'm working on a hybrid approach to define and tasks, workflows & launch plans via the

org.flyte.flytekitscala

SDK and register via

jflyte

tooling. That works well enough, be we know there are significant gaps when it comes to Flyte Admin tooling on the JVM. To mitigate, I took the protos from the Flight IDL, generated an API, and have the full scope of the Flyte service, specifically the admin service. I am now able to launch workflows via code, get execution status, etc. The next piece of the puzzle is to come up with a strategy to bind task/workflow inputs/outputs, so that I can dynamically: 1. Launch workflow executions with dynamic inputs, and, 2. Retrieve type-safe workflow execution output values. I've put together . While the example shows primitives, our use case is likely going to rely on JSON payloads throughout (for now), so that may significantly simplify the strategy here. It'd be great if I could get some thoughts, maybe from the Spotify folks who built the SDK, or anyone else who is trying to adopt the platform from the JVM.

How would I go about write a map task using

flytekit-java

?

Here's a video that builds on the last one, that shows the unfortunate challenges of using the generic/struct type to represent JSON payloads in

flytekig-java

.

Is this possible?

Can a task read/write from/to a database?

Does

flytekit-java

include the FlyteFile abstraction?

@dazzling-spring-85404 has left the channel