https://flyte.org logo
Join Slack
Powered by
# flyte-bazel
  • e

    elegant-australia-91422

    01/29/2023, 6:04 PM
    set the channel description: Channel for discussing Flyte & Bazel integration
  • e

    elegant-australia-91422

    01/29/2023, 6:04 PM
    Just set this up to discuss Bazel <> Flyte integration & any particulars that come up during development, as well as any work to get
    rules_flyte
    back up to date/upstream some of our own work
    ➕ 2
    🔥 3
  • c

    crooked-artist-67935

    01/29/2023, 6:16 PM
    I'd be happy to contribute to
    rules_flyte
    . The implementation I put together this week is fairly encapsulated from our stack simply because its so new -- though that is a double edged sword
  • e

    elegant-australia-91422

    01/29/2023, 6:19 PM
    The only specifics we use are our internal py_binary/py_library targets. Historically we added linters as sidecar targets to that, though we’ve recently moved those to aspects. So we might be able to ditch the custom targets soon
    👍 1
    🔥 1
  • e

    elegant-australia-91422

    01/29/2023, 6:19 PM
    Other than that we’re close to the same state
  • s

    sticky-art-97180

    02/01/2023, 7:24 PM
    Joining late to the party, but we (Woven by Toyota/ex-level5) have also open sourced some parts of our monorepo.. Happy to conttibute/upstream our work as well.
    🔥 4
  • b

    brash-london-45337

    02/22/2023, 1:07 AM
    Would love to contribute as well. We based much of our integration at stripe on the bazel rules but will need some time to remove the stripe specific components
    💯 2
    🔥 3
    🙇 2
  • f

    freezing-airport-6809

    02/22/2023, 4:11 AM
    Cc @crooked-artist-67935 / @elegant-australia-91422
  • e

    elegant-australia-91422

    02/23/2023, 2:59 PM
    One thing that'd be awesome to share is how each of our implementations is handling serialization, registration & execution. Do you all have separate targets for each, or are any of the steps handled together? In our case, we handle both serialization & registration in a single
    .register
    target, and handle execution w/ a
    py_binary
    target that takes in the launch plan name & user-specified CLI args that are passed to
    flyte_remote.execute
    inside that binary. One interesting concern is that it requires the user to remember to register prior to attempting to launch an execution, and this is an area we'd really like to clean up and make more seamless. Curious if anyone else has different solutions for this?
    g
    • 2
    • 1
  • f

    freezing-airport-6809

    03/17/2023, 2:33 PM
    Folks remember Flyte slack is free. Good thing would be to put things in a doc
  • g

    gifted-train-81198

    03/20/2023, 3:55 PM
    Hi Guys, I am using bazel to build the flyte target but when I use bazel output binaries to serialize and register the workflows it lead to ModuleNotFoundError: No module named 'pyflyte' at the time of task execution. The error occurred because the value of
    resolver
    in task container argument changed from usual
    flytekit.core.python_auto_container.default_task_resolver
    to
    pyflyte.pypi_flytekit.site-packages.flytekit.core.python_auto_container.default_task_resolver
    . Did anyone face this issue ? is there a way to control this?
    t
    e
    • 3
    • 4
  • e

    elegant-australia-91422

    03/24/2023, 2:41 PM
    @crooked-artist-67935 also we figured out how to avoid the container_image step; since the entrypoint in our container delegates to the appropriate
    pyflyte-*
    command, just using the py3_image as-is worked (w/ the way that Flyte sets
    args
    for the pods)
  • e

    elegant-australia-91422

    03/24/2023, 2:42 PM
    This shaved ~45-60s off build time, which adds up over time
    🎯 2
  • f

    freezing-airport-6809

    04/18/2023, 4:32 PM
    cc @many-wire-75890
    👀 1
  • f

    freezing-airport-6809

    04/18/2023, 4:32 PM
    Folks @gifted-train-81198 / @elegant-australia-91422 / @crooked-artist-67935 can we get an example repo of Flyte with Bazel?
  • f

    freezing-airport-6809

    04/18/2023, 4:32 PM
    there are a lot of users who keep on asking
  • m

    many-wire-75890

    04/18/2023, 8:09 PM
    Yes, if there's any example Bazel build graphs that you guys are able to share, it would be greatly appreciated 🙇
  • v

    victorious-controller-47160

    05/22/2024, 3:05 PM
    👋 I wanted to see if I could bump https://github.com/flyteorg/bazelrules_flyte/pull/6 We’re looking to integrate Flyte with Bazel and this ruleset looks like it is a decent starting point but unfortunately doesn’t have a LICENSE file configured.
    a
    • 2
    • 1