https://flyte.org logo
Join Slack
Powered by
# new-ray-debug
  • p

    polite-ability-4005

    04/24/2024, 7:02 PM
    Hi
  • p

    polite-ability-4005

    04/24/2024, 7:02 PM
    Let’s discuss ray issue here
    👍 1
  • p

    polite-ability-4005

    04/24/2024, 7:02 PM
    Scheduling a call might be the most efficient way?
  • p

    polite-ability-4005

    04/24/2024, 7:02 PM
    I am PST timezone
  • h

    helpful-lighter-24518

    04/26/2024, 9:31 AM
    Hi everyone! After trying out different versions, with Kuberay 1.0.0 and Flyte-binary 1.11.0 (versions suggested by @crooked-zebra-68589) now we can launch Ray tasks successfully. But now we have another problem: kuberay-operator is not able to stop the RayJobs after they have finished (meaning head and worker nodes stay running forever). These are the logs we get from kuberay-operator:
    Copy code
    2024-04-26T08:35:02.249Z        INFO    controllers.RayJob      Remove the finalizer no matter StopJob() succeeds or not.       {"finalizer": "<http://ray.io/rayjob-finalizer|ray.io/rayjob-finalizer>"}
    2024-04-26T08:35:03.937Z        INFO    controllers.RayJob      reconciling RayJob      {"NamespacedName": "flytesnacks-development/angw645vlpnv6pts2tjk-n0-0"}
    2024-04-26T08:35:03.937Z        INFO    controllers.RayJob      RayJob is being deleted {"DeletionTimestamp": "2024-04-23 09:23:16 +0000 UTC"}
    2024-04-26T08:35:03.937Z        INFO    controllers.RayJob      Stop a ray job  {"rayJob": "angw645vlpnv6pts2tjk-n0-0-sbjsb"}
    2024-04-26T08:35:03.937Z        INFO    controllers.RayJob      Failed to stop job for RayJob   {"error": "Post \"http:///api/jobs/angw645vlpnv6pts2tjk-n0-0-sbjsb/stop\": http: no Host in request URL"}
    rayjob.txt
    g
    p
    f
    • 4
    • 9
  • a

    average-finland-92144

    02/27/2025, 6:37 PM
    archived the channel