Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Had to share before I forgot but would it be possible to make the “View Logs” thingy resizable? I’d like to be able to read the values of the different keys on 1 line if possible.

These are task details right? We are working on the correct design of task details with its own page. <@U01J8L5AR2S>
Cc <@U0231BEP02E> . 

Yeah it’s what you get when the “View logs” and that is for the details of the task in particular indeed

Yeah hey <@U01J8L5AR2S> - as <@UNZB4NW3S> mentioned we have comps for a new task detail page that will provide a larger canvas for displaying all these data; will definitely be useful when viewing nested data like that. We're hoping to pick it up in the new couple weeks - stay tuned! :sunglasses:

Hi, From the UI, I see we currently only fetch 10 launch-plans for a given wf . We have a few users who have &gt; 10 launch-plans and don't have any way of fetching/triggering those via the UI.

Believe <https://github.com/flyteorg/flyteconsole/blob/master/src/components/Launch/LaunchForm/useLaunchWorkflowFormState.ts#L58> is the code

Guessing this should be easy to bump up . Any concerns if I make a quick PR and set this to say 25 or 50 ?

<@UP2HQ5K7F> Feel free to bump it to 25-30

we will add work for adding “Load more” button in case users will hit limit in future.

<@UP2HQ5K7F> can you please run on your branch locally:
```git rebase HEAD~1 --signoff```
and then push force it again to the brunch. It will add your signature to the PR. I will sign off and merge afterwards

yup, just did . I quickly checked and we have some wfs with ~30-35 launch-plans at Lyft so bumped it to 50 for now. Don't think this should cause any latency/perf issues.

Should be fine for now, we will cover it separately, thanks :slightly_smiling_face:

Just wanted to say that it’s really nice to see the progress on the UI/UX side for Flyte, really good stuff :sparkles::raised_hands:

<@U01J8L5AR2S> please create more issues and have suggestions

Hey peeps,

Do you think it’d be a great idea to make the code that is used to run tasks available somewhere in the Flyte UI? Airflow has something like that and I find it quite handy when I wanna check what code was used to create a task for example. It would definitely make it easier for me when I need to help Data Scientists :sparkles:

we can use links, but i dont think we can upload code to Flyte?

Links could be a solution as well. But I didn’t mean as becoming git more copy pasting the code that was used.

That is what I use with Airflow when I quickly wanna check what the code looks like and if there is some quick fix that I could do for example

Also, code privacy might become a problem?

Don’t know about code privacy, for now if you have access to Flyte on our side then you also have access to github. But it was more like an idea than anything else, it might not be relevant for Flyte.

I guess you’re not talking about that when you talk about Dochub? <https://dochub.com/>

Aah no, it's a code name. It's one of the issues

But I do like the idea of a link to the GitHub function 

Created an issue <https://github.com/flyteorg/flyte/issues/2608>
Sorry I missed that message

Hey, everyone! Just a heads up: I think 1.x.x broke some things on the UI. When I enter a workflow's page, there's no description, even though there's a docstring defined on it (see first image). Also, if I click "View All" to view all the versions of the workflow, an error happens (see second image).

Also, did you guys remove the support to DataFrames? Now DF inputs/outputs are identified with "unknown" and "This type is not yet supported"

Thanks for reporting this <@U03247826MB> , we are looking into it. 

<@U03247826MB> can you please check which FlyeConsole version do you have (click on `i` icon in the top right corner.) I believe at least one of this two issues is fixed in flyteConsole v.1.1.0

wow, my version is 0.54.1, even though I'm using 1.x.x! how can I update it??

do you have anything on your sandbox you want to save? (prior executions or whatnot)  if you don’t, i’d recommend just killing it and starting a new one

yeah, I'm using the image     <http://cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3|cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3>, and deploying Flyte "locally", i.e. by using FLYTE_TEST=local

this variable is used on the entrypoint: <https://github.com/flyteorg/flyte/blob/master/docker/sandbox/flyte-entrypoint-default.sh>

I changed it because I wanted to increase the maximum amount of memory a task could use. The only way I was able to do this with the original sandbox was to change the `/flyteorg/share/flyte/values.yaml` file and use the local helm repo to start the server

you didn’t want to adjust it afterwards?

just updating the admin config after installing

<https://github.com/flyteorg/flyte/blob/1e3d515550cb338c2edb3919d79c6fa1f0da5a19/charts/flyte-core/values.yaml#L35,L43>

you’re using old sandbox right, not the new one.

you can also customize things on a project/domain basis

<https://docs.flyte.org/en/latest/deployment/cluster_config/general.html#deployment-cluster-config-general>

Cc <@U02MEAKMMU1> can we document this better

Hey <@UNR3C6Y4T> I'm trying to configure the resources on a system-wide level, but I'm confused with that first link you sent me. On it, the memory limit is 500m, but when I try to register tasks, flytectl tells me that the limit is 1Gi. I noticed this value was at .flyte.configmap.task_resource_defaults.task_resources.limits.memory, and that's where I modified it. Shouldn't it be there that I should apply modifications?

And I'm still not sure how to configure the resources. The documentation says "In the absence of an override, the global default values in the FlyteAdmin config are used. The override values from the database are assigned at execution, rather than registration time.", but how can I override values?

Also, even removing the FLYTE_TEST variable and the initialization script, my UI version is still 0.54.1. The command I ran was:
```docker run -d --name flyte-sandbox --privileged --mount type=bind,src=$PWD,dst=/root -p 30080-30082:30080-30082 -p 30084:30084 -p 30086:30086 -p 30088:30088 -p 30089:30089 --entrypoint /bin/sh <http://cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3|cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3> -c 'tini flyte-entrypoint.sh'```
(there were more commands after -c that changed the values for the max memory, but I removed them)

hey <@U03247826MB> you can override can you try the following

`cat &gt; update.yaml`
```defaults:
    cpu: "1"
    memory: 150Mi
limits:
    cpu: "2"
    memory: 450Mi
project: flyteexamples
domain: development```


and then run
```flytectl update task-resource-attribute --attrFile update.yaml```


this is probably better than mucking around with the admin configmaps (since they’re even harder to get to in the `demo` version of the sandbox)

wrt the console version however, you have `kubectl` access right?

can you `kubectl -n flyte edit deploy flyteconsole` and change the image to `<http://ghcr.io/flyteorg/flyteconsole:v1.1.0|ghcr.io/flyteorg/flyteconsole:v1.1.0>`

I did the change, and now it worked. I'm on 1.1.0 now. FYI, the old version was actually flyteconsole-release:1.0.1, and it was on Flyte's repository

like that, I think: <http://cr.flyte.org/flyteorg/flyteconsole-release:1.0.1|cr.flyte.org/flyteorg/flyteconsole-release:1.0.1>

and yeah we try to tag each individual version

and then was also add a flyte release tag

like that?
```bash-5.1# kubectl -n flyte get cm
NAME                               DATA   AGE
kube-root-ca.crt                   1      39m
flyte-contour                      1      39m
datacatalog-config                 4      39m
kubernetes-dashboard-settings      0      39m
flyte-propeller-config             11     39m
clusterresource-template           2      39m
flyte-clusterresourcesync-config   3      39m
flyte-console-config               3      39m
flyte-admin-config                 9      39m
flyte-scheduler-config             4      39m
propeller-leader                   0      37m
leader-elect                       0      39m```

can you `kubectl -n flyte edit flyte-admin-config`

there should be a requests and a limits key underneath it

should look like this
```  task_resource_defaults.yaml: |
    task_resources:
      defaults:
        cpu: 1000m
        memory: 1000Mi
        storage: 1000Mi
      limits:
        cpu: 128
        gpu: 1
        memory: 192Gi
        storage: 2000Mi```


you’ll probably have to update your admin deployment afterwards

either kill all the admin pods or i think it’s something like `kubectl -n flyte rollout restart deploy flyteadmin`

I was able to reconfigure! I just don't know if I'll use this in my sandbox environment (I'm setting up a script so that my team can easily set up a sandbox when they don't want to deploy to the remote server), I think it'll become too complex

where are they setting up the sandbox?  like on their local machines?

we’re trying to make the deploy-to-remote-server step as easy as possible.

yeah, on their local machines. we're doing that while our remote server is not yet configured (this will be the next step!)

<@U016RNZDZRS> <@UPTRGR537> - whenever you guys get back on… is there a quick and easy way to provide configuration to flytectl sandbox to configure things like the default task resources in admin’s configmap?

i’ve always just edited after the fact, but maybe there’s a better way

Nope <@UNR3C6Y4T> flytectl doesn’t allow to edit any kubernetes objects directly and it’s only uses admin apis to change any configs. 
For changing the system level resource configuration you are right with suggestion of bumping the resources in the admin config map.

The one that we set from flytectl are values that admins stores as overrides but they cannot breach the system limits. 

We should probably allows setting overrides for these default task resource system wide values through helm , have those values exposed as ENV variables in entry point script and pass those from flytectl  sandbox/demo --env similar to FLYTE_TIMEOUT env variable .

there might be no end in sight for that though, but i kinda agree for this one.

this is the one thing that i’ve often needed to change too, just in running examples and such

Also i think this should be an advanced usecase , if the current limits in the charts are not enough for regular use cases then we should bump those in the helm chart directly and get it checkedin.

<@U016RNZDZRS> the storage values seems to be very less for sandbox <https://github.com/flyteorg/flyte/blob/2762eab96c1a5c90d794e850799eec3a39f547ab/charts/flyte/values.yaml#L457>
```    task_resource_defaults:
      # -- Task default resources parameters
      task_resources:
        defaults:
          cpu: 100m
          memory: 200Mi
          storage: 5Mi
        limits:
          cpu: 2
          memory: 1Gi
          storage: 20Mi
          gpu: 1```
Not sure if users want to bump up resources for this reason.

Also should we remove the platform config checks which act as cealing in flyteadmin. Thats a question for <@UPBBNMXD1> once she is back . This would allow the user to just use flytectl to set there resource configuration for any project domain or workflow and not worry about the platform limits but i guess we have these platform limits for a reason to prevent abuse of k8s resources .

yes the platform limits were intended to act as a global ceiling since there is no authz implementation atm but we can revisit that if it no longer makes sense!

I think the activated part is coming- I love te idea to show when it will come next - cc <@U01TEGHNJ2F> / <@U0231BEP02E> to chime. Cc <@U01J8L5AR2S> also can you recommend/propose a UX

If you do, please put it in an issue and we will take it in