This message was deleted.

update on this: I still can't get yatai-deployment configured with our internal docker registry but for now I've circumvented this issue by installing an in-cluster registry as per the docs and I push the base images I need to this registry beforehand. This results in yatai being able to successfully build the service image from the bento and then pushes them to this registry (I can see the Docker Image Builder Pods step succeeds) however it then gets stuck in the step to actually create the deployment from the image (see error logs below). I'm just trying to create the deployment through the front-end. Can anyone help on this? Error logs:

[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetBento] Fetching Bento florida-all-models:ut6o7ktq42qcsasc
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetBento] Fetched Bento florida-all-models:ut6o7ktq42qcsasc
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [MakeSureDockerRegcred] Successfully made sure docker registry credentials
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetMajorCluster] Fetching major cluster
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetMajorCluster] Successfully fetched major cluster
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetYataiVersion] Fetching yatai version
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetYataiVersion] Successfully fetched yatai version
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CheckImageExists] Checking image docker-registry.yatai-deployment.svc.cluster.local:5000/yatai-bentos:yatai.florida-all-models.ut6o7ktq42qcsasc exists
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CheckImageExists] Image docker-registry.yatai-deployment.svc.cluster.local:5000/yatai-bentos:yatai.florida-all-models.ut6o7ktq42qcsasc exists
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetDeployment] Getting Deployment yatai/florida-test
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CreateDeployment] Creating a new Deployment yatai/florida-test
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CreateDeployment] Created Deployment yatai/florida-test
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetHPA] Getting HPA yatai/florida-test
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CreateHPA] Creating a new HPA yatai/florida-test
[2022-11-30 22:29:42] [Pod] [florida-test-85d98f9b5f-tbxh6] [Scheduled] Successfully assigned yatai/florida-test-85d98f9b5f-tbxh6 to minikube
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CreateHPA] Created HPA yatai/florida-test
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [GetService] Getting Service yatai/florida-test
[2022-11-30 22:29:41] [BentoDeployment] [florida-test] [CreateService] Creating a new Service yatai/florida-test
[2022-11-30 22:29:42] [BentoDeployment] [florida-test] [CreateService] Created Service yatai/florida-test
[2022-11-30 22:29:42] [BentoDeployment] [florida-test] [GenerateIngressHost] Generating hostname for ingress
[2022-11-30 22:29:42] [Pod] [florida-test-85d98f9b5f-tbxh6] [Pulling] Pulling image "127.0.0.1:5000/yatai-bentos:yatai.florida-all-models.ut6o7ktq42qcsasc"
[2022-11-30 22:29:52] [BentoDeployment] [florida-test] [GenerateIngressHost] Generated hostname for ingress: <http://florida-test-yatai.192.168.49.2.sslip.io|florida-test-yatai.192.168.49.2.sslip.io>
[2022-11-30 22:29:52] [BentoDeployment] [florida-test] [GetIngress] Getting Ingress yatai/florida-test
[2022-11-30 22:29:52] [BentoDeployment] [florida-test] [CreateIngress] Creating a new Ingress yatai/florida-test
[2022-11-30 22:29:52] [BentoDeployment] [florida-test] [CreateIngress] Created Ingress yatai/florida-test
[2022-11-30 22:29:52] [BentoDeployment] [florida-test] [GetYataiDeployment] Fetching yatai deployment florida-test
[2022-11-30 22:29:55] [BentoDeployment] [florida-test] [UpdateYataiDeployment] Updating yatai deployment florida-test
[2022-11-30 22:30:27] [HorizontalPodAutoscaler] [florida-test] [FailedGetResourceMetric] failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
[2022-11-30 22:30:27] [HorizontalPodAutoscaler] [florida-test] [FailedComputeMetricsReplicas] invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
[2022-11-30 22:31:41] [Pod] [florida-test-85d98f9b5f-tbxh6] [Failed] Failed to pull image "127.0.0.1:5000/yatai-bentos:yatai.florida-all-models.ut6o7ktq42qcsasc": rpc error: code = Unknown desc = context deadline exceeded
[2022-11-30 22:31:44] [Pod] [florida-test-85d98f9b5f-tbxh6] [Created] Created container metrics-transformer
[2022-11-30 22:31:46] [Pod] [florida-test-85d98f9b5f-tbxh6] [Started] Started container metrics-transformer
[2022-11-30 22:31:49] [Pod] [florida-test-85d98f9b5f-tbxh6] [BackOff] Back-off pulling image "127.0.0.1:5000/yatai-bentos:yatai.florida-all-models.ut6o7ktq42qcsasc"
[2022-11-30 22:31:49] [Pod] [florida-test-85d98f9b5f-tbxh6] [Failed] Error: ImagePullBackOff

I am having the same issue with that. https://bentoml.slack.com/archives/CKRANBHPH/p1669833078827869