This message was deleted.

@Scott Hedstrom Currently there isn’t a way to deploy as a spot instance. We should add that capability on the EC2 deploy operator cc @jjmachan

Seems like its almost 4k a month to keep a g4dn running in the supported way. Could see this being necessary in a few months, but especially during prototyping would be much more feasible to spot instance when renders take place.

@Scott Hedstrom I think it depends on the g4dn, the smaller ones with 16gb and 1 gpu are only $338 I think a month. Which was the more expensive one that you wanted to run? Also, when you do a bentoctl generate, it should create a main.tf which is a terraform script to deploy your bento. You could modify it to use a spot instance with this terraform feature: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/spot_instance_request

Perhaps I was looking at the wrong prices, but it seemed the smallest g4dn dedicated was costin 10x that...338 a month is totally reasonable....quick question, if there are multiple requests coming thru does the service handle queuing them? or would I need to throttle and queue requests myself?

I think setting the timeout high enough should generally give you what you want.

We are in prototyping phase but working towards having a service which I am sure will need some sort of scaling ability at some point.

@Scott Hedstrom happy to hop on a call and learn more

Thanks @Bo! I'm a lil tied up for next bit...Ill be available 430MST (in about 5 hours). Otherwise can just connect anytime u around when I am. Our needs are pretty simple, we're building a UI that wraps using stable diffusion in a way that empowers non-tech style folks to really take advantage of it. So will need to have an endpoint that our platform can hit to request a stable-diffusion generation, and then continue on w our platforms usual stuff...Perhaps we will need to do queue-ing ourself on the platform side.