This message was deleted.

Hey! Hamilton should work happily with most of them, so it should all work from that perspective. As to managed platforms, I won’t pick favorites (they’re all good options, and I have experience with all except predibase), but its about trade-offs: 1. Metaflow is closer to an orchestration system — it’ll help you run jobs, etc… It can work with modal/ray (I think, see https://medium.com/@riley.hun_3260/supercharging-ml-workflows-integrating-metaflow-with-ray-18b0b67cf7a6) 2. Modal/ray are competitors (in some way), they both execute things remotely. You’ll still need a place to run your code, but it can be simpler as you’re delegating the heavy lifting to a remote server. I’ll let @Stefan Krawczyk and @Thierry Jean chime in with more thoughts! We have a POC for a modal integration that puts specific hamilton nodes on Modal, but the simplest way is likely to run the whole Hamilton DAG on Modal. We also have some fine-tuning examples (and more we’re working on). See this to get started! https://hub.dagworks.io/docs/Users/skrawcz/fine_tuning/ You can download/tune to your liking. It uses QLora, but it should be a configurable start.

@Carlos Leyson do you have constraints on where you can send your data? You can probably separate inference from fine tuning — since those are different workloads.

No constraints

@Carlos Leyson cool. yeah I don’t have strong advice, other than make sure you keep your code modular so that you can swap where things run, and how things are loaded 🙂 Maybe someone with more experience (e.g. @Gireesh Ramji) could chime in?

Thank you!