This message was deleted.

@Chaoyu - great question!! I think the most common issue we see is teams not realizing how much work they are biting off when deciding to build a model monitoring solution in house. How to handle various model versions and training/test/prod environments, embeddings, ranking models metrics, the UI visualization to debug model metrics, looking at various segments not just aggregate metrics…. its a LOT. We have seen teams start internally first and then realize how much time they are investing in building this when it isn’t their company/team objective to have the best ml observability platform … it’s to build the best models. It is not just the initial build, but the continual maintenance. If you are still deciding to build, I’d check out this checklist of the fundamentals to build: https://arize.com/resource/machine-learning-observability-checklist/