I've always built this kind of thing myself. E.g., track summary statistics of your target as an
mlflow run metric, and then have a separate process for pulling that data and analyzing it (could be a dashboard, slack alert based on some threshold, etc.).
You might get some inspiration from
https://docs.evidentlyai.com/ (I haven't used this in prod before, but it looks nice)