This message was deleted.

This makes a ton of sense! In fact, maybe we should add SQL "runners" as a feature!

That would be so cool.

Currently, I am using Feast feature store in BentoML 1.0.7. Our company uses Google Cloud Platform. So, the offline store is BigQuery and online store is Google Datastore. I am using Feast SDK to fetch features from the Datastore. I instantiated feature store in the runner. For each online prediction, I fetch a list of features from the request given to me in batch. Since Datastore is very fast for online serving, it is not an issue for us.

That does it. I've officially heard good things about Feast from one too many people. Time to google it 🤣

@Eric Riddoch good stuff! Feast was revamped and adopted simple approach to allow it to be integrated with any applications. for google Datastore, the recommendation is to fetch data in batch, fits right into the bentoml paradigm. pretty sweet! the way we do it is we convert the batch request in our runner to a dataframe. then we grab the ids as list and pass it to the datastore. the feast sdk gives us back a dataframe of features, which allows us to concatenate original dataframe with new dataframe from feast before sending the final dataframe to xgboost for inference. voila! i got batch prediction back and can return it so that bentoml can distribute the predictions to each caller. for computer vision, Feast is working on storing embedding i last heard.

Whoa. That is really cool. I'm saving this thread so it doesn't get eaten by the free slack plan

Whoa, super cool idea! Love the application of batching to something like feature requests!