This message was deleted.

Response received after running the command:

2022-10-25T15:37:11+0100 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "prediction_service:svc" can be accessed at <http://localhost:3000/metrics>.
2022-10-25T15:37:12+0100 [INFO] [cli] Starting development HTTP BentoServer from "prediction_service:svc" running on <http://0.0.0.0:3000> (Press CTRL+C to quit)
2022-10-25 15:37:20.384071: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with 
oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-25 15:37:21.242426: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3967 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
2022-10-25 15:37:31.527252: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with 
oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

hello @Rizdi Aprilian You shouldnt need to use the model_ref itself for prediction. Instead of

with tf.device('/CPU:0'):
        img_pred = model_ref.predict(input_featuremap)

You should call `model_runner.predict.run(input_featuremap)

After a long while of from notebook to python scripts, some points that I find from producing Keras service with BentoML: 1. In updating model from training, I used callbacks (both Checkpoint and Earlystopping) that allows the model to be saved with the best performance or stopped if the training performance remains unchanged. 2. As such, a collection of model checkpoint in directory is generated. Previously, I used a trained keras file rather than the directory produced from checkpoint. That explains why the prediction service does not work properly when receiving an input image in numpy array. 3. Realizing this, I write a file that will load the model directory from checkpoint and save it into bentoml package. Then serving it as usual and finally it delivers the predictions to the given new input.