This message was deleted.

Hi Taeef, could you give me an example of your model's input and expected output? Thanks

Hi @larme (shenyang), While training the model takes tensors (

torch.FloatTensor(X_train)

and

torch.LongTensor(y_train)

as input. While testing it takes a

NumpyNdarray

such as

[[10,168,74,0,0,38,0.537,34]]

as input and is expected to return the class

0

or

1

as the output. Here's a portion of my code if that helps:

# Creating ANN model with PyTorch
class ANN_Model(nn.Module):
    def __init__(self,input_features=8,hidden1=20,hidden2=20,out_features=2):
        super().__init__()
        self.f_connected1=nn.Linear(input_features,hidden1)
        self.f_connected2=nn.Linear(hidden1,hidden2)
        self.out=nn.Linear(hidden2,out_features)
    def forward(self,x):
        x=F.relu(self.f_connected1(x))
        x=F.relu(self.f_connected2(x))
        x=self.out(x)
        return x

# Training model
def train(hp: Hyperparameters, X_train: torch.FloatTensor, y_train: torch.LongTensor) -> nn.Module:
    torch.manual_seed(hp.random_state)
    model=ANN_Model()
    # Defining the loss function and the optimizer
    loss_function=nn.CrossEntropyLoss()
    optimizer=torch.optim.Adam(model.parameters(),lr=<http://hp.lr|hp.lr>)
    final_losses=[]
    for i in range(hp.epochs):
        i=i+1
        y_pred=model.forward(X_train)
        loss=loss_function(y_pred,y_train)
        final_losses.append(loss)
        if i%10==1:
            print(f"Epoch: {i}    Loss: {loss.item()}")
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    return model

Please let me know if you need more info about my code.

Hi Taeef, one possible cause is that if you enable batching when saving the model with bentoml like

bentoml.pytorch.save(
    model,
    "my_torch_model",
    signatures={"__call__": {"batchable": True, "batch_dim": 0}},
)

Then the input of

model_runner.async_run

should be a batch input. So I'd like to suggest expand_dim the input_tensor at axis=0 like:

inp = np.expand_dims(inp, axis=0)
input_tensor = torch.FloatTensor(inp)
...

and see if this fix the issue

@larme (shenyang) thanks for the reply. I didn't use

signatures = {"__call__"" {"batchable": True, "batch_dim":0}},

but i tested with

inp = np.expand_dim(inp, axis=0)

and it returned

[[[-11.270812034606934, -9.508756637573242]]]

now. Previously it returned

[[-11.270812034606934, -9.508756637573242]]

By any chance, can I use this in

service.py

to return

0

or

1

?

import numpy as np

output = [[-11.270812034606934, -9.508756637573242]]
prediction = 0 if output[0][0] > output[0][1] else 1
print(prediction)

The problem is very strange, but I think it's ok to use this way to return 0 and 1

Thanks @larme (shenyang) for always helping