Compute log loss for logistic regression from scratch

Question

compute log-loss

def logloss(y_true,y_pred):
    '''In this function, we will compute log loss '''
    log_loss = (-((y_true * np.log10(y_pred)) + (1-y_true) * np.log10(1-y_pred)).mean())
    return log_loss

Computing logistic regression

def train(X_train,y_train,X_test,y_test,epochs,alpha,eta0):
 w,b = initialize_weights(X_train[0])
    train_loss = []
    test_loss = []
    for e in range(epochs):
        for x,y in zip(X_train,y_train):
            dw = gradient_dw(x,w,y,b,alpha,N)
            db = gradient_db(x,y,w,b)
            w = w + (eta0 * dw)
            b = b + (eta0 * db)
        train_pred = []
        for i in X_train:
            y_pred = sigmoid(np.dot(w.T, i) + b)
            train_pred.append(y_pred)
        train_loss.append(logloss(y_train, train_pred))
        
        test_pred = []
        for j in X_test:
            y_pred_test = sigmoid(np.dot(w.T, j) + b)
            test_pred.append(y_pred_test)
        test_loss.append(logloss(y_test, test_pred))
    return w,b

alpha=0.0001
eta0=0.0001
epochs=50
N = len(X_train)
w,b = train(X_train,y_train,X_test,y_test,epochs,alpha,eta0)

Error that I am getting

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-112-9a34879eb072> in <module>
      3 epochs=50
      4 N = len(X_train)
----> 5 w,b = train(X_train,y_train,X_test,y_test,epochs,alpha,eta0)

<ipython-input-110-db0e3d88382d> in train(X_train, y_train, X_test, y_test, epochs, alpha, eta0)
     30             y_pred = sigmoid(np.dot(w.T, i) + b)
     31             train_pred.append(y_pred)
---> 32         train_loss.append(logloss(y_train, train_pred))
     33 
     34         test_pred = []

<ipython-input-108-f272288a384c> in logloss(y_true, y_pred)
      1 def logloss(y_true,y_pred):
      2     '''In this function, we will compute log loss '''
----> 3     log_loss = (-((y_true * np.log10(y_pred)) + (1-y_true) * np.log10(1-y_pred)).mean())
      4     return log_loss

TypeError: unsupported operand type(s) for -: 'int' and 'list'

I have mentioned the full code just the codes for which I am getting error.I am confused whether to make changes to logloss or make changes to logistic regression code i.e def train(). How to rectify this error?

Answer 1

You have made  train_pred as python list. When logloss function is used, you calculate(1- train_pred), which means integer minus python list.
This is the reason you get an error.

TypeError: unsupported operand type(s) for -: 'int' and 'list'

Hope this helps.

Elevate Your Expertise with Our Machine Learning Certification Program!