Python:ValueError: shapes (3,) and (118,1) not aligned: 3 (dim 0) != 118 (dim 0)

Question

I am trying to do logistic regression using fmin but there is an error showing up due to different shapes of array. Here is the code.

import numpy as np

import scipy.optimize as sp

data= #an array of dim (188,3)

X=data[:,0:2]
y=data[:,2]
m,n=np.shape(X)
y=y.reshape(m,1)
x=np.c_[np.ones((m,1)),X]
theta=np.zeros((n+1,1))


def hypo(x,theta): 
    return np.dot(x,theta)

def sigmoid(z):
    return 1/(1+np.exp(-z))
    
def gradient(x,y,theta):#calculating Gradient
    m=np.shape(x)[0]
    t=hypo(x,theta)
    hx=sigmoid(t)
    J=-(np.dot(np.transpose(np.log(hx)),y)+np.dot(np.transpose(np.log(1-hx)),(1-y)))/m
    grad=np.dot(np.transpose(x),(hx-y))/m
    J= J.flatten()
    grad=grad.flatten()
    return J,grad
    
def costFunc(x,y,theta):    
    return gradient(x,y,theta)[0]
    
def Grad():
   return gradient(x,y,theta)[1]

sp.fmin( costFunc, x0=theta, args=(x, y), maxiter=500, full_output=True)

error that is showing

File "<ipython-input-3-31a0d7ca38c8>", line 35, in costFunc

return gradient(x,y,theta)[0]

File "<ipython-input-3-31a0d7ca38c8>", line 25, in gradient

t=hypo(x,theta)

File "<ipython-input-3-31a0d7ca38c8>", line 16, in hypo

return np.dot(x,theta)

ValueError: shapes (3,) and (118,1) not aligned: 3 (dim 0) != 118 (dim 0)

Any kind of help will be appreciated

Answer 1

data= #an array of dim (188,3)

X=data[:,0:2]
y=data[:,2]
m,n=np.shape(X)
y=y.reshape(m,1)
x=np.c_[np.ones((m,1)),X]
theta=np.zeros((n+1,1))

so after this

In [14]: y.shape
Out[14]: (188, 1)   # is this (118,1)?
In [15]: x.shape
Out[15]: (188, 3)
In [16]: theta.shape
Out[16]: (3, 1)

This x and theta can dotted - np.dot(x,theta), and (188,3) with (3,1) - matching the 3's.

But that's not what your costFunc is getting. Tracing back from the error message it looks like x is (3,), and theta is (118,1). which obviously cannot be dotted.

You need to review how fmin calls your function. Do you have the parameters in the right order? For example, maybe costFunc(theta, x, y) is the correct order (assuming the x and y in costFunc are meant to match with the args=(x,y).

The docs for fmin include:

func : callable func(x,*args)
    The objective function to be minimized.
x0 : ndarray
    Initial guess.
args : tuple, optional
    Extra arguments passed to func, i.e. ``f(x,*args)``.

It looks like fmin is feeding your costFunc 3 arguments, corresponding in size to your (theta, x, y), i.e. (3,), (118,3), (118,1). The numbers don't quite match, but I think you get the idea. The first argument to consFunc is the one that the fmin will vary, the rest you provide in args.