Some frequent errors ==================== In this section we collect some frequent errors typically found in beginner's ``numpy`` code. We try to show where the problems come from by some easy examples and explain typical fixes. Error: ``data type not understood`` ----------------------------------- We wish to construct an array, for example filled with zeros, but only get a ``data type not understood`` error message like this: .. sourcecode:: ipython In [11]: zeros(3,4) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () ----> 1 zeros(3,4) TypeError: data type not understood The reason is that the call of ``zeros`` is slightly wrong. The correct call syntax is: .. sourcecode:: ipython In [12]: zeros((3,4)) Out[12]: array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]]) where we have to provide the *shape* as a python tuple (as always). There are similar pitfalls with ``ones``, ``array`` and ``eye``. The error occurs because ``zeros`` understands more arguments than the ``shape`` parameter only:: zeros(...) zeros(shape, dtype=float, order='C') Return a new array of given shape and type, filled with zeros. Parameters ---------- shape : int or sequence of ints Shape of the new array, e.g., ``(2, 3)``. dtype : data-type, optional The desired data-type for the array, e.g., `numpy.int8`. Default is `numpy.float64`. In the above example, ``shape`` would be ``3`` and ``dtype`` is ``4`` which is of course an invalid type description. The following are all valid examples: .. sourcecode:: ipython In [21]: zeros((3,4), "int") Out[21]: array([[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]) In [22]: zeros((3,4), "float") Out[22]: array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]]) In [23]: zeros((3,4), "complex") Out[23]: array([[ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j]]) Error: ``setting an array element with a sequence`` --------------------------------------------------- The error ``setting an array element with a sequence`` happens if we try to write something into a single place (`array cell`, `matrix entry`) of an array and this `something` is not a scalar value. .. sourcecode:: ipython In [29]: A = zeros((3,3)) In [30]: A[1,1] = array([1,2]) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 A[1,1] = array([1,2]) ValueError: setting an array element with a sequence. This happens most often when we store the result of some computation which we think is scalar but in fact is not due to an error inside that computation. In the following example we compute a scalar product and store its value inside a matrix: .. sourcecode:: ipython In [34]: v = array([[1,2,3]]) In [35]: A[1,1] = dot(transpose(v), v) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 A[1,1] = dot(transpose(v), v) ValueError: setting an array element with a sequence. Why does this fail? Well, the result of ``dot`` is in this case not a scalar at all: .. sourcecode:: ipython In [37]: v.shape Out[37]: (1, 3) In [38]: dot(transpose(v), v) Out[38]: array([[1, 2, 3], [2, 4, 6], [3, 6, 9]]) we started with a row vector and put the ``transpose`` on the wrong side. The other way round we get what we wanted: .. sourcecode:: ipython In [39]: A[1,1] = dot(v, transpose(v)) In [40]: A Out[40]: array([[ 0., 0., 0.], [ 0., 14., 0.], [ 0., 0., 0.]]) Error: ``matrices are not aligned`` ----------------------------------- This error message comes from a whole class of more or less generic errors occurring during array manipulation. They can sometimes be very hard to find: .. sourcecode:: ipython In [111]: A = ones((3,3)) In [112]: b = array([[1, 2, 3]]) In [113]: dot(A, b) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 dot(A, b) ValueError: matrices are not aligned Why does this fail? We want to multiply a 3 times 3 matrix with a vector of 3 elements, everything should be fine from the mathematical point of view, no? If we look at the operands more closely we see it's not: .. sourcecode:: ipython In [114]: A Out[114]: array([[ 1., 1., 1.], [ 1., 1., 1.], [ 1., 1., 1.]]) In [115]: b Out[115]: array([[1, 2, 3]]) In [116]: A.shape Out[116]: (3, 3) In [117]: b.shape Out[117]: (1, 3) The vector is really a *row vector*! We need to transpose it first: .. sourcecode:: ipython In [118]: dot(A, transpose(b)) Out[118]: array([[ 6.], [ 6.], [ 6.]]) Error: ``operands could not be broadcast`` ------------------------------------------ The error message we look at in this section tells us that we violated the *broadcasting rules*. This is not uncommon as broadcasting can give very unexpected results if done wrong or not understood properly. Remember that we can (for example) add a matrix and a vector but *only* if their shapes fulfill certain criteria: .. sourcecode:: ipython In [124]: A = ones((3,5)) In [125]: A Out[125]: array([[ 1., 1., 1., 1., 1.], [ 1., 1., 1., 1., 1.], [ 1., 1., 1., 1., 1.]]) In [126]: v = ones((4,1)) In [127]: A + v --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 A + v ValueError: operands could not be broadcast together with shapes (3,5) (4,1) If we look at ``v``: .. sourcecode:: ipython In [128]: v Out[128]: array([[ 1.], [ 1.], [ 1.], [ 1.]]) we see that this column vector is one element to long. Let's construct another one with matching length: .. sourcecode:: ipython In [129]: w = ones((3,1)) In [130]: w Out[130]: array([[ 1.], [ 1.], [ 1.]]) In [131]: A + w Out[131]: array([[ 2., 2., 2., 2., 2.], [ 2., 2., 2., 2., 2.], [ 2., 2., 2., 2., 2.]]) Now it works as expected. The exact broadcasting rules can be found at: - http://docs.scipy.org/doc/numpy/reference/ufuncs.html#broadcasting Error: ``array dimensions must agree`` -------------------------------------- Here another simple case of a shape mismatch. We try to stack some matrices together in the usual block-matrix manipulation fashion: .. sourcecode:: ipython In [41]: v Out[41]: array([[1, 2, 3]]) In [42]: w = array([[4,5,6,7]]) In [43]: vstack([v,w]) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 vstack([v,w]) /usr/lib64/python2.7/site-packages/numpy/core/shape_base.pyc in vstack(tup) 224 225 """ --> 226 return _nx.concatenate(map(atleast_2d,tup),0) 227 228 def hstack(tup): ValueError: array dimensions must agree except for d_0 We can not stack together arrays if their shapes do not match! Let's correct our mistake and build a matrix from two individual rows: .. sourcecode:: ipython In [44]: w = array([[4,5,6]]) In [45]: w Out[45]: array([[4, 5, 6]]) In [46]: vstack([v, w]) Out[46]: array([[1, 2, 3], [4, 5, 6]]) Error: ``LinAlgError: ...`` --------------------------- Errors originating from true mathematical linear algebra issues raise an exception called ``LinAlgError``. Here we show two typical examples: .. sourcecode:: ipython In [72]: O = ones((3,3)) In [73]: inv(O) --------------------------------------------------------------------------- LinAlgError Traceback (most recent call last) in () ----> 1 inv(O) /usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in inv(a) 443 """ 444 a, wrap = _makearray(a) --> 445 return wrap(solve(a, identity(a.shape[0], dtype=a.dtype))) 446 447 /usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in solve(a, b) 326 results = lapack_routine(n_eq, n_rhs, a, n_eq, pivots, b, n_eq, 0) 327 if results['info'] > 0: --> 328 raise LinAlgError, 'Singular matrix' 329 if one_eq: 330 return wrap(b.ravel().astype(result_t)) LinAlgError: Singular matrix Indeed the matrix has no inverse: .. sourcecode:: ipython In [74]: det(O) Out[74]: 0.0 Another one is: .. sourcecode:: ipython In [75]: R = ones((3,4)) In [76]: inv(R) --------------------------------------------------------------------------- LinAlgError Traceback (most recent call last) in () ----> 1 inv(R) /usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in inv(a) 443 """ 444 a, wrap = _makearray(a) --> 445 return wrap(solve(a, identity(a.shape[0], dtype=a.dtype))) 446 447 /usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in solve(a, b) 310 b = b[:, newaxis] 311 _assertRank2(a, b) --> 312 _assertSquareness(a) 313 n_eq = a.shape[0] 314 n_rhs = b.shape[1] /usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in _assertSquareness(*arrays) 158 for a in arrays: 159 if max(a.shape) != min(a.shape): --> 160 raise LinAlgError, 'Array must be square' 161 162 def _assertFinite(*arrays): LinAlgError: Array must be square This can of course never work: .. sourcecode:: ipython In [77]: R.shape Out[77]: (3, 4) A failing matrix decomposition: .. sourcecode:: ipython In [9]: A = diag([1,0,-1]) In [10]: A Out[10]: array([[ 1, 0, 0], [ 0, 0, 0], [ 0, 0, -1]]) In [11]: cholesky(A) --------------------------------------------------------------------------- LinAlgError Traceback (most recent call last) in () ----> 1 cholesky(A) /usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in cholesky(a) 529 if results['info'] > 0: 530 raise LinAlgError, 'Matrix is not positive definite - \ --> 531 Cholesky decomposition cannot be computed' 532 s = triu(a, k=0).transpose() 533 if (s.dtype != result_t): LinAlgError: Matrix is not positive definite - Cholesky decomposition cannot be computed By construction the matrix ``A`` was not a valid input for Cholesky factorization. Error: ``x and y must have same first dimension`` ------------------------------------------------- This errors happens when plotting things goes wrong. It is a very typical error thrown by ``matplotlib``. Let's plot a simple graph with a few points: .. sourcecode:: ipython In [87]: x = arange(5) In [88]: y = array([[2,4,6,7,1]]) In [89]: figure() Out[89]: In [90]: plot(x, y) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 plot(x, y) /usr/lib64/python2.7/site-packages/matplotlib/pyplot.pyc in plot(*args, **kwargs) 2284 ax.hold(hold) 2285 try: -> 2286 ret = ax.plot(*args, **kwargs) 2287 draw_if_interactive() 2288 finally: /usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in plot(self, *args, **kwargs) 3783 lines = [] 3784 -> 3785 for line in self._get_lines(*args, **kwargs): 3786 self.add_line(line) 3787 lines.append(line) /usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in _grab_next_args(self, *args, **kwargs) 315 return 316 if len(remaining) <= 3: --> 317 for seg in self._plot_args(remaining, kwargs): 318 yield seg 319 return /usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in _plot_args(self, tup, kwargs) 292 x = np.arange(y.shape[0], dtype=float) 293 --> 294 x, y = self._xy_from_xy(x, y) 295 296 if self.command == 'plot': /usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in _xy_from_xy(self, x, y) 232 y = np.atleast_1d(y) 233 if x.shape[0] != y.shape[0]: --> 234 raise ValueError("x and y must have same first dimension") 235 if x.ndim > 2 or y.ndim > 2: 236 raise ValueError("x and y can be no greater than 2-D") ValueError: x and y must have same first dimension The reason for this failure is that the two arrays ``x`` and ``y`` have the same number of elements but still a *different shape*: .. sourcecode:: ipython In [91]: x Out[91]: array([0, 1, 2, 3, 4]) In [92]: y Out[92]: array([[2, 4, 6, 7, 1]]) In [93]: x.shape Out[93]: (5,) In [94]: y.shape Out[94]: (1, 5) In [95]: x.ndim Out[95]: 1 In [96]: y.ndim Out[96]: 2 This is very nasty and happens now and then not to beginners only. Because we were not careful enough, one array is two-dimensional! Mostly this actually happens when one part is a matrix having one row or column. The solution is to call ``squeeze`` to remove the singular dimension(s): .. sourcecode:: ipython In [97]: figure() Out[97]: In [98]: plot(x, squeeze(y)) Out[98]: []