's
Scientific Python Tools
In this section we collect some frequent errors typically found in beginner’s numpy code. We try to show where the problems come from by some easy examples and explain typical fixes.
We wish to construct an array, for example filled with zeros, but only get a data type not understood error message like this:
In [11]: zeros(3,4)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-dc85bc86d3a9> in <module>()
----> 1 zeros(3,4)
TypeError: data type not understood
The reason is that the call of zeros is slightly wrong. The correct call syntax is:
In [12]: zeros((3,4))
Out[12]:
array([[ 0., 0., 0., 0.],
[ 0., 0., 0., 0.],
[ 0., 0., 0., 0.]])
where we have to provide the shape as a python tuple (as always). There are similar pitfalls with ones, array and eye. The error occurs because zeros understands more arguments than the shape parameter only:
zeros(...)
zeros(shape, dtype=float, order='C')
Return a new array of given shape and type, filled with zeros.
Parameters
----------
shape : int or sequence of ints
Shape of the new array, e.g., ``(2, 3)``.
dtype : data-type, optional
The desired data-type for the array, e.g., `numpy.int8`. Default is
`numpy.float64`.
In the above example, shape would be 3 and dtype is 4 which is of course an invalid type description. The following are all valid examples:
In [21]: zeros((3,4), "int")
Out[21]:
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
In [22]: zeros((3,4), "float")
Out[22]:
array([[ 0., 0., 0., 0.],
[ 0., 0., 0., 0.],
[ 0., 0., 0., 0.]])
In [23]: zeros((3,4), "complex")
Out[23]:
array([[ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
[ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
[ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j]])
The error setting an array element with a sequence happens if we try to write something into a single place (array cell, matrix entry) of an array and this something is not a scalar value.
In [29]: A = zeros((3,3))
In [30]: A[1,1] = array([1,2])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-30-3f21ee5db7a1> in <module>()
----> 1 A[1,1] = array([1,2])
ValueError: setting an array element with a sequence.
This happens most often when we store the result of some computation which we think is scalar but in fact is not due to an error inside that computation. In the following example we compute a scalar product and store its value inside a matrix:
In [34]: v = array([[1,2,3]])
In [35]: A[1,1] = dot(transpose(v), v)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-35-cebb0c885c8d> in <module>()
----> 1 A[1,1] = dot(transpose(v), v)
ValueError: setting an array element with a sequence.
Why does this fail? Well, the result of dot is in this case not a scalar at all:
In [37]: v.shape
Out[37]: (1, 3)
In [38]: dot(transpose(v), v)
Out[38]:
array([[1, 2, 3],
[2, 4, 6],
[3, 6, 9]])
we started with a row vector and put the transpose on the wrong side. The other way round we get what we wanted:
In [39]: A[1,1] = dot(v, transpose(v))
In [40]: A
Out[40]:
array([[ 0., 0., 0.],
[ 0., 14., 0.],
[ 0., 0., 0.]])
This error message comes from a whole class of more or less generic errors occurring during array manipulation. They can sometimes be very hard to find:
In [111]: A = ones((3,3))
In [112]: b = array([[1, 2, 3]])
In [113]: dot(A, b)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-113-ed12d1ad26b4> in <module>()
----> 1 dot(A, b)
ValueError: matrices are not aligned
Why does this fail? We want to multiply a 3 times 3 matrix with a vector of 3 elements, everything should be fine from the mathematical point of view, no? If we look at the operands more closely we see it’s not:
In [114]: A
Out[114]:
array([[ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., 1., 1.]])
In [115]: b
Out[115]: array([[1, 2, 3]])
In [116]: A.shape
Out[116]: (3, 3)
In [117]: b.shape
Out[117]: (1, 3)
The vector is really a row vector! We need to transpose it first:
In [118]: dot(A, transpose(b))
Out[118]:
array([[ 6.],
[ 6.],
[ 6.]])
The error message we look at in this section tells us that we violated the broadcasting rules. This is not uncommon as broadcasting can give very unexpected results if done wrong or not understood properly. Remember that we can (for example) add a matrix and a vector but only if their shapes fulfill certain criteria:
In [124]: A = ones((3,5))
In [125]: A
Out[125]:
array([[ 1., 1., 1., 1., 1.],
[ 1., 1., 1., 1., 1.],
[ 1., 1., 1., 1., 1.]])
In [126]: v = ones((4,1))
In [127]: A + v
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-127-cf7178c26507> in <module>()
----> 1 A + v
ValueError: operands could not be broadcast together with shapes (3,5) (4,1)
If we look at v:
In [128]: v
Out[128]:
array([[ 1.],
[ 1.],
[ 1.],
[ 1.]])
we see that this column vector is one element to long. Let’s construct another one with matching length:
In [129]: w = ones((3,1))
In [130]: w
Out[130]:
array([[ 1.],
[ 1.],
[ 1.]])
In [131]: A + w
Out[131]:
array([[ 2., 2., 2., 2., 2.],
[ 2., 2., 2., 2., 2.],
[ 2., 2., 2., 2., 2.]])
Now it works as expected.
The exact broadcasting rules can be found at:
Here another simple case of a shape mismatch. We try to stack some matrices together in the usual block-matrix manipulation fashion:
In [41]: v
Out[41]: array([[1, 2, 3]])
In [42]: w = array([[4,5,6,7]])
In [43]: vstack([v,w])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-43-83da883f09da> in <module>()
----> 1 vstack([v,w])
/usr/lib64/python2.7/site-packages/numpy/core/shape_base.pyc in vstack(tup)
224
225 """
--> 226 return _nx.concatenate(map(atleast_2d,tup),0)
227
228 def hstack(tup):
ValueError: array dimensions must agree except for d_0
We can not stack together arrays if their shapes do not match! Let’s correct our mistake and build a matrix from two individual rows:
In [44]: w = array([[4,5,6]])
In [45]: w
Out[45]: array([[4, 5, 6]])
In [46]: vstack([v, w])
Out[46]:
array([[1, 2, 3],
[4, 5, 6]])
Errors originating from true mathematical linear algebra issues raise an exception called LinAlgError. Here we show two typical examples:
In [72]: O = ones((3,3))
In [73]: inv(O)
---------------------------------------------------------------------------
LinAlgError Traceback (most recent call last)
<ipython-input-73-bdcade699894> in <module>()
----> 1 inv(O)
/usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in inv(a)
443 """
444 a, wrap = _makearray(a)
--> 445 return wrap(solve(a, identity(a.shape[0], dtype=a.dtype)))
446
447
/usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in solve(a, b)
326 results = lapack_routine(n_eq, n_rhs, a, n_eq, pivots, b, n_eq, 0)
327 if results['info'] > 0:
--> 328 raise LinAlgError, 'Singular matrix'
329 if one_eq:
330 return wrap(b.ravel().astype(result_t))
LinAlgError: Singular matrix
Indeed the matrix has no inverse:
In [74]: det(O)
Out[74]: 0.0
Another one is:
In [75]: R = ones((3,4))
In [76]: inv(R)
---------------------------------------------------------------------------
LinAlgError Traceback (most recent call last)
<ipython-input-76-eb140fe92b28> in <module>()
----> 1 inv(R)
/usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in inv(a)
443 """
444 a, wrap = _makearray(a)
--> 445 return wrap(solve(a, identity(a.shape[0], dtype=a.dtype)))
446
447
/usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in solve(a, b)
310 b = b[:, newaxis]
311 _assertRank2(a, b)
--> 312 _assertSquareness(a)
313 n_eq = a.shape[0]
314 n_rhs = b.shape[1]
/usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in _assertSquareness(*arrays)
158 for a in arrays:
159 if max(a.shape) != min(a.shape):
--> 160 raise LinAlgError, 'Array must be square'
161
162 def _assertFinite(*arrays):
LinAlgError: Array must be square
This can of course never work:
In [77]: R.shape
Out[77]: (3, 4)
A failing matrix decomposition:
In [9]: A = diag([1,0,-1])
In [10]: A
Out[10]:
array([[ 1, 0, 0],
[ 0, 0, 0],
[ 0, 0, -1]])
In [11]: cholesky(A)
---------------------------------------------------------------------------
LinAlgError Traceback (most recent call last)
<ipython-input-11-59f690b5723f> in <module>()
----> 1 cholesky(A)
/usr/lib64/python2.7/site-packages/numpy/linalg/linalg.pyc in cholesky(a)
529 if results['info'] > 0:
530 raise LinAlgError, 'Matrix is not positive definite - \
--> 531 Cholesky decomposition cannot be computed'
532 s = triu(a, k=0).transpose()
533 if (s.dtype != result_t):
LinAlgError: Matrix is not positive definite - Cholesky decomposition cannot be computed
By construction the matrix A was not a valid input for Cholesky factorization.
This errors happens when plotting things goes wrong. It is a very typical error thrown by matplotlib. Let’s plot a simple graph with a few points:
In [87]: x = arange(5)
In [88]: y = array([[2,4,6,7,1]])
In [89]: figure()
Out[89]: <matplotlib.figure.Figure at 0x4e77550>
In [90]: plot(x, y)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-90-500b81ce9ab8> in <module>()
----> 1 plot(x, y)
/usr/lib64/python2.7/site-packages/matplotlib/pyplot.pyc in plot(*args, **kwargs)
2284 ax.hold(hold)
2285 try:
-> 2286 ret = ax.plot(*args, **kwargs)
2287 draw_if_interactive()
2288 finally:
/usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in plot(self, *args, **kwargs)
3783 lines = []
3784
-> 3785 for line in self._get_lines(*args, **kwargs):
3786 self.add_line(line)
3787 lines.append(line)
/usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in _grab_next_args(self, *args, **kwargs)
315 return
316 if len(remaining) <= 3:
--> 317 for seg in self._plot_args(remaining, kwargs):
318 yield seg
319 return
/usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in _plot_args(self, tup, kwargs)
292 x = np.arange(y.shape[0], dtype=float)
293
--> 294 x, y = self._xy_from_xy(x, y)
295
296 if self.command == 'plot':
/usr/lib64/python2.7/site-packages/matplotlib/axes.pyc in _xy_from_xy(self, x, y)
232 y = np.atleast_1d(y)
233 if x.shape[0] != y.shape[0]:
--> 234 raise ValueError("x and y must have same first dimension")
235 if x.ndim > 2 or y.ndim > 2:
236 raise ValueError("x and y can be no greater than 2-D")
ValueError: x and y must have same first dimension
The reason for this failure is that the two arrays x and y have the same number of elements but still a different shape:
In [91]: x
Out[91]: array([0, 1, 2, 3, 4])
In [92]: y
Out[92]: array([[2, 4, 6, 7, 1]])
In [93]: x.shape
Out[93]: (5,)
In [94]: y.shape
Out[94]: (1, 5)
In [95]: x.ndim
Out[95]: 1
In [96]: y.ndim
Out[96]: 2
This is very nasty and happens now and then not to beginners only. Because we were not careful enough, one array is two-dimensional! Mostly this actually happens when one part is a matrix having one row or column. The solution is to call squeeze to remove the singular dimension(s):
In [97]: figure()
Out[97]: <matplotlib.figure.Figure at 0x515fd50>
In [98]: plot(x, squeeze(y))
Out[98]: [<matplotlib.lines.Line2D at 0x5206050>]