How to create a numpy array of lists?

As you discovered, np.array tries to create a 2d array when given something like

 A = np.array([[1,2],[3,4]],dtype=object)

You have apply some tricks to get around this default behavior.

One is to make the sublists variable in length. It can’t make a 2d array from these, so it resorts to the object array:

In [43]: A=np.array([[1,2],[],[1,2,3,4]])
In [44]: A
Out[44]: array([[1, 2], [], [1, 2, 3, 4]], dtype=object)

And you can then append values to each of those lists:

In [45]: for i in A: i.append(34)
In [46]: A
Out[46]: array([[1, 2, 34], [34], [1, 2, 3, 4, 34]], dtype=object)

np.empty also creates an object array:

In [47]: A=np.empty((3,),dtype=object)
In [48]: A
Out[48]: array([None, None, None], dtype=object)

But you then have to be careful how you change the elements to lists. np.fill is tempting, but has problems:

In [49]: A.fill([])
In [50]: A
Out[50]: array([[], [], []], dtype=object)
In [51]: for i in A: i.append(34)
In [52]: A
Out[52]: array([[34, 34, 34], [34, 34, 34], [34, 34, 34]], dtype=object)

It turns out that fill puts the same list in all slots, so modifying one modifies all the others. You can get the same problem with a list of lists:

In [53]: B=[[]]*3
In [54]: B
Out[54]: [[], [], []]
In [55]: for i in B: i.append(34)
In [56]: B
Out[56]: [[34, 34, 34], [34, 34, 34], [34, 34, 34]]

The proper way to initial the empty A is with an iteration, e.g.

In [65]: A=np.empty((3,),dtype=object)
In [66]: for i,v in enumerate(A): A[i]=[v,i]
In [67]: A
Out[67]: array([[None, 0], [None, 1], [None, 2]], dtype=object)
In [68]: for v in A: v.append(34)
In [69]: A
Out[69]: array([[None, 0, 34], [None, 1, 34], [None, 2, 34]], dtype=object)

It’s a little unclear from the question and comments whether you want to append to the lists, or append lists to the array. I’ve just demonstrated appending to the lists.

There is an np.append function, which new users often misuse. It isn’t a substitute for list append. It is a front end to np.concatenate. It is not an in-place operation; it returns a new array.

Also defining a list to add with it can be tricky:

In [72]: np.append(A,[[1,23]])
Out[72]: array([[None, 0, 34], [None, 1, 34], [None, 2, 34], 1, 23],     dtype=object)

You need to construct another object array to concatenate to the original, e.g.

In [76]: np.append(A,np.empty((1,),dtype=object))
Out[76]: array([[None, 0, 34], [None, 1, 34], [None, 2, 34], None], dtype=object)

In all of this, an array of lists is harder to construct than a list of lists, and no easier, or faster, to manipulate. You have to make it a 2d array of lists to derive some benefit.

In [78]: A[:,None]
Out[78]: 
array([[[None, 0, 34]],
       [[None, 1, 34]],
       [[None, 2, 34]]], dtype=object)

You can reshape, transpose, etc an object array, where as creating and manipulating a list of lists of lists gets more complicated.

In [79]: A[:,None].tolist()
Out[79]: [[[None, 0, 34]], [[None, 1, 34]], [[None, 2, 34]]]

===

As shown in https://stackoverflow.com/a/57364472/901925, np.frompyfunc is a good tool for creating an array of objects.

np.frompyfunc(list, 0, 1)(np.empty((3,2), dtype=object))

Related Posts:

  1. numpy-equivalent of list.pop?
  2. TypeError: only integer arrays with one element can be converted to an index 3
  3. TypeError: list indices must be integers or slices, not str
  4. ValueError: setting an array element with a sequence
  5. Numpy ValueError: setting an array element with a sequence. This message may appear without the existing of a sequence?
  6. numpy: Invalid value encountered in true_divide
  7. How do I create an empty array/matrix in NumPy?
  8. numpy matrix vector multiplication
  9. Convert pandas dataframe to NumPy array
  10. size of NumPy array
  11. Python List vs. Array – when to use?
  12. How to find all occurrences of an element in a list
  13. ValueError: all the input arrays must have same number of dimensions
  14. ValueError: setting an array element with a sequence
  15. Using Numpy Vectorize on Functions that Return Vectors
  16. Is there a NumPy function to return the first index of something in an array?
  17. Numpy/Python Array Value error
  18. Python: Array v. List
  19. ‘list’ object has no attribute ‘shape’
  20. initialize a numpy array
  21. load csv into 2D matrix with numpy for plotting
  22. How to access the ith column of a NumPy multidimensional array?
  23. Dump a NumPy array into a csv file
  24. Python: find position of element in array
  25. numpy with python: convert 3d array to 2d
  26. Python: slicing a multi-dimensional array
  27. How to remove specific elements in a numpy array
  28. TypeError: list indices must be integers or slices, not str
  29. Add single element to array in numpy
  30. ‘list’ object has no attribute ‘shape’
  31. How to normalize a 2-dimensional numpy array in python less verbose?
  32. Make list of arrays in python
  33. Understanding NumPy’s einsum
  34. How to read a text file into a list or an array with Python
  35. How can I create an array/list of dictionaries in python?
  36. How to fix “TypeError: len() of unsized object”
  37. TypeError: Invalid dimensions for image data when plotting array with imshow()
  38. Numpy.dot TypeError: Cannot cast array data from dtype(‘float64’) to dtype(‘S32’) according to the rule ‘safe’
  39. How to get the unit vector from a numpy array
  40. Using numpy to build an array of all combinations of two arrays
  41. Efficiently sorting a numpy array in descending order?
  42. How to plot an array in python?
  43. How to create a numpy array of all True or all False?
  44. Removing nan values from an array
  45. Root mean square of a function in python
  46. What are the differences between numpy arrays and matrices? Which one should I use?
  47. Numpy – add row to array
  48. Is there any numpy group by function?
  49. Mean Squared Error in Numpy?
  50. How to normalize a NumPy array to within a certain range?
  51. numpy array concatenation error: 0-d arrays can’t be concatenated
  52. Convert a 1D array to a 2D array in numpy
  53. inverting image in Python with OpenCV
  54. How to print the full NumPy array, without truncation?
  55. Sorting arrays in NumPy by column
  56. How is Python’s List Implemented?
  57. How can I reverse a list in Python?
  58. How can I reverse a list in Python?
  59. What is the difference between Python’s list methods append and extend?
  60. Understanding slice notation
  61. How to convert list to string [duplicate]
  62. numpy max vs amax vs maximum
  63. What is the purpose of meshgrid in Python / NumPy?
  64. why numpy.ndarray is object is not callable in my simple for python loop
  65. numpy division with RuntimeWarning: invalid value encountered in double_scalars
  66. TypeError: ‘float’ object is not subscriptable
  67. Most efficient way to find mode in numpy array
  68. How do I read CSV data into a record array in NumPy?
  69. Print a list in reverse order with range()?
  70. Why is it string.join(list) instead of list.join(string)?
  71. numpy array concatenate: “ValueError: all the input arrays must have same number of dimensions”
  72. ‘DataFrame’ object has no attribute ‘sort’
  73. LinAlgError: Last 2 dimensions of the array must be square
  74. What does axis in pandas mean?
  75. Understanding slice notation
  76. Add list to set?
  77. How to remove an element from a list by index
  78. Apply function to each element of a list
  79. How do I calculate percentiles with python/numpy?
  80. AttributeError: ‘list’ object has no attribute ‘replace’ when trying to remove character
  81. What does ‘index 0 is out of bounds for axis 0 with size 0’ mean?
  82. Saving a Numpy array as an image
  83. NumPy List Comprehension Syntax
  84. Linear regression with matplotlib / numpy
  85. What does the “x for x in” syntax mean?
  86. Numpy extract submatrix
  87. Matplotlib: TypeError: can’t multiply sequence by non-int of type ‘numpy.float64’
  88. Create nice column output in python
  89. How to add column to numpy array
  90. How to fix ‘RuntimeWarning: divide by zero encountered in double_scalars’
  91. IndexError: index 2 is out of bounds for axis 0 with size 2
  92. Coalesce values from 2 columns into a single column in a pandas dataframe
  93. Compute a confidence interval from sample data
  94. convert csv file to list of dictionaries
  95. How do I compute derivative using Numpy?
  96. ImportError: cannot import name NUMPY_MKL
  97. How to split elements of a list?
  98. Find object in list that has attribute equal to some value (that meets any condition)
  99. Iterating through list of list in Python
  100. Get difference between two lists

Leave a Comment