Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Efficient way of horizontal concatenation without tiling

I have two (large) arrays. For illustration purposes I'm using a simple example below:

In [14]: arr1 = np.arange(32*512).reshape(32, 512)
In [15]: arr2 = np.arange(512).reshape(1, 512)

And I wanted to do a horizontal concatenation of these arrays (i.e. concatenation along axis 1). I came up with the following approach to achieve this:

In [16]: np.hstack([arr1, np.tile(arr2, (arr1.shape[0], 1))]).shape
Out[16]: (32, 1024)

This works as intended. However, I would like to know whether there are any other efficient ways of doing this concatenation without using numpy.tile. I'm afraid I would blow-up my memory requirements since the arrays are really huge.

If it's possible avoid this duplication of rows (to match the dimensions of arr1), maybe using broadcasting, then it'd be great!


P.S. The reason why I want to avoid this copying is because of linear growth of memory requirements:

In [20]: arr2.nbytes
Out[20]: 4096

In [19]: np.tile(arr2, (arr1.shape[0], 1)).nbytes
Out[19]: 131072

In [22]: arr1.shape[0] * arr2.nbytes
Out[22]: 131072
like image 623
kmario23 Avatar asked Dec 18 '25 06:12

kmario23


1 Answers

You can preallocate and use broadcasting but it won't save much (I'd expect peak memory usage to go down by roughly a quarter):

arr1 = np.arange(32*512).reshape(32, 512)
arr2 = np.arange(512).reshape(1, 512)
out = np.empty((32, 1024), arr1.dtype)
out[:, :512] = arr1
out[:, 512:] = arr2
out
#array([[    0,     1,     2, ...,   509,   510,   511],
#       [  512,   513,   514, ...,   509,   510,   511],
#       [ 1024,  1025,  1026, ...,   509,   510,   511],
#       ...,
#       [14848, 14849, 14850, ...,   509,   510,   511],
#       [15360, 15361, 15362, ...,   509,   510,   511],
#       [15872, 15873, 15874, ...,   509,   510,   511]])
like image 90
Paul Panzer Avatar answered Dec 19 '25 22:12

Paul Panzer



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!