How to efficiently apply a function element-wise to multiple NumPy arrays ?

Introduction

When working with large datasets in Python, it is often necessary to apply functions element-wise to multiple NumPy arrays. This means that the function will be applied to each element of the arrays individually, resulting in a new array with the same shape as the original ones.

In this guide, we will explore different methods for efficiently applying functions element-wise to multiple NumPy arrays in Python.

Using numpy.ufunc.reduce() method

The numpy.ufunc.reduce() method is an efficient way to perform element-wise operations on multiple NumPy arrays in Python.

To use this method, you need to first import the NumPy library in your code. Once imported, you can use the reduce() method on any NumPy universal function (ufunc).

To demonstrate its functionality, let's define three arrays of the same size and dimensions.

import numpy as np

np.random.seed(42) # Generate the same random numbers

A = np.random.randint(0,10,(1,5))
B = np.random.randint(0,10,(1,5))
C = np.random.randint(0,10,(1,5))

print(A)
print(B)
print(C)

The provided code will generate three arrays in the following order

[[6 3 7 4 6]]
[[9 2 6 7 4]]
[[3 7 7 2 5]]

Element-wise addition between multiple NumPy arrays

np.add.reduce([A,B,C])

In this example, the add() ufunc is applied to each element of both arrays and the results are then added together using the reduce() method:

array([[18, 12, 20, 13, 15]])

One advantage of using numpy.ufunc.reduce() is its efficiency. It can handle large arrays and perform the operations in a much faster way than using traditional for loops.

Element-wise multiplication between multiple NumPy arrays

If we want to perform an element-wise multiplication between them and reduce the result to a single array, we can simply use the np.multiply.reduce() function as follows:

np.multiply.reduce([A,B,C])

The provided code will generate

array([[162,  42, 294,  56, 120]])

Element-wise maximum between multiple NumPy arrays

Let's take a simple example to understand how this works:

import numpy as np

np.random.seed(42) # Generate the same random numbers

A = np.random.randint(0,10,(4,5)) # dataset 1

B = np.random.randint(0,10,(4,5)) # dataset 2

C = np.random.randint(0,15,(4,5)) # dataset 2

Output A

[[6 3 7 4 6]
 [9 2 6 7 4]
 [3 7 7 2 5]
 [4 1 7 5 1]]

Output B

[[4 0 9 5 8]
 [0 9 2 6 3]
 [8 2 4 2 6]
 [4 8 6 1 3]]

Output C

[[ 8 11 13  1  9]
 [ 8  9  4  1  3]
 [11 14 11  6 11]
 [12  7 14  2 13]]

One common operation is finding the element-wise maximum between two or more arrays. The basic syntax for this operation is

np.maximum.reduce([A,B,C])

The shape of the output array will be the same as that of the input arrays.

array([[ 8, 11, 13,  5,  9],
       [ 9,  9,  6,  7,  4],
       [11, 14, 11,  6, 11],
       [12,  8, 14,  5, 13]])

Creating your custom ufunc

Using the arrays mentioned previously, let's proceed with the following:

import numpy as np

np.random.seed(42) # Generate the same random numbers

A = np.random.randint(0,10,(4,5)) # dataset 1

B = np.random.randint(0,10,(4,5)) # dataset 2

C = np.random.randint(0,15,(4,5)) # dataset 2

Let's create our own basic function that accepts two numpy arrays as inputs and performs a simple element-wise operation:

def myfunc(x1,x2):
    data = np.add(x1,x2) + 1
    return data

Applying this function to two arrays

myfunc(A,B)

yields the following result:

array([[11,  4, 17, 10, 15],
       [10, 12,  9, 14,  8],
       [12, 10, 12,  5, 12],
       [ 9, 10, 14,  7,  5]])

To apply this function element-wise to multiple arrays using the reduce() method, we need to first transform it into a ufunc:

my_ufunc = np.frompyfunc(myfunc, 2, 1)

Now we can do:

my_ufunc.reduce([A,B,C])

which will give the following result:

array([[20, 16, 31, 12, 25],
       [19, 22, 14, 16, 12],
       [24, 25, 24, 12, 24],
       [22, 18, 29, 10, 19]], dtype=object)

References