Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How should I use an array (or tuple) as key and values for a Numba typed dictionary?

Tags:

numba

I have the following code that attempts to store a key-value pair into a numba dictionary. The official page of Numba says that the new typed dictionary supports array as the key, but I could not get it to work. The error message says that the key cannot be hash. Any idea how to get this working?

In [7]: from numba.typed import Dict 
   ...: from numba import types 
   ...: import numpy as np        

In [15]: dd = Dict.empty(key_type=types.int32[::1], value_type=types.int32[::1],)                                                                                                                                  

In [16]: key = np.asarray([1,2,3], dtype=np.int32)                                                                                                                                                                 

In [17]: dd[key] = key   

Error Message:

TypingError: Failed in nopython mode pipeline (step: nopython frontend) Unknown attribute 'hash' of type array(int32, 1d, C)

EDIT: I am probably missing something. I could use types.UniTuple in the interpreter (without the @jit decorator). However, when I put the following function into a script a.py and run it with command "python a.py", I got the UniTuple not found error.

@jit(nopython=True)
def go_fast2(date, starttime, id, tt, result): # Function is compiled and runs in machine code
    prev_record = Dict.empty(key_type=types.UniTuple(types.int64, 2),  value_type=types.UniTuple(types.int64, 3),)
    for i in range(1, length):
        key = np.asarray([date[i], id[i]], dtype=np.int64)
        thistt = tt[i]
        thistime = starttime[i]
        if key in prev_record:
            prev_time = prev_record[key][0]
            prev_tt = prev_record[key][1]
            prev_res = prev_record[key][2]
            if thistt == prev_tt and thistime - prev_time <= 30 * 1000 * 1000: # with in a 10 seconds window
                result[i] = prev_res + 1
            else:
                result[i] = 0
            prev_record[key] = np.asarray((thistime, thistt, result[i]), dtype=np.int64)
        else:
            result[i] = 0
            prev_record[key] = np.asarray((thistime, thistt, result[i]), dtype=np.int64)
    return 
like image 476
user1559897 Avatar asked Oct 18 '25 00:10

user1559897


1 Answers

The current documentation says that:

Acceptable key/value types include but are not limited to: unicode strings, arrays, scalars, tuples.

The wording does make it seem like you might be able to use an array as a key type, but that is incorrect as an array is not hashable because it is mutable. It wouldn't work with a standard python dict either. You could convert the array to a tuple and that would work:

dd = Dict.empty(
    key_type=types.UniTuple(types.int64, 3), 
    value_type=types.int64[::1],)
key = np.asarray([1,2,3], dtype=np.int64)
dd[tuple(key)] = key

Note that the int32 dtype you were using previously won't work on 64-bit machines since the tuple of int32s will be automatically converted to int64 when calling tuple() on the array.

The other issue is that a tuple has a fixed size, so you couldn't use arrays of arbitrary size as the key.

like image 88
JoshAdel Avatar answered Oct 22 '25 05:10

JoshAdel



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!