I have been doing some extensive searching, however I have found no interface between NumPy and NodeJS. Is there a particular reason writing a wrapper or NodeJS Addon to for NumPy is a bad idea?
The main value add from being able to use NumPy in my case is extremely fast linear algebra operations on large dimensional matrices.
I know of some packages such as furiousJS and ndarray that provide multidimensional array operations, however NumPy is still a clear winner.
If there is no reason why a Numpy Wrapper/Addon would be an utter disaster, I am compelled to write one.
Yes, one could write a Node.js add-on which exposes underlying NumPy C APIs; however, whether such an undertaking is advisable is another matter for the following reasons:
- Underlying NumPy C APIs are designed to work well with Python objects, thus entailing procedures for marshaling and unmarshaling data. These procedures do not generally map nicely to JavaScript data types. Hence, for many C APIs, any add-on will need to wrap the underlying interfaces in order to massage JavaScript data types into palatable form. This may entail considerable work, will be difficult to automate, and may not afford comparable performance.
- Certain Python data types have no equivalent in JavaScript (e.g., float16, complex, etc). Any attempt to provide support for these data types will need to be carefully considered in order to ensure performance. Complex number arrays are a good example of such consideration, particularly with regard to memory layout and return value shape.
- Even if one is able to automate add-on generation, I would argue that one will want to maintain the JavaScript idiom (e.g., naming conventions, argument lists versus options objects, etc). This, again, entails wrapper APIs, which will need to be written and maintained as NumPy and JavaScript co-evolve.
- Granted, the development of NumPy entailed hundreds (thousands?) of hours of work spread over many years, and one would be right to be leery about reinventing the wheel. However, if performance is what you want, along with consistency in idiom and aesthetics, one should also ask whether, instead of writing wrappers, one should implement comparable NumPy functionality, but built with JavaScript, Node.js, and the Web (including WebAssembly) in mind. Careful API design can allow one to not only exceed NumPy performance, but also unlock more ergonomic APIs. (Efforts are underway to this end.)
- Any add-on will have to make a choice: ship a custom NumPy distribution (i.e., source files) along with the add-on, or rely on the presence of an already installed NumPy distribution. The former negates the advantage of not having to maintain multiple sources of truth (i.e., one will need to devise mechanisms to avoid underlying API drift), while the latter will entail more rigid system dependency requirements (i.e., one will either need to require particular minimum versions, which may be incompatible with a user's existing installation, or one may have to branch based on the locally installed version via wrappers which handle any API consistencies between NumPy versions). In short, writing an add-on which interfaces directly with NumPy is not as straightforward as many might initially believe.
- JavaScript development has moved away from "kitchen sink" style libraries, particularly given the priority of small bundle sizes and installations. Given how NumPy is composed, a kitchen sink style library is the only form possible. Given how NumPy is implemented, if one decides to "carve up" NumPy into smaller packages, this will present a formidable task; at which point, we return to the question as to whether reinventing the wheel might just be a more efficient use of time.
In short, yes, writing an add-on is possible, but whether this is a good idea is a matter of debate. My view is, while writing an add-on which interfaces with underlying NumPy C APIs may fulfill a short term goal, the better long term solution is for JavaScript and Node.js developers to undertake the task of designing and implementing a NumPy for JavaScript in order to achieve better performance, better ergonomics, and a better user experience.
Disclaimer: I and others are actively developing stdlib, a standard library for JavaScript and Node.js, with an emphasis on scientific computing, which includes many NumPy-like APIs for working with and operating on multidimensional arrays.
Writing numpy wrapper for nodejs can be done. Numpy actually has core files written in c and it uses swig to enable creation of python wrapper. So it should be possible to do that. swig has also support for nodejs.
Menu
599
