How to include and execute Go from within an Elixir Phoenix application

Question

I would like to execute some performance heavy tasks within an Elixir Phoenix application. Rather than sending the data elsewhere (external service), processing it, and returning it to the Phoenix app, I'd like to process it in place. Go has a lot of data science-y packages and so I'd like to use those within the Phoenix application. How can I do this?

I've researched and discovered Rustler (Rust) which uses NIFs. I could not find a Go variant or similar.

Answer 1

If you can, do not use NIFs, as panic in such code can cause your whole VM to crash. Instead BEAM has few different options for FFI:

• IMHO simplest one - Ports which in simple words is running external process which you can communicate via STDIO
• erl_interface which is Port, but with some utilities for simplifying communication
• Port Drivers which are again, Ports, with even more defined API
• C Nodes (despite the name, do not need to be written in C) are something that "simulates" being Distributed Erlang node to which you can connect like to any other node and communicate with it by exchanging messages
• NIF which offers minimal overhead, but as it was stated earlier, you should be really careful as it can bring whole VM down if done improperly (it will probably require some Cgo from you)

Answer 2

Use System.cmd/3 to spawn the external Go process.

I frankly doubt there are any native bindings to Go available.