How to apply meta-tables on Lua functions in order to enable custom operators between them?

Question

I'm trying to set a meta-table on specific functions to enable a custom operator for functional composition between them.

The reason I want to introduce a composition operator in my library is because I don't want to nest the argument parentheses one into another like this: f(g(h(x))) but rather: f * g * h(x), or something alike that doesn't require nesting parentheses.

So far I had two approaches to achieve my goal:

1. Attempt to set a __mul meta-method for Lua functions, I'm not sure how exactly.

2. Re-define all the compositable functions as callable functionally-composite tables via meta-tables, by providing meta-methods for __call and __mul.

I wrote an experimental (and a working) implementation for the 2nd approach today (functionally-composite tables). But I don't think it's elegant, it's rather a hack with significant overhead in memory and processing for what is merely syntactic sugar. It's so complex that it has its own call-stack queue because Lua evaluates custom operators left-associative while functional composition is actually right-associative (I mistakenly thought so when I implemented it, and it turned out necessary for a whole different reason. See in the comments).

My current attempt to implement the 1st approach with functions and meta-tables looks like this:

local compositable =
{
    __mul = function(a, b)
        a(b)
    end
}

local function f(x) return x*x end
local function g(x) return -x end

setmetatable(f, compositable) -- Error, table expected, got function.
setmetatable(g, compositable) -- "

local result = f * g(4)

print(result) -- Expected result: -16

But this doesn't work, it appears that only Lua tables and strings are allowed to be set with a meta-tables in Lua.

Answer 1

While all Lua values can have metatables, setmetatable only sets metatables for tables. Tables and full userdata can have metatables set on individual values, while values of different types all share a metatable for each type. All strings have the same metatable, for example.

So you can't set a metatable for a function, but you can set a metatable for all functions. Only the debug library can do that, through debug.setmetatable. Again, this will apply to all functions everywhere.

Answer 2

As mentioned in the comments, you need to write (f * g)(x). Then the code below works.

debug.setmetatable(function()end,
    {__mul=function (f,g) return function (x) return f(g(x)) end end})

function f(x) return x*x end
function g(x) return -x end

print((f*g)(2))
print((g*f)(2))

This code is a bit wasteful because it creates a composite function in every call.