Julia macro expansion order

Question

I would like to write a macro @unpack t which takes an object t and copies all its fields into local scope. For example, given

immutable Foo
    i::Int
    x::Float64
end
foo = Foo(42,pi)

the expression @unpack foo should expand into

i = foo.i
x = foo.x

Unfortunately, such a macro cannot exist since it would have to know the type of the passed object. To circumvent this limitation, I introduce a type-specific macro @unpackFoo foo with the same effect, but since I'm lazy I want the compiler to write @unpackFoo for me. So I change the type definition to

@unpackable immutable Foo
    i::Int
    x::Float64
end

which should expand into

immutable Foo
    i::Int
    x::Float64
end
macro unpackFoo(t)
    return esc(quote
        i = $t.i
        x = $t.x
    end)    
end

Writing @unpackable is not too hard:

macro unpackable(expr)
    if expr.head != :type
        error("@unpackable must be applied on a type definition")
    end

    name = isa(expr.args[2], Expr) ? expr.args[2].args[1] : expr.args[2]
    fields = Symbol[]
    for bodyexpr in expr.args[3].args
        if isa(bodyexpr,Expr) && bodyexpr.head == :(::)
            push!(fields,bodyexpr.args[1])
        elseif isa(bodyexpr,Symbol)
            push!(fields,bodyexpr)
        end
    end

    return esc(quote
        $expr
        macro $(symbol("unpack"*string(name)))(t)
            return esc(Expr(:block, [:($f = $t.$f) for f in $fields]...))
        end
    end)
end

In the REPL, this definition works just fine:

julia> @unpackable immutable Foo
           i::Int
           x::Float64
       end

julia> macroexpand(:(@unpackFoo foo))
quote 
    i = foo.i
    x = foo.x
end

Problems arise if I put the @unpackFoo in the same compilation unit as the @unpackable:

julia> @eval begin
       @unpackable immutable Foo
           i::Int
           x::Float64
       end
       foo = Foo(42,pi)
       @unpackFoo foo
       end
ERROR: UndefVarError: @unpackFoo not defined

I assume the problem is that the compiler tries to proceed as follows

1. Expand @unpackable but do not parse it.
2. Try to expand @unpackFoo which fails because the expansion of @unpackable has not been parsed yet.
3. If we wouldn't fail already at step 2, the compiler would now parse the expansion of @unpackable.

This circumstance prevents @unpackable from being used in a source file. Is there any way of telling the compiler to swap steps 2. and 3. in the above list?

---

The background to this question is that I'm working on an iterator-based implementation of iterative solvers in the spirit of https://gist.github.com/jiahao/9240888. Algorithms like MinRes require quite a number of variables in the corresponding state object (8 currently), and I neither want to write state.variable every time I use a variable in e.g. the next() function, nor do I want to copy all of them manually as this bloats up the code and is hard to maintain. In the end, this is mainly an exercise in meta-programming though.

Answer 1

Firstly, I would suggest writing this as:

immutable Foo
  ...
end

unpackable(Foo)

where unpackable is a function which takes the type, constructs the appropriate expression and evals it. There are a couple of advantages to this, e.g. that you can apply it to any type without it being fixed at definition time, and the fact that you don't have to do a bunch of parsing of the type declaration (you can just call fieldnames(Foo) == [:f, :i] and work with that).

Secondly, while I don't know your use case in detail (and dislike blanket rules) I will warn that this kind of thing is frowned upon. It makes code harder to read because it introduces a non-local dependency; suddenly, in order to know whether x is a local or global variable, you have to look up the definition of a type in a whole different file. A better, and more general, approach is to explicitly unpack variables, and this is available in MacroTools.jl via the @destruct macro:

@destruct _.(x, i) = myfoo
# now we can use x and i

(You can destruct nested data structures and indexable objects too, which is nice.)

To answer your question: you're essentially right about how Julia runs code (s/parse/evaluate). The whole block is parsed, expanded and evaluated together, which means in your example you're trying to expand @unpackFoo before it's been defined.

However, when loading a .jl file, Julia evaluates blocks in the file one at a time, rather than all at once.

This means that you can happily write a file like this:

macro foo()
  :(println("hi"))
end

@foo()

and run julia foo.jl or include("foo.jl") and it will run fine. You just can't have a macro definition and its use in the same block, as in your begin block above.