-
Notifications
You must be signed in to change notification settings - Fork 0
Home
(Note: most of the content/expertise is thanks to @fcard)
Goals:
-
Teach through minimal targeted functional/useful/non-abstract examples (e.g.
@swapor@assert) that introduce concepts in suitable contexts -
prefer Q&A style (or Socratic Dialogue style) -- it's the best for learning!
-
Prefer to let the code illustrate/demonstrate the concepts over paragraphs of explanation
-
Avoid linking 'required reading' to other pages -- it interrupts the narrative
-
Present things in a sensible order that will making learning easiest
ToDo:
- more simple examples
- exercises!
- "what would the following macro do?"
- design a macro to do this
- diagnose and fix the following
- etc.
- split out more pages
Resources:
julialang.org
wikibook (@Cormullion)
5 layers (Leah Hanson)
SO-Doc Quoting (@TotalVerb)
SO-Doc -- Symbols that are not legal identifiers (@TotalVerb)
SO: What is a Symbol in Julia (@StefanKarpinski)
Discourse thread (@p-i-)
http://stackoverflow.com/documentation/julia-lang/1945/metaprogramming#t=201612021739596755987
Most of the material has come from the discourse channel, most of that has come from fcard... please prod me if I had forgotten attributions.
julia> mySymbol = Symbol("myName") # or 'identifier'
:myName
julia> myName = 42
42
julia> mySymbol |> eval # 'foo |> bar' puts output of 'foo' into 'bar', so 'bar(foo)'
42
julia> :( $mySymbol = 1 ) |> eval
1
julia> myName
1Passing flags into functions:
function dothing(flag)
if flag == :thing_one
println("did thing one")
elseif flag == :thing_two
println("did thing two")
end
end
julia> dothing(:thing_one)
did thing one
julia> dothing(:thing_two)
did thing twoA hashkey example:
number_names = Dict{Symbol, Int}()
number_names[:one] = 1
number_names[:two] = 2
number_names[:six] = 6(Advanced)
(@fcard) :foo a.k.a. :(foo) yields a symbol if foo is a valid identifier, otherwise an expression.
# NOTE: Different use of ':' is:
julia> :mySymbol = Symbol('hello world')
#(You can create a symbol with any name with Symbol("<name>"),
# which lets us create such gems as:
julia> one_plus_one = Symbol("1 + 1")
Symbol("1 + 1")
julia> eval(one_plus_one)
ERROR: UndefVarError: 1 + 1 not defined
...
julia> valid_math = :($one_plus_one = 3)
:(1 + 1 = 3)
julia> one_plus_one_plus_two = :($one_plus_one + 2)
:(1 + 1 + 2)
julia> eval(quote
$valid_math
@show($one_plus_one_plus_two)
end)
1 + 1 + 2 = 5
...Basically you can treat Symbols as lightweight strings. That's not what they're for, but you can do it, so why not. Julia's Base itself does it, print_with_color(:red, "abc") prints a red-colored abc .
(Almost) everything in Julia is an expression, i.e. an instance of Expr, which will hold an AST.
# when you type ...
julia> 1+1
2
# Julia is doing: eval(parse("1+1"))
# i.e. First it parses the string "1+1" into an `Expr` object ...
julia> ast = parse("1+1")
:(1 + 1)
# ... which it then evaluates:
julia> eval(ast)
2
# An Expr instance holds an AST (Abstract Syntax Tree). Let's look at it:
julia> dump(ast)
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Int64 1
typ: Any
# TRY: fieldnames(typeof(ast))
julia> :(a + b*c + 1) ==
parse("a + b*c + 1") ==
Expr(:call, :+, :a, Expr(:call, :*, :b, :c), 1)
trueNesting Exprs:
julia> dump( :(1+2/3) )
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol /
2: Int64 2
3: Int64 3
typ: Any
typ: Any
# Tidier rep'n using s-expr
julia> Meta.show_sexpr( :(1+2/3) )
(:call, :+, 1, (:call, :/, 2, 3))julia> blk = quote
x=10
x+1
end
quote # REPL[121], line 2:
x = 10 # REPL[121], line 3:
x + 1
end
julia> blk == :( begin x=10; x+1 end )
true
# Note: contains debug info:
julia> Meta.show_sexpr(blk)
(:block,
(:line, 2, Symbol("REPL[121]")),
(:(=), :x, 10),
(:line, 3, Symbol("REPL[121]")),
(:call, :+, :x, 1)
)
# ... unlike:
julia> noDbg = :( x=10; x+1 )
quote
x = 10
x + 1
end... so quote is functionally the same but provides extra debug info.
(*) TIP: Use let to keep x within the block
Expr(:quote, x) is used to represent quotes within quotes.
Expr(:quote, :(x + y)) == :(:(x + y))
Expr(:quote, Expr(:$, :x)) == :(:($x))
QuoteNode(x) is similar to Expr(:quote, x) but it prevents interpolation.
eval(Expr(:quote, Expr(:$, 1))) == 1
eval(QuoteNode(Expr(:$, 1))) == Expr(:$, 1)
:(foo) means "don't look at the value, look at the expression"
$foo means "change the expression to its value"
:($(foo)) == foo.
$(:(foo)) is an error.
$(...) isn't an operation and doesn't do anything by itself, it's an "interpolate this!" sign that the quoting syntax uses. i.e. It only exists within a quote.
No!
$foo is exchanged for the compile-time value
eval(foo) means to do that at runtime
eval will occur in the global scope
interpolation is local
eval(:<expr>) should return the same as just <expr> (assuming <expr> is a valid expression in the current global space)
eval(:(1 + 2)) == 1 + 2
eval(:(let x=1; x + 1 end)) == let x=1; x + 1 end
Ready? :)
# let's try to make this!
julia> x = 5; @show x;
x = 5macro log(x)
:(
println( "Expression: ", $(string(x)), " has value: ", $x )
)
end
u = 42
f = x -> x^2
@log(u) # Expression: u has value: 42
@log(42) # Expression: 42 has value: 42
@log(f(42)) # Expression: f(42) has value: 1764
@log(:u) # Expression: :u has value: u5 layers (Leah Hanson) <-- explains how Julia takes source code as a string, tokenizes it into an Expr-tree (AST), expands out all the macros (still AST), lowers (lowered AST), then converts into LLVM (and beyond -- at the moment we don't need to worry what lies beyond!)
Q: code_lowered acts on functions. Is it possible to lower an Expr?
A: yup!
# function -> lowered-AST
julia> code_lowered(*,(String,String))
1-element Array{LambdaInfo,1}:
LambdaInfo template for *(s1::AbstractString, ss::AbstractString...) at strings/basic.jl:84
# Expr(i.e. AST) -> lowered-AST
julia> expand(:(x ? y : z))
:(begin
unless x goto 3
return y
3:
return z
end)
julia> expand(:(y .= x.(i)))
:((Base.broadcast!)(x,y,i))
# 'Execute' AST or lowered-AST
julia> eval(ast)If you want to only expand macros you can use macroexpand:
# AST -> (still nonlowered-)AST but with macros expanded:
julia> macroexpand(:(@show x))
quote
(Base.println)("x = ",(Base.repr)(begin # show.jl, line 229:
#28#value = x
end))
#28#value
end...which returns a non-lowered AST but with all macros expanded.
esc(x) returns an Expr that says "don't apply hygiene to this", it's the same as Expr(:escape, x). Hygiene is what keeps a macro self-contained, and you esc things if you want them to "leak". e.g.
macro swap(p, q)
quote
tmp = $(esc(p))
$(esc(p)) = $(esc(q))
$(esc(q)) = tmp
end
end
x,y = 1,2
@swap(x,y)
println(x,y) # 2 1$ allows us to 'escape out of' the quote.
So why not simply $p and $q? i.e.
# FAIL!
tmp = $p
$p = $q
$q = tmpBecause that would look first to the macro scope for p, and it would find a local p i.e. the parameter p (yes, if you subsequently access p without esc-ing, the macro considers the p parameter as a local variable).
So $p = ... is just a assigning to the local p. it's not affecting whatever variable was passed-in in the calling context.
Ok so how about:
# Almost!
tmp = $p # <-- you might think we don't
$(esc(p)) = $q # need to esc() the RHS
$(esc(q)) = tmpSo esc(p) is 'leaking' p into the calling context. "The thing that was passed into the macro that we receive as p"
julia> macro swap(p, q)
quote
tmp = $p
$(esc(p)) = $q
$(esc(q)) = tmp
end
end
@swap (macro with 1 method)
julia> x, y = 1, 2
(1,2)
julia> @swap(x, y);
julia> @show(x, y);
x = 2
y = 1
julia> macroexpand(:(@swap(x, y)))
quote # REPL[34], line 3:
#10#tmp = x # REPL[34], line 4:
x = y # REPL[34], line 5:
y = #10#tmp
end As you can see tmp gets the hygiene treatment #10#tmp, whereas x and y don't.
Julia is making a unique identifier for tmp, something you can manually do with gensym, ie:
julia> gensym(:tmp)
Symbol("##tmp#270")But: There is a gotcha:
julia> module Swap
export @swap
macro swap(p, q)
quote
tmp = $p
$(esc(p)) = $q
$(esc(q)) = tmp
end
end
end
Swap
julia> using Swap
julia> x,y = 1,2
(1,2)
julia> @swap(x,y)
ERROR: UndefVarError: x not definedAnother thing julia's macro hygiene does is, if the macro is from another module, it makes any variables (that were not assigned inside the macro's returning expression, like tmp in this case) globals of the current module, so $p becomes Swap.$p, likewise $q -> Swap.$q.
In general, if you need a variable that is outside the macro's scope you should esc it, so you should esc(p) and esc(q) regardless if they are on the LHS or RHS of a expression, or even by themselves.
people have already mentioned gensyms a few times and soon you will be seduced by the dark side of defaulting to escaping the whole expression with a few gensyms peppered here and there, but... Make sure to understand how hygiene works before trying to be smarter than it! It's not a particularly complex algorithm so it shouldn't take too long, but don't rush it! Don't use that power until you understand all the ramifications of it... (@fcard)
(@Ismael-VC)
"until loop"
macro until(condition, block)
quote
while ! $condition
$block
end
end |> esc
end
julia> i=1; @until( i==5, begin; print(i); i+=1; end )
1234
(@fcard) |> is controversial, however. I am surprised a mob hasn't come to argue yet. (maybe everyone is just tired of it). There is a recommendation of having most if not all of the macro just be a call to a function, so:
macro until(condition, block)
esc(until(condition, block))
end
function until(condition, block)
quote
while !$condition
$block
end
end
end...is a safer alternative.
###Why is quote ... end |> esc bad? (thanks @TotalVerb)
Say our macro does macro M(p,q); esc( :($p+$q) ) ; end; @M(1,2) #gives 3. Note it is also escaping the + function, and it is possible that in the calling scope the user is being a cheeky monkey and occluding the system +(a,b) function with their own, e.g. let x+y = x-y; @M(1,2); end will give -1 not 1!
Bonus Question: Why can't we do :( esc($p+$q) )
##@fcard's simple macro challenge
Task: Swap the operands, so swaps(1/2) gives 2.00 i.e. 2/1
macro swaps(e)
e.args[2:3] = e.args[3:-1:2]
e
end
@swaps(1/2)
2.00More macro challenges from @fcard here
http://docs.julialang.org/en/release-0.5/manual/metaprogramming/#building-an-advanced-macro
macro assert(ex)
return :( $ex ? nothing : throw(AssertionError($(string(ex)))) )
endQ: Why the last $?
A: It interpolates, i.e. forces Julia to eval that string(ex) as execution passes through the invocation of this macro.
i.e. If you just run that code it won't force any evaluation. But the moment you do assert(foo) Julia will invoke this macro replacing its 'AST token/Expr' with whatever it returns, and the $ will kick into action.
(@fcard) I don't think there is anything technical keeping {} from being used as blocks, in fact one can even pun on the residual {} syntax to make it work:
julia> macro c(block)
@assert block.head == :cell1d
esc(quote
$(block.args...)
end)
end
@c (macro with 1 method)
julia> @c {
print(1)
print(2)
1+2
}
123*(unlikely to still work if/when the {} syntax is repurposed)
So first Julia sees the macro token, so it will read/parse tokens until the matching end, and create what? An Expr with .head=:macro or something? Does it store "a+1" as a string or does it break it apart into :+(:a, 1)? How to view?
?
(@fcard) In this case because of lexical scope, a is undefined in @Ms scope so it uses the global variable...
I actually forgot to escape the flipplin' expression in my dumb example, but the "only works within the same module" part of it still applies.
julia> module M
macro m()
:(a+1)
end
end
M
julia> a = 1
1
julia> M.@m
ERROR: UndefVarError: a not definedThe reason being that, if the macro is used in any module other than the one it was defined in, any variables not defined within the code-to-be-expanded are treated as globals of the macro's module.
julia> macroexpand(:(M.@m))
:(M.a + 1)Challenge:
# Want to be able to do:
@if cond expr
# rather than the clumsy
if cond; expr; end
# .. or the ugly short-circuit
cond && exprProblem (if is a reserved keyword):
julia> macro if() end`
`syntax: invalid name "if"`
Solution:
# @TotalVerb's solution:
macro _if()
esc(:if)
end
macro (@_if)() end# @IsmaelVC's solution:
julia> @eval macro $(:if)(cond, then::Symbol, block)
if then != :then
error()
end
quote
if $cond
$block
end
end |> esc
end
@if (macro with 1 method)
julia> @if true then :bar
:barSee @eval
macro ===(x, y)
quote
$(esc(x)) = $(esc(y))
println($(Meta.quot(x)), " = ", $(esc(y)))
end
end
macro ===(x, y, s, t...)
quote
@=== $(esc(x)) $(esc(y))
@=== $(esc(s)) $(map(esc, t)...)
end
end
julia> @=== a 1 b 2
a = 1
b = 2