Sunday, May 1, 2011

Revealing the Scala magician's code: expression

Scala is truly magical. However, sometimes it is not easy to understand how it performs the magic. Below are some common questions and their answers.

Why some of expressions below work (can be evaluated and printed) but the other don't?



Math.min //doesn't work
Math.min _ //works
val f: (Int, Int)=>Int = Math.min //works

The first expression doesn't work because Math.min is a method, but a method is not a value in Scala. The second expression works because the underscore asks Scala to convert the method to a function, which is indeed a value in Scala. The third expression also works because when Scala is expecting a function value from the expression but finds a method, it will convert it to a function automatically.

Why some of expressions below work but the other don't?



List(2, 3, 5) foreach println //works
List(2, 3, 5) foreach println(_) //doesn't work
List(2, 3, 5) foreach (println(_)) //works

The first one works because foreach is expecting a function, while println (of the Predef object which has been imported automatically) is a method, not a function, but as mentioned above Scala will convert it into a function automatically because a function is expected. So it works.
The second one is trying to create an anonymous function:

List(2, 3, 5) foreach ((x)=>println(x))

However, in Scala only a "top level" expression can be a function. In this case, println(_) is only a "term" in an expression (foreach is the operator) but not a top level expression, so the compiler won't try to make it an anonymous function. Instead, it will search further to find the first enclosing top level expression (in this case, the whole expression you entered) and turn it into an anonymous function:

(x)=>List(2, 3, 5) foreach println(x)

But then the type of x can't be inferred, so it is an error. Also, println(x) returns a value of (), the only value of the class Unit, which is not what foreach wants anyway (a function taking an Int).
With this knowledge, you can see why the third expression works:

List(2, 3, 5) foreach (println(_)) //works

This is because with the parentheses, a top level expression is expected, so Scala will make println(_) an anonymous function:

List(2, 3, 5) foreach ((x)=>println(x)) //works

Why some of expressions below work but the other don't?



List(2, 3, 5) foreach (println("hi"+_)) //doesn't works
List(2, 3, 5) foreach (println "hi"+_) //doesn't works
List(2, 3, 5) foreach (Predef println "hi"+_) //works

The first expression doesn't work because the first top level expression found is "hi"+_ due to the parentheses. So, Scala will treat it as:

List(2, 3, 5) foreach (println((x)=>"hi"+x)) //doesn't works

So you're printing a function to the console and returning a unit value () to foreach. In order to fix the problem, you may try to get rid of the parentheses so "hi"+_ is no longer a top level expression:

List(2, 3, 5) foreach (println "hi"+_)

The problem is that Scala will now try to parse:

println "hi"+_

as:

expr1 op1 expr2 op2 ...

println is assumed to be an expression instead of a prefix operator because only !, ~, +, - can be prefix operators. So, println will be treated as an expression while the String "hi" will be treated as an operator, which is obviously incorrect.
To fix this problem, you can provide a real object as expr1, which is the Predef object:

List(2, 3, 5) foreach (Predef println "hi"+_)

Note that there are two operators: println and +. Because all symbolic operators have higher precedence than identifier operators, + will be applied first.
So, the first enclosing top level expression is turned into an anonymous function:

List(2, 3, 5) foreach ((x)=>Predef println "hi"+x)

Because in Scala "e1 op e2" is treated as e1.op(e2), the code is treated as:

List(2, 3, 5) foreach ((x)=>Predef.println("hi".+(x)))

4 comments:

  1. Good explanations on errors I (a Scala newbie) am likely to make... Not only you show what doesn't work, but you explain why, and how to fix that. Excellent. I particularly appreciate the function _ trick, it explains the error message which always seemed obscure.
    Note: compiling your code to see what error messages I get on the "wrong" code, I also got deprecation warnings on "good" code (I always compile -deprecation -Xmigration flags as we often find obsolete code on the Net...).
    warning: object Math in package scala is deprecated: use scala.math package instead
    (no preview of comments =>code tag can be ignored...)
    Applying the given advice removes the warning. It doesn't depreciate the value of your article, though.

    ReplyDelete
  2. Hi PhiLho,
    I am glad that you like my article. Also thanks to pointing out the warning! Just like you I also use the -deprecation option. I guess that you're using a more recent version of Scala than I. That's why I didn't see the warning :-)
    BTW, you may want to use the Scala interpreter to experiment with the code. Just run "scala" and type the code line by line. It's a great way to get quick feedback.

    ReplyDelete
  3. Hi Kent,
    thanks for this explanation especially the step by step discussion and what the Scala compiler generates from the code.
    Just one thing where I hesitated a bit while reading, was the term “top level” expression.
    Level in which sense, had been a question in my mind. What qualifies an expression to be a top level expression?
    Apart from that your post is worth to be bookmarked. :-)
    Thanks for that.

    ReplyDelete
  4. you explained the concept very good, keep posting the good stuff . thanks

    ReplyDelete