Friday, January 24, 2014

Which Lisp Should I Learn?

I don't know why this keeps coming up lately, but it does. So, here we go:

My Recommendation

If you want to learn your first Lisp and already know something about the JVM[1], you should learn Clojure[2]. Otherwise, you should learn Scheme. Specifically, I recommend going the route that takes you through Racket[3], and possibly through SICP or HTDP.

If you absolutely, positively must, I guess go ahead and learn Common Lisp.

Why

Clojure the language, as opposed to the current, main, half-fused-with-JVM implementation, is cleaner and more consistent than Common Lisp, which should help you learn it more easily. I've gotten back talk about how there are lots more noobs learning Clojure, and as a result their libraries are in some disarray, and about the fact that the JVM is a sack of donkey balls you have to bite into every time you hit some sort of error[4], and about the general Clojure community pre-disposition to fashion trends. All of which may or may not be true, but I'm specifically talking about the language, not its ecosystem or stalwarts. Now granted, all of Racket, Clojure and Common Lisp

  1. are built out of s-expressions
  2. have defmacro

so depending on how much work you're willing to put in, you can do whatever the fuck you want in all of them[5]. However, in addition to knowing about prefix notation, and macros, and general Lisp program structure, here's an incomplete list of idiosyncrasies of Common Lisp that you have to commit to memory before you can be effective in it:

  • Functions and variables are in different namespaces, and each has constructs that deal with them explicitly. You need to use defvar/defun and let/flet depending on whether you're using functions or variables. If you're passing symbol names around, symbols that denote variables can be sent around as 'foo whereas symbols that denote functions should be sent around as #'foo. If you're trying to pass a function around in the variable foo, you need to (funcall foo arg) or possibly (apply foo args), rather than just (foo arg).
  • Most functions that deal with lists are functional, except the standard sort and the default mapcan, both of which destructively modify the list you pass them.
  • You can define methods for your classes, but can't easily use certain default names that are bound to top-level functions. Such as length, or the arithmetic primitives. Which is why you frequently see methods like duration-add or matrix-mult.
  • There are 7 commonly used equality operators, eq, eql, equal, equalp, string=, char= and =[6], each of which has mildly different, sometimes implementation-specific, behavior. Granted, because CL isn't a pure language, you need at least 2 of those, but 7 is still a bit much to have people memorize.
  • There are three different local binding mechanisms that you must decide between depending on whether you want to be able to refer to earlier symbols in the same binding set, or whether you want symbols to be able to refer to themselves. You use let/flet if you don't care, let* for variables where you want later bindings to be able to refer to earlier ones, and labels for functions where you want bindings to be able to refer to themselves or earlier bindings.
  • There are many, many implementations of Common Lisp. The popular ones at the moment are SBCL and CCL, but I've personally seen CLISP, ECL and Lispworks around too. More are available, and you might run into a lot of them in the wild. If you want to write portable code, you have to jump through some hoops. The implementation-specifics range from the finer points of equality operator behaviors, to the behavior of handler-case[7], to the types you can specialize on with defmethod, to the presence and behavior of threads, to the contexts in which you can pass streams around, to the names of various extension functions. For a small but representative example, take a look at what it takes to temporarily change your current directory in an implementation-independent way.
  • Indexing into different constructs takes different forms. You need nth for lists, aref for arrays and vectors, gethash for hashes and slot-value for CLOS instances[8].
  • You can't use the symbol t anywhere. No, not even local scopes. If you try, you'll get warnings or errors[9], because t is the top-level designated Truth symbol, even though anything other than NIL evaluates truthily in a boolean context.
  • A hash isn't a list, and a CLOS instance isn't anything like either. One way you'd like them to be similar is when you're traversing them. It seems fairly reasonable to expect map-likes to operate on hashes by treating them as a sequence of key/value pairs, and instances by treating their bound slots as key/value pairs. This is not how things work. If you want to map over instances that way, you need to do something like this and this. If you want to map over hashes, you either use the hilariously mis-named maphash[10] or some idiosyncratic piece of the loop DSL that lets you iterate over hash keys and hash values.
  • Common Lisp is case-insensitive. It takes whatever symbol input from you and upcases it internally. So foo-bar and foo-Bar both become FOO-BAR behind the scenes. This is usually not a huge problem, unless you try to interoperate cleanly with newer data standards like CSS or JSON. That leaves you fumbling with strings in situations where symbols and/or keywords really ought to do.

Like I said, this is a small sample. Just the stuff I thought of off the top of my head. I'm sure I could come up with more if I put a day or two into it. And I'm far from the most experienced Lisper out there, others would have more finer points for you, I'm sure. But that's half the problem with little issues like this; experienced Lispers completely forget about them. It's the newbs that have trouble cramming these things into their heads.

When I take a good look at that list, and then imagine the situations that led to each element, it's difficult to conclude that a wrong decision was made at any given point in time. Unfortunately, the sum of all of those potentially correct decisions is a giant system, the inherent rules of which look inconsistent if not outright hostile to human minds.

I don't know if Clojure solves all of them.

I've done very little work with it, for reasons entirely unrelated to the language. For all I know, when you get deep enough into it, you get to inconsistencies and/or restrictions which are worse than anything I've pointed out or could.

But do me a favor, if you're a CL user, either hop over to this web REPL, or install leiningen then hop into your local lein repl and type along here:

user=> (def thing [8 7 6 5 4 3 2 1])
#'user/thing
user=> (thing 0)
8
user=> (thing 3)
5
user=>(map (fn [n] (+ 3 n)) thing)
(11 10 9 8 7 6 5 4)
user=> (def thing {:a 1 :b 2 :c 3})
#'user/thing
user=> (thing :c)
3
user=> (thing :d 6)
6
user=> (thing :a 6)
1
user=> (map (fn [[k v]] (+ v 2)) thing)
(3 4 5)
user=> (def thing #{1 2 3 4 5}) ;; a set, in case you were wondering
#'user/thing
user=> (thing 3)
3
user=> (thing 0) 
nil
user=> (map (fn [a] (+ a 2)) thing)
(3 4 5 6 7)
user=> (def triple (fn [a] (* a 3)))
#'user/triple
user=> (triple 4)
12
user=> (map triple thing)
(3 6 9 12 15)
user=> (map (fn [a] (let [t (- (triple a) 5)] (* 2 t))) thing)
(-4 2 8 14 20)

Now think about how you would go about explaining to a novice programmer that it has to be more complicated than that.


Footnotes

1 - [back] - And don't have a strong dislike for it, obviously.

2 - [back] - Install it through Leiningen, which is available in the Debian repos in stable and unstable.

3 - [back] - Yes, I'm fully aware that the Racket guys are trying to push this "We're totally not Scheme" thing. They're close enough from an external perspective. Just don't tell Jay McCarthy I said so.

4 - [back] - Which is certainly true, but mildly preferable to the alternative as long as you're used to that sort of thing.

5 - [back] - Except that Clojure is apparently missing Reader macros, which I always thought were kind of half-assedly implemented in Common Lisp. For what I consider the full-ass version, take a look at how Haskell does it.

6 - [back] - Plus how many ever *-equal functions you define for your own classes.

7 - [back] - The Common Lisp answer to the problems that call for try/catch in other languages.

8 - [back] - For the last, you can also define your own selectors using :reader or :accessor declarations.

9 - [back] - Which specific warning or error depends on implementation.

10 - [back] - Because it's not very much like map. It returns nil and works by side-effect. Meaning that if you expect a sequence from it, you'll need to construct it yourself.

3 comments:

  1. I feel the criticism exposed here for Common Lisp is unfair. I kind of agree with the recommendations of Scheme and Racket, they are nice languages, but i don't really understand why they, and Common Lisp, should be replaced with "Clojure" as "the lisp to learn", considering, for example:

    - Clojure implements just a subset of what can be done with Scheme and CL.

    - The JVM (classes, java standard libraries) "percolates" through Clojure, and sooner or later the Clojurian will need to be familiar with them. So there are two languages to learn: Java (and it's OOP model) and Clojure.

    - There is much more syntax in Clojure than in the other Lisps.

    - Naming "Clojure" as a Lisp dialect is controversial on itself, for example, one of the main operators in ALL Lisp languages, "cons", has an entirely different meaning in Clojure. Also, Clojure isn't list-oriented, an essential difference from all other Lisps.

    In any case, regarding Common Lisp, your criticisms are fun but at the same time make me think that perhaps you have just used the language superficially, for the decisions involved in making Common Lisp the way as it is are more about practical use in real-world scenarios.

    Here are my take on each of your items regarding Common Lisp:

    >here's an incomplete list of idiosyncrasies of Common Lisp
    >Functions and variables are in different namespaces

    There is a nice saying that reads There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors". The nice thing about Common Lisp (and what makes it a "Lisp-2"), is precisely that functions, variables, classes, keywords and variables are in different namespaces. Thus, they don't collide with each other, thus, things are easy to name.

    Thus, if you have an object "foo", you have prefixes, to indicate Common Lisp if you're referring to the keyword "foo" (:foo), the variable binding foo (foo), function foo (#'foo), etc.

    It sounds complicated, but the advantage of being able to name things as needed and to refer to them unambigously is very nice in practice.

    >Most functions that deal with lists are functional, except the standard sort and the default mapcan, both of which destructively modify the list you pass them.

    Another important consideration is that Common Lisp wasn't created for purity or elegance for elegance's sake; it was created for using it on real-world problems, like for example verifying card transactions or auto-piloting the Deep Space 1 spaceship (for real.) Thus, **performance** is also a concern, and Common Lisp considers performance important. That's why it has, for example, (optional) static type declarations: to improve performance.

    That's the very same reason sorts ought to be destructive; because otherwise there would be a performance hit. If you want a non-destructive sort, it would be trivial to create your own.

    >Indexing into different constructs takes different forms. You need nth for lists, aref for arrays and vectors, gethash for hashes and slot-value for CLOS instances

    And thus, because of performance, you have functions that are more specialized than others. ELT is more general, and can operate in any sequence: arrays, non-circular lists. NTH is intended for lists only, and can work in circular-reference lists as well. AREF is specific for arrays. If you are, for example, using arrays instead of lists, chances are you're doing them for performance, and thus you would naturally use AREF.

    There is also an emphasis on readability -- if I write "aref foo" , i am also immediately telling you that "foo" is an array. Coversely the same if i use NTH. Common Lisp has stuff like that, where two operators apparently do the same, but choosing one over the other wins on readability. I consider readability important for code.

    (continues...)

    ReplyDelete
  2. >There are 7 commonly used equality operators

    Well, as would any newbie to Java that is introduced to the .equals(), .compareTo() and .compare() methods, one has to realize that there are different kinds of equality. Are we wanting to compare that a and b refer to the same object? Or do we mean that a and b should have the same contents? Etc.

    That is one of the reasons for all these equality operators. The other, is choosing the best one for performance. That includes, for example, using (char= ) instead of the more generic (equals) for improved performance when writing code that requires to read big files, (string=) for string comparisons and so on. Again, you can choose the more generic, slow operator, or the more specific, fast operator. Common Lisp gives you a choice.

    >There are three different local binding mechanisms that you must decide between depending on whether you want to be able to refer to earlier symbols in the same binding set, or whether you want symbols to be able to refer to themselves. You use let/flet if you don't care, let* for variables where you want later bindings to be able to refer to earlier ones, and labels for functions where you want bindings to be able to refer to themselves or earlier bindings.

    Yes, that's called flexibility and you will thank for them later when they allow shorter or more readable code.

    >There are many, many implementations of Common Lisp.

    And this gives you more options.
    Want to compile your Lisp to C to embed in some device or program? --> ECL
    Want to easily talk to java libraries? --> ABCL
    Want to produce really fast code? --> CCL, SBCL
    Want a lisp implementation that is small for downloads yet quick enough for many applications --> CLISP
    Your employeer requires you to use an implementation that requires commercial, quality support? --> LispWorks, Allegro CL, Scieneer, etc.


    >If you want to write portable code, you have to jump through some hoops.

    It's 2018 and it's really easy to write portable code, there are already libraries to achieve easy portability. For example if i need to call a C library from Lisp, i can use CFFI and write my code. Then it will do it correctly on many Lisp implementations.

    Or if i need threading, Bordeaux-Threads will do it portably and so on, and so on.

    So the hoops aren't so high, really.

    >You can't use the symbol t anywhere.

    CL is based around the notion that something that is non-NIL is true. This, in practice, allows quite easy writing of code. In truth (no pun intended), there's little need for using the symbol t in Lisp.

    >A hash isn't a list, and a CLOS instance isn't anything like either. One way you'd like them to be similar is when you're traversing them.

    They are not the same and they shouldn't be traversed the same.

    But well, if you want to access all those stuff (and more) using the same operator, just use a Common Lisp library like CL21. Problem solved.

    >If you want to map over hashes

    ... There is an easy way to do it using LOOP. And you should know how to use LOOP, that DSL you qualify as "idiosyncratic", because LOOP can make some things really easy to implement in a few lines.

    >Common Lisp is case-insensitive.

    Not really. Again, it gives you flexibility. You can create a case-sensitive symbol in Common Lisp, if you need it.

    >This is usually not a huge problem, unless you try to interoperate cleanly with newer data standards like CSS or JSON.

    ... for example here, outputing a JSON, you can use case-sensitive keyword for "name":

    (jojo:to-json '(:|name| "Common Lisp"))

    >(...) The Common Lisp answer to the problems that call for try/catch in other languages.

    (continues...)

    ReplyDelete
  3. To be honest, programmers ought to read about CL's "conditions and restart" system. Try/catch (as it exists on many languages) is about catching exceptions. Common Lisp goes beyond, providing you a mechanism to *recover* from exceptions so your code can continue running. This ought to be appreciated, i guess.

    >But do me a favor, if you're a CL user, either hop over to this web REPL, or install leiningen then hop into your local lein repl and type along here (...) Now think about how you would go about explaining to a novice programmer that it has to be more complicated than that.

    But there's a problem here; "novice to lisp" isn't the same as "novice programmer". And a novice programmer should perhaps start with an infix-notation language like Python!

    An experienced programmer, alien to Lisp, shouldn't feel uncomfortable at all with the plethora of Common Lisp features. Quite the opposite, he/she will be more interested in them. For example, to a person used to OOP, mentioning in detail the possibilities of CLOS (the Common Lisp Object System) would immediately make them understand how much more power it is giving him/her instead of more usual OOP languages.

    A final point:

    CL was born the way it is, because it unified the features of previous Lisp languages, languages that had features that were already "battle tested" for decades in fields that not only include AI but also CAD/CAM, image processing, symbolic computation, and others. Thus, the emphasis is not on "oh, look, how simple is this language" or "how easy to learn". What it bring is: "Oh, i'm glad I can keep easy things easy, make difficult things just complex (instead of complicated or convoluted), and make things that require to be performant, fast. "

    Thus, I feel it's an excellent recommendation for experienced developers that are beginners to Lisp.

    ReplyDelete