REBOL Without A Cause

So Thursday was this months' Code Retreat over at Bento. We were solving the Poker Hands kata that I've already written about, so Gaelan and I decided to make an attempt using REBOL3. Because I've already written about it, I'm not going to explain the problem, or go very deeply into code-review-style exposition.

I'll show you some REBOL3 code, point out the highlights and the confusing bits, and call it a day. Hit up the chat room if you have questions.

The First Crack

REBOL []

map-f: func [ fn a-list ] [
    res: make block! 5
    foreach elem a-list [ append res do [fn elem] ]
    res    
]

group: func [ a-list ] [
    res: make map! 5
    foreach elem a-list [
        either res/(elem)
        [ poke res elem res/(elem) + 1 ]
        [ append res reduce [ elem 1 ]]
    ]
    res
]

test-hand: [[ 1  hearts ] [ 2 clubs ] [ 3 clubs ] [ 4 diamonds ] [ 5 hearts ]]
test-flush: [[ 1  hearts ] [ 2 hearts ] [ 3 hearts ] [ 4 hearts ] [ 5 hearts ]]

group-by-rank: func [ hand ] [
    group map-f func [ a ] [ first a ] hand
]

group-by-suit: func [ hand ] [
    group map-f func [ a ] [ second a ] hand
]

is-flush: func [ hand ] [
    1 = length? group-by-suit hand
]

is-pair: func [ hand ] [
    grouped: group-by-rank hand
    foreach k grouped [
        if grouped/(k) = 2
    ]
]

Our first attempt was pretty pathetic, all things considered. Most of that comes down to lack of familiarity with the language, and a desire on my part to do things functionally. The first meant that we spent about 15 minutes trying to figure out how to set the value of a particular map slot[1]. The second meant that I had to implement a couple of basics myself, one of which I was used to having provided even in batteries-not-included languages like Common Lisp. The above isn't actually a valid approach because of r3's default scope. Which means

>> do %poker-hands.r
do %poker-hands.r
Script: "Untitled" Version: none Date: none
>> res: "Foobarbaz"
res: "Foobarbaz"
== "Foobarbaz"

>> map-f func [ a ] [ a + 1 ] [ 1 2 3 4 5 ]
map-f func [ a ] [ a + 1 ] [ 1 2 3 4 5 ]
== [2 3 4 5 6]

>> res
res
== [2 3 4 5 6]

Don't worry; there's a way around this which I'll discuss later. After the event, I made a few refinements and got it up to

The Second Crack

REBOL []

fn: make object! [
    map: func [ fn a-list ] [
        res: make block! 5
        foreach elem a-list [ append res do [fn elem] ]
        res
    ]
    range: func [ start end ] [
        res: make block! 10
        step: either start < end [ 1 ] [ -1 ]
        for i start end step [ append res i ]
        res
    ]
    frequencies: func [ a-list ] [
        res: make map! 5
        foreach elem a-list [
            either res/(elem)
            [ poke res elem res/(elem) + 1 ]
            [ append res reduce [ elem 1 ]]
        ]
        res
    ]
    val-in?: func [ val map ] [
        foreach k map [
            if map/(k) = val [ return true ]
        ]
        return false
    ]
]

hands: make object! [
    straight: [[ 1  hearts ] [ 2 clubs ] [ 3 clubs ] [ 4 diamonds ] [ 5 hearts ]]
    straight-flush: [[ 1  hearts ] [ 2 hearts ] [ 3 hearts ] [ 4 hearts ] [ 5 hearts ]]
    pair: [[ 2  hearts ] [ 2 clubs ] [ 3 clubs ] [ 4 diamonds ] [ 5 hearts ]]
    two-pair: [[ 2  hearts ] [ 2 clubs ] [ 3 clubs ] [ 3 diamonds ] [ 5 hearts ]]
]

ranks: func [ hand ] [ fn/map func [ a ] [ first a ] hand ]
suits: func [ hand ] [ fn/map func [ a ] [ second a ] hand ]

count-ranks: func [ hand ] [ fn/frequencies ranks hand ]
count-suits: func [ hand ] [ fn/frequencies suits hand ]


has-flush: func [ hand ] [
    1 = length? group-by-suit hand
]

has-straight: func [ hand ] [
    rs: sort ranks hand
    rs = fn/range rs/1 (rs/1 + (length? rs) - 1)
]

has-straight-flush: func [ hand ] [
    all [ has-straight hand has-flush hand ]
]

has-group-of: func [ size hand ] [
    fs: count-ranks hand
    fn/val-in? size fs
]

has-pair: func [ hand ] [ has-group-of 2 hand ]
has-three: func [ hand ] [ has-group-of 3 hand ]
has-four: func [ hand ] [ has-group-of 4 hand ]
has-two-pair: func [ hand ] [
    fs: fn/frequencies values-of count-ranks hand
    2 = fs/2
]
has-full-house: func [ hand ] [ all [ has-pair hand has-three hand ]]

Not much trouble taking that step, once I kind of sort of got what I was doing, but I'd be coding along and occasionally get invalid argument errors. And it would always turn out to be a problem with the separation of arguments and calls. It happened in quite a few places, but the worst offender was

has-straight: func [ hand ] [
    rs: sort ranks hand
    rs = fn/range rs/1 (rs/1 + (length? rs) - 1)
]

That line starting with rs = , specifically. Initially, it read rs = fn/range rs/1 rs/1 + length? rs - 1. Interpreter says: WTFYFWWYETT?[2]. What the snippet means is what you can read from the parenthesized version above. That is,

Apply the function fn/range to the argument "rs/1" and the argument "one less than the length? of rs added to rs/1".

This is probably an expressive edge-case, but it's slightly concerning that I ran into it so soon. That scope issue is still outstanding, by the way. Object!s don't have internal scope by default either, which begs the question of why they're called "Objects", so the net effect is still the same.

>> do %poker-hands.r
do %poker-hands.r
Script: "Untitled" Version: none Date: none
>> res: "Foobarbaz"
res: "Foobarbaz"
== "Foobarbaz"

>> fn/map func [ a ] [ a + 1 ] [ 1 2 3 4 5 ]
fn/map func [ a ] [ a + 1 ] [ 1 2 3 4 5 ]
== [2 3 4 5 6]

>> res
res
== [2 3 4 5 6]

Anyhow, it technically runs. As long as you don't nest map or frequency calls. After a trip over to the Rebol/Red chat room on SO for some quick review by actual rebollers[3], I got to

The Third Crack

REBOL []

fn: context [
    map: funct [ fn a-list ] [
        res: make block! 5
        foreach elem a-list [ append/only res do [fn elem] ]
        res
    ]
    range: funct [ start end ] [
        res: make block! 10
        step: either start < end [ 1 ] [ -1 ]
        for i start end step [ append res i ]
        res
    ]
    frequencies: funct [ a-list ] [
        res: make map! 5
        foreach elem a-list [
            either res/(elem)
            [ poke res elem res/(elem) + 1 ]
            [ append res reduce [ elem 1 ]]
        ]
        res
    ]
    val-in?: funct [ val map ] [
        foreach k map [
            if map/(k) = val [ return true ]
        ]
        return false
    ]
]

hands: make object! [
    straight: [ ♥/1  ♣/2  ♣/3  ♦/4  ♠/5 ]
    straight-flush: [ ♥/1  ♥/2  ♥/3  ♥/4  ♥/5 ]
    pair: [ ♥/2  ♣/2  ♣/3  ♦/4  ♠/5 ]
    two-pair: [ ♥/2  ♣/2  ♣/3  ♦/3  ♠/5 ]
]

read-hand: func [ hand-string ] [
    suits-table: [ #"H" ♥  #"C" ♣  #"D" ♦  #"S" ♠ ]
    ranks-table: "--23456789TJQKA"
    fn/map func [ c ] [
        to-path reduce [ 
            select suits-table c/2 
            offset? ranks-table find c/1 ranks-table ]
    ] parse hand-string " "
]

ranks: func [ hand ] [ fn/map func [ c ] [ probe second c] hand ]
suits: func [ hand ] [ fn/map func [ c ] [ probe first c ] hand ]

count-ranks: func [ hand ] [ fn/frequencies ranks hand ]
count-suits: func [ hand ] [ fn/frequencies suits hand ]


has-flush: func [ hand ] [
    1 = length? group-by-suit hand
]

has-straight: func [ hand ] [
    rs: sort ranks hand
    rs = fn/range rs/1 (rs/1 + (length? rs) - 1)
]

has-straight-flush: func [ hand ] [
    all [ has-straight hand has-flush hand ]
]

has-group-of: func [ size hand ] [
    fs: count-ranks hand
    fn/val-in? size fs
]

has-pair: func [ hand ] [ has-group-of 2 hand ]
has-three: func [ hand ] [ has-group-of 3 hand ]
has-four: func [ hand ] [ has-group-of 4 hand ]
has-two-pair: func [ hand ] [
    fs: fn/frequencies values-of count-ranks hand
    2 = fs/2
]
has-full-house: func [ hand ] [ all [ has-pair hand has-three hand ]]

Note that the definitions of fn, and in particular fn/map have changed subtly. The change to fn in general is that each of its functions is now a funct instead of just a func. This is the solution to that scope problem from earlier; funct provides an implicit scope for its body block where func doesn't. Meaning that if you define fn in this new way, you can now actually do

>> res: "Foobarbaz"
res: "Foobarbaz"
== "Foobarbaz"

>> fn/map func [ a ] [ a + 1] [ 1 2 3 4 5 ]
fn/map func [ a ] [ a + 1] [ 1 2 3 4 5 ]
== [2 3 4 5 6]

>> res
res
== "Foobarbaz"

>> 

and you can safely nest fn/map/fn/frequencies calls.

The other subtle change to fn/map specifically is that it now uses append/only rather than append. The reason for this is that append implicitly splices its arguments. That is

>> do %poker-hands.r ;; map defined with plain append
do %poker-hands.r ;; map defined with plain append
Script: "Untitled" Version: none Date: none
>> read-hand "1H 2C 3C 4D 5S"
read-hand "1H 2C 3C 4D 5S"
== [♥ 1 ♣ 2 ♣ 3 ♦ 4 ♠ 5]

>> do %poker-hands.r ;; changed to append/only
do %poker-hands.r ;; changed to append/only
Script: "Untitled" Version: none Date: none
>> read-hand "1H 2C 3C 4D 5S"
read-hand "1H 2C 3C 4D 5S"
== [♥/1 ♣/2 ♣/3 ♦/4 ♠/5]

>> 

Apparently the original author found that he was doing sequence splicing more than actual appending. But instead of writing a separate splice function, or maybe a /splice refinement to append, he made splicing appends' default behavior. No, I have no idea what he was smoking at the time.

In order to get the behavior you'd probably expect from plain append, you have to run the refinement /only, which as far as I can tell, generally means "do what you actually wanted to do" on any function it's provided for. A guy calling himself Hostile Fork says it better than I could:

We don't tell someone to take out the garbage and then they shoot the cat if you don't say "Oh...wait... I meant ONLY take out the garbage"! The name ONLY makes no semantic sense; if it did make sense, then it's what should be done by the operation without any refinements!-Hostile Fork

Afermath

So that's that. I didn't get to a working solution yet, because this script doesn't compare two hands to determine a winner (or a draw), and it doesn't handle the aces-low edge case, but I'll leave those as an exercise for the reader. It'll tell you what hand you have, and it can elegantly read the specified input. At the language level, REBOL3 is interesting. And the community is both enthusiastic and smart. And I really hope the r2/3 transition gives them the excuse to clean up the few counter-intuitive things that slipped in over time. It's enough that I'm making an addition to the logo bar, which I don't do lightly[4].

This series of tinkering had no particular cause. I was just playing around with a problem I had lying around in a language I was curious about. Next time, I'll pick one, probably some kind of lightweight application server, and see how far I can push it. Hopefully that doesn't get too far in the way of my LISP project...

---

Footnotes

1 - [back] - Using poke, in case you're curious.

2 - [back] - What The Fuck You Fucker, Why Would You Ever Type That?

3 - [back] - I have no idea why they don't just call themselves "rebels".

4 - [back] - PHP logo notwithstanding.

Dear The Internet

Dear The Internet,

I see you're having security problems, so I'm going to let you in on a technique for doing proper authentication. I've discussed it before, but I get the feeling you thought I was trafficking in trade secrets, and scrupulously decided not to hear too much. Let me be clear that this is public knowledge, and is meant for sharing.

Proper Authentication

To start with, your server should have a public/private keypair, and so should your users. When a user registers, ask them for their public key, and publish the server's public key in a few disparate places on the web. Then, when a user wants to log in

1. the user specifies their account with an account name
2. the server generates a piece of random state, encrypts it with the accounts' public key, signs it, and sends both the cyphertext and the signature to the client
3. the client verifies the signature, decrypts the cyphertext message, signs the resulting cleartext and sends the signature back to the server
4. the server verifies the signature against the state it sent out for that account

Assuming everything went well, the server can act on a successful authentication.

What just happened?

• The user knows that the server they're communicating with has access to the private key they expect
• The server knows that the user they're speaking to has access to the private key that corresponds to the user account asking for authentication
• Finally, critically, neither has enough information to allow impersonation of the other

There! That's the secret! Now you'll never fuck it up again!

This is a way to prevent any further "Oh noez, our server got hacked!" garbage forever, because if a server using this auth method got hacked, all the hackers actually got is information that's already public, or can reasonably be.

Before you pipe up with the "But users are too stupid to use private keys" thing, shut up.

Just shut up.

The user doesn't have to do this manually; it's easy to imagine a series of plugins, one for each browser, that implement key generation, encryption and management for a user without them having to really understand what's inside the black box. More importantly, even a stupid, simplified, operationally insecure PK authentication system with full focus on ease-of-use would be better than using passwords on the server side.

Please please consider this, The Internet, I'm getting really worried about you.

Sincerely yours,

-Inaimathi

REBOL

One of the things we talk about at the Toronto Common Lisp User Group meetings is, possibly surprisingly, other interesting languages, whether classical or up-and-coming.

REBOL (pronounced the same as "rebel") is one that got mentioned a few times. And it sounded quite interesting. But I never talked about it here because it was released under a proprietary license, and as you've probably guessed if this blog wasn't evidence enough, I'm a GPL nerd. Well, as of REBOL3, the language is released under the Apache v2.0 license, which officially makes it Free Software. You can find the complete source here.

It's a fairly recent development, so this isn't one you can apt-get install quite yet. So, here's how you go about building it on Debian.

Before We Get Started...

You'll obviously need git and make installed.

apt-get install git make

Then...

...you'll need to clone the REBOL3 repo.

git clone https://github.com/rebol/r3.git

And then you'll need to download the r3 binary from this page. If you're on an x86 linux machine, you have a choice of three depending on what version of libc you have installed. To find that out, run ldd --version[1]. Once you've go that, unpack it, and rename the new r3 file to r3-make.

On 32-bit machines...

You're pretty much done. Enter

make make # re-generate the makefile
make prep # generate relevant header files
make      # compile REBOL

After a minute or so, you should have a binary file called r3 that you can add to your path as a REBOL3 interpreter.

On 64-bit machines...

... you have a couple more things to do. Specifically, you need to run this as root[2]

dpkg --add-architecture i386
aptitude update
apt-get install ia32-libs 
apt-get install libc6-dev-i386

That will add the 32-bit versions of libc and some other libraries so that you can actually run the compilation step.

Now Then

You can find the basic primer here, but the thing that most interests me about REBOL so far is its implementation and use of parse, which you can see demonstrated here, here and here, though there have been changes between REBOL2 and REBOL3. You can find the appropriate Emacs mode here, and I'm already thinking of the changes I want to make to it. Other interesting documentation includes the REBOL3 guide, the list of REBOL3 functions and this SO answer which includes a quick REBOL3 CGI script, though really, anything in the rebol3 tag is pretty interesting.

---

Footnotes

1 - [back] -Note that if you're just out to use the language, and don't really care about any of this Software Freedom business, you've already wasted some time. You can just get the appropriate binary and call it a day. I'm getting it because REBOL3 builds part of itself using REBOL3 scripts. And I'm compiling my own because I like being able to see inside of the languages I use, and I'm a big enough nerd to actually do it from time to time, and I've probably spent more time than is strictly healthy listening to Richard Stallman. Proceed or ignore the remaining parts of the process at your discretion.

2 - [back] -Thank you user Fork from this thread.

Ping

I'm still alive, just so you know.

LISP Contest

The past little while has seen me refine my entry to the Lisp In Summer Projects contest. You can find the code here, in case you'd like to keep an eye on progress, but it's not playable yet, and I don't want to talk about it until I've at least ironed out some of the big questions. Don't worry, I'm keeping a journal, so you'll see all the gory details rather than just a finished product, but I want to have a product before I show it off. On a related note, I've been told that there's a local Bento-based group called Games With Friends that regularly tests tabletop card and board games in meatspace. I'm seriously considering dropping by, both before and after I get a working system together.

Work

I've started my new job, and it's fun so far. On a scale of 1 to 10, the levels of paranoia and bureaucracy here are Dilbert, and apparently that's all I'll be able to tell you. Not that I ever blogged about the actual systems I was working on at my old job, but we're going to be doing some very interesting things here[1] and I was looking forward to being able to talk about them. I won't though; anything past what I've already said could compromise some of my employers' IP, or at least run a significant risk of doing so, and no offense, but that risk isn't worth it for the sake of a hobby blog. So that's that.

• We're doing R&D work
• It involves embedded systems
• It's very interesting
• Lots of the locals are severely, sometimes paralyzingly, paranoid about security
• It's sometimes necessary to requisition a requisition-form-requisition form

...and you won't hear anything else about what I'm working on at work until I start working somewhere else.

Fiction

I just finished reading Neptune's Brood by Charlie Stross, which can best be thumbnailed as "Accountants In Spaaaaace!", and it was an excellent read. This is the sort of stuff I go to Stross for; not the usual Star-Wars-esque naval battles in 3 dimensions, but a hard look at what space battles would actually look like in the absence of hand-waivium and plottite. I'm not sure how sympathetic the characters are since I'm a lousy judge of these things[2], but the world and in particular its finance system is constructed in such a way as to make space-colonization by humans[3] plausible both in the physics and economics senses. The societal implications about our deep future are less than encouraging, but I don't want to spoiler any part of this before anyone reading this has likely gotten a chance to read it. I got the hard-back through an Amazon pre-order, but you can probably walk into your local Chapters and just pick up a fresh one by this point.

---

Footnotes

1 - [back] - Well, by my definition of "very" and "interesting" at any rate.

2 - [back] - I thought the crew of Blindsight was very well thought out and understandable in the human sense, only to find out that the author had gotten feedback about how un-cuddly they were. That's another very interesting piece of non-hand-wavium sci-fi that I can recommend, by the way. Easily the best vampire story I've ever read.

3 - [back] - Or at least post-humans.

Bittersweet

So I'm leaving another company today.

It's ok, everyone knows.

Actually, they knew about four weeks ago, I gave them ample notice because I genuinely liked working with them. They're on the market for a Common Lisp/Python/JavaScript developer, by the by. Company details here if you're both "in Toronto" and "interested". There's an Employment link at the bottom of the sidebar.

This is probably the first time I've left an employer with, on balance, positive feelings. I don't think there's anything here I'm glad to be getting away from, unlike last time. I mean, you know, all the usual complaints that apply to any less-than-10-man shop, but nothing that actually prevented me from enjoying damn-near all of it. We made our best effort at avoiding the classic Agile tar-pit, we put together the best practices we could, including source-control, bug-tracking and project wikis. We used tools appropriate to the situation, and tried to solve problems in scalable, reliable and secure ways. If there was anyone other than me there who knew Common Lisp and Python, we'd have done extensive code-reviews too.

We didn't GPL any of our code, which always disappointed me, because it meant that I

• couldn't get an outside opinion without collecting NDAs, which I don't like doing
• won't be able to reasonably work on the projects any more after I leave

That's in addition to the usual argument about how sufficiently interesting projects are just plain better off becoming open source, so that they can leverage as many developers as they can attract, rather than merely as many as their owners can pay for. There's a threshold at which the first number is so much larger than the second that it really doesn't make sense to keep secrets, and I think at least two of the projects I shepherded could cross it given the chance. Such is life, I suppose.

The new place is going to offer some serious challenges. Starting with, I'm sure, a week or two of severe culture-shock. You see, I've never actually worked at a company with more than about 200 employees. I mean, my employers have contracted for various bureaucracies, ranging from multi-national food chains, to hardware suppliers, to actual governments, but I've always been the visiting contractor or IT consultant. I'm not sure I'll like it, but I'll try almost anything once.

The work will be different. Instead of a bunch of projects, we have The Project, and from what I understand it's a fairly ambitious piece of R&D/prototyping work that has a good chance of changing the world by end of next year. In a good way, I think. The other really big draw for me is that I get to work with people who are, by my reckoning, much more skillful developers than I am. It's been a really long time since I've done that, and I wasn't far enough along my learning process that I could take advantage of it last time.

I've been told not to worry about the culture, and that the biggest challenge will be learning about systems and techniques. Which is not a problem, as you know if you've met me.

"Learning things" is my default state.

Elm In Practice

So I've gotten some time in with it. Not quite enough to finalize the new interface, though I do have an unstyled 95% version running on my local with an apropos choice of music. Firstly, here's the code.

The Code

module Mote where

import JavaScript.Experimental (toRecord)
import Json (fromString, toJSObject)
import Graphics.Input (button, buttons, customButtons)
import Window (middle)
import Http (sendGet, send, post)
import Maybe (maybe)

----- Signal Declarations
uriDir str = "/show-directory?dir=" ++ str
reqPlay str = post ("/play?target=" ++ (maybe "" id str)) ""
reqCmd str = post ("/command?command=" ++ (maybe "" id str)) ""

command = buttons Nothing
playing = buttons Nothing
files = buttons "root"

dir = sendGet $ lift uriDir files.events
cmd = send $ lift reqCmd command.events
ply = send $ lift reqPlay playing.events

----- Utility
jstrToRec jStr = let conv = toRecord . toJSObject
                 in maybe [] conv $ fromString jStr

----- Application
box n = container 350 n midTop

cmdButton name = height 42 $ width 80 $ command.button (Just name) name

controls = flow down [ box 48 $ flow right $ map cmdButton ["backward", "stop", "pause", "forward"]
                     , box 50 $ flow right $ map cmdButton ["volume-down", "volume-off", "volume-up"]]
           
entry { name, path, entryType } = let btn = if | entryType == "return" -> files.button path
                                               | entryType == "directory" -> files.button path
                                               | otherwise -> playing.button (Just path)
                                           in width 350 $ btn name

showEntries res = case res of
  Success str -> flow down . map entry $ jstrToRec str
  _ -> plainText "Waiting..."

showMe entries = flow down [ box 100 $ controls
                           , showEntries entries ] 

main = showMe <~ dir

And that's all. Seriously. This replaces all of the ~200 lines of JS/HTML/CSS that comprised the Angular.js edition, and the ~300 lines of its jQuery/Backbone predecessor.

So, if nothing else, Elm is very terse.

module Mote where

import JavaScript.Experimental (toRecord)
import Json (fromString, toJSObject)
import Graphics.Input (button, buttons, customButtons)
import Window (middle)
import Http (sendGet, send, post)
import Maybe (maybe)

That first part is the module declaration and imports, hopefully self-explanatory.

----- Signal Declarations
uriDir str = "/show-directory?dir=" ++ str
reqPlay str = post ("/play?target=" ++ (maybe "" id str)) ""
reqCmd str = post ("/command?command=" ++ (maybe "" id str)) ""

command = buttons Nothing
playing = buttons Nothing
files = buttons "root"

dir = sendGet $ lift uriDir files.events
cmd = send $ lift reqCmd command.events
ply = send $ lift reqPlay playing.events

This declares the main signals of the interaction, and some uri/request helper functions they'll need. command is the group of buttons that issues playback commands, playing is the group of buttons sending play instructions specifically, and files is the group of buttons sending show-directory commands. These were all handled by the same callback mechanism in earlier versions of the interface, but it makes sense to separate them if we're dealing with their signal streams. dir, cmd and ply just take event signals from those button groups, make appropriate Ajax requests when necessary, and return signals of responses.

----- Utility
jstrToRec jStr = let conv = toRecord . toJSObject
                 in maybe [] conv $ fromString jStr

That is a short utility function that converts a JSON string to a (potentially empty) list of records. The empty list situation happens in two cases

• if the server sends back an empty list
• if the server sends back a malformed JSON string

----- Application
box n = container 350 n midTop

cmdButton name = height 42 $ width 80 $ command.button (Just name) name

controls = flow down [ box 48 $ flow right $ map cmdButton ["backward", "stop", "pause", "forward"]
                     , box 50 $ flow right $ map cmdButton ["volume-down", "volume-off", "volume-up"]]
           
entry { name, path, entryType } = let btn = if | entryType == "return" -> files.button path
                                               | entryType == "directory" -> files.button path
                                               | otherwise -> playing.button (Just path)
                                           in width 350 $ btn name

showEntries res = case res of
  Success str -> flow down . map entry $ jstrToRec str
  _ -> plainText "Waiting..."

showMe entries = flow down [ box 100 $ controls
                           , showEntries entries ] 

main = showMe <~ dir

This is the meat of the front-end. box is just a positioning helper function. cmdButton is a helper function to define a playback command element. Note that these are missing a piece of functionality from the old interface: clicking and holding the rewind/forward/volume-up/volume-down buttons doesn't do anything. It used to make serial requests to the server for the appropriate command, but Elm doesn't have very good support for HTML events. I'll talk more about that in a bit.

controls defines the two-row, centered placement of those command elements. entry defines a button for the main show/play buttons which comprise the principal interaction with Web Mote. These are missing the play/shuffle sub-buttons for directories and they subtle styling, but that's just because I didn't do it yet. There's no obviously missing feature that would prevent me from implementing all of it; I'd just need to define the appropriate customButton and slot it in. I'd call it five lines at the outside. Thing is, I want to get to writing this article first, so it'll probably happen in an addendum.

Now that we've got that out of the way, here's what I think.

What I Think

To summarize, very good, but obviously not finished yet. Which makes sense, since it's only at 0.8. I'm going to go through the headaches first, then note the things I particularly like about working with it.

Headaches

Signal Hell

Or, alternately, "Type Hell". I'm putting this one front-and-center, because Elm's author is fiercely anti-callback, but seems to be just fine with introducing a similar situation with the type system.

The argument against callbacks goes like this in a nutshell: if you write one, you're separating pieces of a procedure that should really be unified. You want to express "do this stuff", but part of it has to happen after an asynchronous request, so you have to break your procedure up into pre-async and post-async stuff, then have the request call the function that completes post-async stuff after the request returns. It gets even worse if you need to do multiple async requests as part of your tasks; you might need to split the work up arbitrarily among a large number of functions, all of which should actually be unified conceptually.

Now, I'm not disagreeing with this argument, but take a look at the bottom of that code from Mote.elm.

showMe entries = flow down [ box 100 $ controls
                           , showEntries entries ] 

main = showMe <~ dir

What I want to express here is "Stack the controls on top of the file entries (figuring out entries based on the signal dir)". But you can't display an Element in the same list as a Signal Element because that would make some type theorist somewhere cry apparently. So instead of doing something like

main = flow down [ box 100 $ controls, showEntries $ id <~ dir]

I have to write a separate callback-like function to accept the sanitized signal value and display that instead.

This is the same situation as callback hell. The only difference is that callbacks separate your code at boundaries determined by asynchronous calls, while these signal display functions do it at boundaries determined by the type system. I guess one of those might be better than the other if you squint hard enough, but I'm not seeing it from here.

Very Few Event Options

A button or customButton send signals when they're clicked. input of type="text", passwords, checkboxes, and dropDowns send signals when their value changes. textarea and radio buttons don't exist. And that's all.

What do you do if you want a given form to submit when you hit Ret in a relevant input? What do you do if you want to define a button that can be held down (requiring a mouse-down event)? How do you implement draggables, or droppables, or datepickers, or any of the interactive pieces that jQuery has trivially provided since something like 2006? You either do it with global signals, or you make liberal use of the JavaScript FFI. Which isn't exactly fun. Since Elm is trying to do all of styling/content/behavior specification, I understand that you need to have elements like image that don't actually have behaviors. That is, they're of type Element rather than of type (Element, Signal a). But the ones that do send signals should have a menu of signals to provide. I mean, you already have this cool record syntax, what you could do is provide an interface for the user where,

button : String -> SignalOptions -> (Element, Signal a)

and SignalOptions is something like { click : a, mouseEnter: a, mouseLeave: a, mouseDown: a, mouseUp: a, keyDown: a }. Granted, maybe that shouldn't be a button, but rather a different multi-signal element, but it would give quite a bit more flexibility to front-end developers. If you had an element like that, you could easily implement any of the interactions I mention above.

No Encoding/Decoding Out-of-the-box

I'll probably implement something here when I get around to poking at the language again, but there's no built-in way to call encodeURI or encodeURIComponent from Elm. Which means that as written, this front-end will fail to play files with & in their name. That's less than ideal. I get the feeling it wouldn't be too hard to implement using the JS FFI, but I'm not diving into that right now.

Gimped Case

The Elm case statement doesn't pattern-match on strings. There's no mention of that behavior in the docs, so I'm not sure whether this is a bug or an unimplemented feature or what, but I missed it once in a ~50 line program. Specifically, in entries

entry { name, path, entryType } = let btn = if | entryType == "return" -> files.button path
                                               | entryType == "directory" -> files.button path
                                               | otherwise -> playing.button (Just path)
                                           in width 350 $ btn name

where I had to resort to using the new, otherwise unnecessary multi-branch if. Unfortunately ...

Gimped if Indentation

Because there's no elm-mode yet, you're stuck using haskell-mode for editing .elms. haskell-mode craps out on indentation of that multi-branch if statement I just mentioned. If you try to indent the following line, it'll yell at you about parse errors rather than inserting the appropriate amount of white-space, which makes working with an already unnecessary-feeling operator just that little bit more annoying. This is similar to that [markdown| |] tag indentation issue I mentioned last time, it's just that the Web Mote front-end port didn't happen to need any markdown.

Gratuitous Differences

Type annotation (::) and cons (:) from Haskell have been switched for no obvious reason, and if seems to have a similar treatment. Unlike most of the other things I bumped into, this and the case "bug" have no hope in hell of being solved by a mere user of the language, so hopefully the designer does something about them.

Nitpicks

These aren't big things, and they're not really related to the language itself, but I noticed them and they were annoying.

No Single-File Option

This is just a nice to have. It would have made this front-end marginally easier to deploy, but I'm not sure how it would work if you needed more than one file served for your program. Elm targets JavaScript as a platform, which means that the base language is deployed as a js file that you have to host manually if you're not using the elm-server. When you compile an Elm project, you have an option that looks like this

  -r --runtime=FILE           Specify a custom location for Elm's runtime
                              system.

It's slightly misleading, because what it actually does is specify where to load elm-runtime.js from in the compiled file. Literally, it determines the src property of the appropriate script tag. For that Mote front-end, I had to elm --make -r "/static/js/elm-runtime.js" --minify Mote.elm, and then make sure to serve elm-runtime.js from that static url (by default, you can find this file in ~/.cabal/share/Elm-0.8.0.3/elm-runtime.js, in case you were wondering).

Anyhow, it would be nice if there was a compiler option you could activate to just have this runtime inlined in your compiled result, rather than served separately.

Unstable Website

elm-lang.org is down pretty frequently. It seems to be up at the moment, but I'm not sure how long that's going to be the case. It happens often enough that I just went ahead and did a checkout from its github. Then I found out that the "Documentation" pages happen to be missing from that repo...

Highlights

Anything I didn't mention above is good, which is to say "most of it", but there are two things I like about the language enough to call out.

Records

This is brilliant. Take a bow, you've nailed record interaction. The approach probably wouldn't fit trivially into GHC, but it would solve some of the problems their records have. It's also something the Erlang devs should probably keep an eye on, because it's much much better than what I remember having access to in Erl-land. Probably the biggest win is that Elm records get first-class treatment in terms of the languages' pattern matching facilities, which lets you do things like

entry { name, path, entryType } = let btn = if | entryType == "return" -> files.button path
...

That's something I miss in almost every single language that has both pattern matching and k/v constructs. As usual, Common Lisp has a 95% solution as part of the Optima pattern matching library.

This dynamic record syntax also lets you trivially handle JSON input from a server. In case you didn't notice, the stuff I was passing into entry originates in ajax responses from the server.

Haskell-grade Terseness

Just a reminder. Despite all those flaws I pointed out above, the Elm version of this particular program weighs in at about 1/4 the code of the reactive Angular.js version, let alone the traditional plain DOM/jQuery approach. It's also more pleasant to work with than JS, but that's an entirely subjective point. Improvements can still be made here; implementing haskell-style sections and multi-line definitions would save a bit of typing, though, to be fair, not as much as I thought it would.

Conclusions

I've already mentioned that I'm going to take a swing at putting together some SSE support, encodeURI(component)? calls and a more appropriate Emacs mode for Elm, but it probably won't be very soon. Thanks to a tip-off from Dann, I managed to squeak into the registration for the Lisp In Summer Projects event, which looks very much like a multi-month NaNoWriMo with parentheses instead of character development and sleep.

I'm going to make a serious attempt at getting a little pet project of mine up-and-running in either Common Lisp or Clojure by September 30, which means I'll have very little time to hack on someone else's up-and-coming language regardless of how interesting it looks.

Dragging in an FRP Context

I made an off-the-cuff remark earlier to the effect that Elm doesn't let you easily define drag/drop functionality, or element-originating clicks. Really, the situation is that you can't easily work with any of the basic HTML events, which also include hovering, element-originating keypresses, various window events, and various form events. When you think about how you'd implement any of them individually, it starts to become obvious why that is.

The first reflex is to reach for callbacks. Which, as was already discussed, is the exact opposite of what Elm is trying to do. The real trouble begins when you consider how you'd do the same thing without callbacks in order to preserve that purity of purpose.

First Pass

The obvious solution is to use a bunch of signals everywhere. One for each of the element-based events. Let the user specify signal values on elements, and dispatch on their results at the other end.

Except thats quite complex.

At first glance, you're looking at twenty or so global signals, each of which are going to have the kind of isolated, complicated dispatch we saw in that Tic Tac Toe example. That sounds worse in every way than callback hell; all your dispatch needs to be centralized, which means that behavior under various circumstances will by definition be separated from the element it pertains to, and you suddenly can't understand any component of your program without understanding the central signal dispatch code.

Second Pass

Another approach might be not to let the user specify signal values. Make them hooks to the relevant element. Expose some kind of interface to the user so that they can pipe other signal values into various properties of that element, and call it a day.

Also, we don't really need to have a signal per HTML event. For the situations I'm currently thinking about, we could get away with exactly two. Keyboard.focus and Mouse.focus will give me most of what I'd want in a pretty simple way. Basically, have mouseover, mouseout, mousedown, mouseup and mouseclick send this over the Mouse.focus signal, and let mouseclick, esc and tab send the same over the Keyboard.focus signal.

You'd then have some idea of what needs to be moved as a result.

User Side

Of course, that's all base implementation stuff. On the client side, you don't want to have to do things like maintain your own table of draggables to dispatch a signal to when relevant. You'd want to be able to do something like

draggable dragDefs $ plainText "This text is draggable"

and have that tap the right signals so that when you mousedown or touch on "This text is draggable", it starts moving along with the cursor. In basic terms what needs to happen is

• when the mouse down signal is being sent
• and the Mouse.focus signal is referring to a draggable
• start piping cursor position, modified by initial deltas, into the x and y coordinates of that element

and I have no idea what the appropriate way to express that is in the framework of the existing Elm language.

It sounds like it might just be easier to avoid those interactions while I'm starting out. SSEs sound like they'd be a much easier first feature, actually.

SSEs

The reason being that, when you think about it, this fits perfectly into the FRP paradigm. A source is a signal whose value is the latest matching message body and/or id. That's it. You'd want the declaration to look something like

src = eventSource "/my/source/uri" ["message type 1", "message type 2" ...]

at which point src should be a signal you can pass around, whose current value will be the latest message coming out of "/my/source/uri" that has one of the message types specified. It might also be useful to handle unlabeled messages, at which point our message needs to look something like

data SSE = SSE { id : Maybe Int, label : Maybe String, body : String }

Manageable, if slightly annoying due to the optional fields.

You'd implement a rolling message by piping src through plainText . .body, and you could put together a very simple chat program with some judicious use of foldp.

These were all just some random thoughts I wanted a good look at, for the time being. Like I said, I'll be throwing my next few spare hours at putting together an Elm-based WebMote front-end. Fortunately, this task doesn't involve any in-depth interaction, and the SSEs aren't central to the exercise.

Elm First Impressions

For the past little while, I've been poking around a new language named Elm. A Haskell-like web front-end language with a heavy focus on FRP. Actually, no, it's not like Haskell, its syntax is Haskell except for a few omissions[1], a couple justifiable small changes, and a couple pointlessly gratuitous differences[2]. To the point that the actual, official recommendation is to just use Haskell mode to edit Elm files.

This works pretty well, except for one thing: Elm has a built-in reader macro for Markdown input. Using this feature in Haskell mode plays all kinds of hell with your indentation and highlighting. Enough that I thought it worth-it to hack a workaround in using two-mode-mode. This is far from ideal, but bear with me. You need to get two-mode-mode from that previous link, do a search/replace for mode-name into major-mode, and delete the line that reads (make-local-hook 'post-command-hook). Then, you have to add the following to your .emacs somewhere:

(require 'two-mode-mode)
(setq default-mode (list "Haskell" 'haskell-mode)
      second-modes (list (list "Markdown" "\[markdown|" "|\]" 'markdown-mode)))

and then run two-mode-mode whenever you're editing .elm files. The end result is that, whenever you enter a markdown block with your cursor, your major mode will automatically change to markdown-mode, and change back to haskell-mode when you leave. There has to be a better solution than this, probably involving one of the other Multiple Modes modules, and I'll put some thought into it when I get a bit of time.

Installation/Basics

Installing is ridiculously easy. If you've ever installed a module for Haskell, you won't have trouble. It's just cabal update; cabal install elm elm-server. Do the update first, like it says there; the language hasn't reached 1.0 status as of this writing, which means that it's quite likely there will be significant changes by the time you get around to following these instructions.

You write code into .elm files, which you can either preview dynamically or compile. You do the dynamic preview thing by running elm-server in your working directory. That starts up a server listening on http://localhost:8000 that automatically compiles or re-compiles any .elm file you request. That server runs on Happstack, and does a good enough job that the official elm-lang site seems to serve directly from it.

If you're like me though, you prefer to use static files for your actual front-end. You can use elm --make --minify [filename] to generate a working .html file[3] that you can serve up along with the elm-runtime from whatever application server you want to use.

Enough with the minutia though. Really, I'm here to give you a paragraph or two on what I think about the language.

What I think about the Language

The usual disclaimers apply.

• you'll easily find more people who are familiar with JS/HTML than those who are familiar with Elm
• if you use it, there's an extra[4] abstraction layer between you and the final front-end
• using it forces your users to enable JavaScript. Ostensibly, you can use the compiler to generate noscript tags, but all these seem to do is statically document what the page would do if JS was on.

That second one in particular means that once again, you really should learn JavaScript before trying to use Elm to save yourself from it.

Once you get past that, it's quite beautiful and elegant. Much better than plain JS for some front-end work. Not that that's a very high bar.

There's some stuff conspicuously missing, like my beloved SSEs, and some basic DOM interactions including draggable and an arbitrary, element-triggered click event. The approaches available out-of-the-box are respectively, Drag Only One Element That You Can't Drop and Detect Mouse Location On A Click, Then Dispatch Based On It. Neither of those seem very satisfying. In fact, the proposed workarounds look strictly worse to me than the "callback hell" this language is trying to save me from.

Those shortcomings are just getting me more interested, to be honest. The reason being that it looks like it's possible to implement additional native functionality fairly easily, so all it'll do is cause me to spend some time writing up the appropriate, signal-based libraries to do these things.

Overall first impressions so far are good, though I'm seriously questioning how useful this language is going to be for more complicated interfaces. In the short term, I'll test out its shallow limits by writing a new WebMote front-end.

I'll let you know how it goes.

---

Footnotes

1 - [back] - Which I'm pretty sure will eventually be addressed. I particularly miss full sections and where, though you'd think the multi-line function declarations would be the biggest gap.

2 - [back] - For no reason I could see, : is Elm's type annotation operator, while :: is Elm's cons. It's precisely the opposite in Haskell, and buys little enough that I hereby formally question the decision. Similar reasoning seems to apply to the operator <|, which seems to do exactly the same thing as Haskells' $, except that it's twice as long.

3 - [back] - Or separate .html and .js files, if you also passed the -s flag.

4 - [back] - Not particularly stable, yet.

Short Ramble and Almost Literate Cooking

Just as a heads up, this piece is brief reflection followed by dinner. No programming information, except perhaps metaphorically.

Short Ramble

I'm sitting at home alone, sipping tea and listening to the almost-silence of the city with light cello overtones. My wife went to the cottage for a couple of weeks and took our kid with her, so the apartment is relatively peaceful for the first time in, oh, about a year. Now that I think about it, this is the first time in about seven or eight years that I'm spending any serious alone-time.

There's a story in that too. It's really bizarre, but everyone I tell the situation to has a reaction resembling "Wow. You must be feeling pretty lonely". And to set that straight, no I don't. I enjoy solitude. It's not a dirty word to me. Oddly, everyone who has this reaction really ought to know better given what I am, but it's still pretty consistent. Enough that I'm beginning to wonder if there isn't some social undertow I hadn't noticed before. Anyway, not important right now.

Man, life has been crazy as fuck lately; I've been rolling with the punches long enough that "rolling" began to feel like the natural state. And that's probably not a good thing. From huge deployment pushes at work, to the disruptive experience of looking for a new place, to the massively disruptive experience of having a child, there hasn't really been anywhere near enough time for me to sit down and reflect on much. So I'm taking the opportunity, while I've got my tea in hand and the clock and cello to keep me company.

Or rather, I started, then discovered that it was too hard to take myself seriously. Boo hoo, Inaimathi, the professional Canadian Lisper, with a happy son and wife, diminishing mortgage and steadily improving physical fitness is feeling sorry for himself. Why is that? Could it be because the software he's writing at his tiny-but-profitable company is getting enough clients attention that maintenance is non-trivial? Because he's only got time to devote about an hour a day to artistic endeavors and hobby blog?

And having run smack into my own snarky sense of self-doubt, it became clear that it would be more fun to be doing something other than sitting here. I was going to head over to the local Pho place for food, but lets put these hands to good use.

Almost Literate Cooking

I've got a tag on this blog called almost-literate-programming. It's not what I write most because it takes quite a bit of effort to slice a program thinly and accurately enough to label its insides for others' consumption, but it's up in the top five[1]. I'm going to try something similar here.

Before I begin, a note on general strategies. The North American approach almost universally seems to be recipe-driven. That is

1. decide what you're going to make
2. consult a recipe
3. collect the ingredients you need to make it
4. portion those out in precise quantities through a learned ritual
5. construct food

This contrasts pretty severely to the traditional eastern European way of doing things, which from what I've observed is usually ingredient driven.

1. see what you've got around
2. construct food
3. if the result is good, record recipe

No real judgment call here by the way; both are valid ways of constructing a meal, the former gets more consistent results while the latter is a lot more fun assuming you have the habit of keeping a stocked kitchen. I'm going to mix and match today; consulting my inventory, I've found that my potatoes are coming up on their best-before date, so I'll make those. I also kind of want some fried chicken, so I'll make that too.

I probably should have defrosted the chicken first, but did the potato peeling instead. Most people, including everyone who's ever cooked for me, throws the skins away, but I'm trying to use the whole buffalo today, so.

This is one of the problems you run into with the ingredient-driven approach; it doesn't lend itself to long-term planning. This'll have to go in the microwave for defrosting, rather than doing it the natural way. If I was having people over, I'd have second thoughts at this point, but I'm not fussy when it comes to chicken.

Right, I want the potatoes sliced into even chunklets. They don't have to be a particular size, they just need to be similar enough to each other that they'll boil at about the same time.

The chicken's still not defrosted, and I just put the pot on, so I prepare the breading stages for that chicken. We've got breadcrumbs[2], flour and eggs. My wife always skips the flour for some reason, but it never makes the chicken taste any worse, so whatever works for you. The two mandatory parts here are eggs and breadcrumbs. I've always found it mildly odd that making a fried chicken meal involves marinating the flesh of an animal in the juices of its unborn offspring. Not really sure who came up with that one, or how, but it lends credence to some theories I've heard.

Anyway, the chicken's out. And the water's warming up sufficiently to accept my offering of potato, so I put those chunks in along with some salt.

This is where the fillet knife comes out. Yes, we own Chef Tony knives. They were on sale when we were moving in together, and the fillet knife and stake knives aren't half bad. Flouring the chicken is just step one.

Checking on the potatoes. If you can do this to one, it's not ready yet. A boiled potato would have just fallen apart right there.

This is my usual fried chicken routine. Flour, egg, flour, egg, breadcrumbs. Like I said, the flour is entirely optional, you can do this part with just eg, then breadcrumbs. As a note, it's much harder doing this with one hand. Especially if your other hand is holding a phone.

In the meantime, the potatoes are boiled, so I drain them and put them back on low heat to dry out. You can't tell from those pictures, but they're already falling apart by the first one. I probably should have kept them boiling for a little bit longer than I did.

You can see the consequences in that mashing picture. If they were really ready, they wouldn't be crumbling like that, they'd be mashing entirely. There's two things wrong with the creme. First, I didn't have time to heat it, so it's going in almost straight out of the fridge. And second...

...it's not creme. I'm lactose intolerant, so I can't have the real stuff, and I couldn't find lactose free creme anywhere. I'm reasonably sure it exists, just haven't seen any in real life.

Oil goes in the pan on medium/high heat. It's at 7 on my stove, yours might be different. This is Canola oil. I'm pretty sure you can use vegetable oil too if you like, but don't use olive for frying. You want enough in there that it'll 1/2 to 3/4 submerge your fillets, not just enough to cover the bottom of the pan.

Every part of the fucking buffalo. The leftover breadcrumbs, eggs and flour come together to make a little bread/cake thing. My grandmother called it a "tortica" in Croatian, which literally means something like "cakette" or "small cake", but it's really just the leftovers after the breading process. I always found it mildly entertaining that this is a "bread" made by adding egg and flour to ground up bread. Maybe I should start calling it zombie bread.

The potatoes are coming along nicely. I'm still keeping them on low heat so that they reduce a bit more.

That's the first wave of chicken on the pan. The oil's been heating for a few minutes at this point. If you did it right, it should start sizzling as soon as you put the first piece of chicken in there. The flip happens pretty soon thereafter; only about three minutes or so. You should be looking to get that darker brown color on each side, rather than counting time.

The potatoes are just about ready at this point, and the chicken is ready not long after that. I like to split a thicker piece just to make sure there's no red inside. The rest of the chicken goes on the same way. Once that's done...

The potato skins go in. This takes a bit longer than the chicken, just because I'm basically trying for a chip-like consistency. They need to be salty and crunchy when they come out, which is why I salt and dry them over the pan. Uh, just to clarify, since I realized after the fact that I only have a "before" and "after", but not a "during" pic, I did actually put the skins into the oil. I just took them out and dried them off on the mesh afterwards.

And that's that. I packed some of it away for tomorrow, and truthfully didn't end up finishing what I put on the plate either.

This won't do at all.

Cleanup is just as much a part of the task as setup is

(defmacro with-kitchen (&body body)
  `(progn (get :cutlery :dishes :ingredients)
          ,@body
          (clean :cutlery :oven)
          (wash :dishes)))

(let ((m (with-kitchen (make-instance 'meal))))
  (eat m))

so I need to get this out of the way. It'll also give me the time to get some water boiling for an accompanying tea.

There. Admittedly, another round of dishes is waiting after these finish their soak, but it's better than nothing. Now then

Fuck.

Yes.

See you next time.

---

Footnotes

1 - [back] - If you don't count the language-specific tags, anyways.

2 - [back] - Store-bought, as it happens, but nothing's stopping you from making your own[3].

3 - [back] - If you do, add some garlic and oregano[4].

4 - [back] - Unless you don't like garlic, I guess.