Boring Update

This has been one hell of a month, mostly for non-technical reasons, but I think I need to discuss some of them regardless. The following is a journal-style entry, so skip it if you're here for any kind of language discussion.

Specialization

Firstly, you may have noticed that I've been hacking Erlang lately. It's verbose, it's obtuse, it works at bizarre cross-purposes with itself, but it has endeared itself to me for reasons I've already discussed. It's not too clear to me why I have this drive to try new languages, and it's not entirely clear whether it gives me an edge or dulls it in the end. It feels like I'm making reasonable progress and gaining perspective on the process of expressing processes precisely, and maybe that's enough. The root of the chain is this bias I have against overspecialization, which may or may not be an evolutionary vestige, but it doesn't seem to have hurt me yet. It seems intuitively obvious that I'd want to avoid the situation where I don't have the right tools for a job, and that means keeping a lot of them around. Admittedly, I haven't practised this in real life, but cognitive tools don't take up space, and are always at my call, so it's much easier to justify.

I've had conversations with quite a few people I respect that go the other way. That is, they seem to think that going deep is much better than going broad, but that honestly only seems to be true if your goal is to end up as a corporate developer or team lead somewhere. I've also had encounters with people almost hard-wired to a particular language. One or two Lispers I keep in touch with seem genuinely concerned that I've been off doing Erlang or Smalltalk work. Pretty much every C++/C#/Java programmer I've met so far in real life have condescendingly stated that [their language] is the only one you should ever consider for production work. To top it off, I've interacted with a worrying number of Haskell douches who aggressively push their preference on other functional programmers.

That can't be the correct approach, regardless of how powerful an individual language is.

Make

The Erlang play I've engaged in has forced me to take a serious look at make. I mentioned a while ago that I reach for Ruby whenever I need to do almost any small bit of scripting. Until about a week ago, this included deployment scripts. It never really occurred to me that make was good for something other than compiling C projects, but taking a closer look, it seems like it can do quite a bit. It has conditionals, loops and functions, and it deals with command line arguments a lot more gracefully than scripts in typical general-purpose languages.

exclude = .git .gitignore *~ docs/* *org config.lisp log

define deploy
        git checkout $(1);
        rsync -rv $(foreach var, $(exclude), --exclude $(var)) ./ $(2);
        ssh $(3);
endef

deploy-public:
        $(call deploy, master, [user]@[server]:[project-root], [user]@[server])

deploy-client-a:
        $(call deploy, [client-branch], [user]@[server]:hhsc-[project-root], [user]@[server])

deploy-client-b:
        $(call deploy, [client-branch], [user]@[server]:hhsc-[project-root], [user]@[server])

ssh:
        ssh [user]@[server]

That saved me about 40 lines when compared to the Ruby script that used to do the same job[1]. Granted, the Makefile makes me type out the [user]@[server] string twice, because : is otherwise interpreted as a control character and there's oddly no way to escape it, but that's an acceptable blemish given the overall line savings. Now that's not to say that make is more elegant than Ruby, just that it's a lot more specialized for the task. Most of the chaff from those 56 lines was doing command-line parsing and some declarations, which again hints that command line argument parsing is a hack.

The other advantage of the Makefile is that using it gives me meaningful completions at the command line. In the above, if I tabbed on make, it would give me the different tasks as potential entries

inaimathi@hermaeus:~/project$ make 
deploy-client-a  deploy-client-b  deploy-public  Makefile       ssh
inaimathi@hermaeus:~/project$ make |

That's going to get more convenient the more clients we start supporting. I'm not going to go through the full make syntax; it's fairly self explanatory and docs exist in any case. A definition looks like that define..endef block, calling a function looks like $(call fn, arg1, arg2, ...), the exclude line shows you what a variable looks like, and the bit that looks like $(foreach ...) is a loop. That should be enough for pretty much anything you need to do with the tool.

Music

I had a fit of OCD the other day, and decided to finally organize my music library to prevent my phone from reporting

Unknown Artist -- 178 songs

instead of correctly sorted collections. I did reach for Ruby here, and two scripts turned out to be particularly useful

#!/usr/bin/ruby

require 'optparse'
require 'fileutils'

class String
  def naive_title_case(split_by = "-")
    split(split_by).map(&:capitalize).join " "
  end
  def strip_song
    s = self.split("--")
    (s[1] ? s[1] : self).gsub(".ogg", "")
  end
end
  
ARGV.each do |target|
  artist = target.gsub("/", "").naive_title_case
  FileUtils.cd(target) do
    Dir.entries(".").find_all{|e| e.end_with? ".ogg"}.each do |file|
      `vorbiscomment -t 'ARTIST=#{artist}' -t 'TITLE=#{file.strip_song.naive_title_case}' -w #{file}`
    end
  end
end

#!/usr/bin/ruby

require 'optparse'
require 'fileutils'

$options = {:sub => "", :downcase => nil}
OptionParser.new do |opts|
  opts.on('-r', '--regex REGEX', String, 
          'Specify the regular expression to replace') {|reg| $options[:regex] = Regexp.new(reg)}
  opts.on('-s', '--sub SUBSTITUTE', String, 
          'Specify what to replace the match with. By default, the empty string (so matches are stripped).') {|$options[:sub]|}
  opts.on('-d', '--downcase', 'If passed, all filenames will be downcased.'){|$options[:downcase]|}
end.parse!

usage unless ARGV.length > 0

def rename(str)
  ($options[:downcase] ? 
   str.downcase : str).gsub($options[:regex], $options[:sub])
end

ARGV.each do |target|
  File.rename(target, rename(target))
end

The first one is a very thin wrapper around vorbiscomment that lets me pass it more than one file at a time and uses my idiosyncratic file storage/naming conventions to infer the title and "artist"[2] of the piece. The second one is just a simple regex application script which lets me format many files at once without going through the mind numbing tedium of one mv call per file[3].

What I listen to these days is actually slightly embarrassing. A little while ago, I was working with some friends, obviously enjoying some tunes on my headphones, and pretty much froze when one of them passed a speaker wire. I'm not even sure why; we've been friends for a pretty fucking long time at this point, and I knew that musical preferences would not be the thing to finally drive us apart, but I still hesitated at listening to some of this shit with another human being.

Not at all sure where that comes from. I guess it's that I used to be a rocker back in the day. The last time I actually bought a related album was back in 2007. Looking at my current, newly-organized library, it's split about half and half between pony/videogame related electronica and classical of some sort, but I honestly didn't notice the change taking place. I'm not even sure if rock is a thing in general anymore, but it's definitely not a thing I listen to. And I guess I wasn't sure whether my friends knew that yet, since we don't tend to talk about it.

It's really odd how the peripheral pieces of my identity are the ones that cause me the most concern. I remember admitting to myself that I was really a programmer/illustrator and not a Graphic Designer, and that didn't have much of an impact on how I behaved. The little things seem to perturb me a lot more when I notice them. Maybe it has to do with the fact that they tend to change while I'm not paying attention, rather than being an effort of conscious will...

---

Footnotes

1 - [back] - wc -l says deploy.rb was 56, while the actual Makefile clocks in at 20.

2 - [back] - "Artist" is in quotes because I actually use it to group playlists, rather than Artists in the usual sense.

3 - [back] - Incidentally, you can see what I mean when I call script arguments a hack, right? More than half of each of those scripts is taken up by a huge, verbose, un-abstractable block whose entire reason for existence is making up for the fact that I'm writing a function that I want to be command-line accessible.

Ruby vs. Python Briefly

Ok, so I figure it's about time to live up to the title of this blog, since I've spent the vast majority of the language discussion firmly planted in parentheses. Aside from the fact that my company is starting a project in Erlang, I've also been scripting Python and Ruby pretty heavily.

They're actually not very different languages. Neither is perfect from my perspective[1], and neither sucks. If I had to, I could get work done in both (and having gone through the Ruby chapter in 7 Languages in 7 Weeks, I'm more inclined to look at Ruby for my next big project than I used to be). To start with, here's a boiled down, no-nonsense table that represents my perspective.

...is more annoying than...
len([1, 2, 3])	[1, 2, 3].length
"foo " + bar + " baz" or "foo %s bar" % bar	"foo #{bar} baz"
", ".join(["one", "two", "three"])	["one", "two", "three"].join ", "
map(lambda a: a + 1, [4, 3, 2, 1]) ## still makes more sense ## than join or len, though	[4, 3, 2, 1].map {|a| a + 1}
a = [4, 3, 2, 1].sort() a[0]	[4, 3, 2, 1].sort[0]
nothing.jpg	foo.methods.sort
require 'optparse' require 'pp' require 'fileutils'	import optparse, fileutils ## I also prefer the more granular ## symbol access I get with python
sudo apt-get install ruby-full irb	python
...is about as annoying as...
def aFunction(foo, bar): #do stuff return baz	def a_function(foo, bar) #do stuff baz end
with tempfile.NamedTempFile() as tmp: tmp.write("Test test\n") ##more stuff tmp.flush() popen(["lp", "-d", "a-printer", tmp.name()])	Tempfile.open() do |tmp| tmp.write("Test test \n") ## more stuff tmp.flush system("lp", "-d", "a-printer", tmp.name) end

So I am slightly biased, but like I said earlier, not enough to actually decry either language. The biggest point in Ruby's favor is its handling of blocks (as seen in that tempfile pseudo-code). I like having an expression that says "Create an entity, do this stuff and then clean up", without having to clean up myself. Python doesn't like that.[2] Gotta admit, I boggled at the join syntax the first time around. Rhetorically, who the hell decided it makes sense that a join operation is something you do to the delimiter, rather than the list? In my opinion, it would even make more sense to make it a standalone function a-la len.

I really like the syntactic whitespace in Python.

def something():
    foo()
    bar()

seems like it's cleaner than the Ruby equivalent. Except that when I want to return the result of bar (which I do quite often, given that I much prefer functional programming to OO), I need to do so explicitly. Ruby has me waste an additional line on end, but returns implicitly. While I'm at it, Python libraries seem to be heavily anti-functional programming. They do the standard "OO" thing of exposing functionality via classes, but they also typically have a heavy reliance on side effects, which makes it harder than it ought to be to compose things. A recent example I had to go through involved using pyPDF and reportlab to process existing PDFs. You can do it, but the amount of fiddling involved is nontrivial if you want to decompose the problem properly because you need to do so by setting up multiple instances of PdfFileReader/canvas and making destructive changes to them.

Also, not represented in the table is how much easier it is to install python packages in Debian. While gem frequently errors on something, I've yet to find a package I need that I can't either apt-get or retrieve using python-setuptools. That's worth something (in fact, it's worth enough that I've been procrastinating on a ruby port of get-youtube-series, which used only default components in Python, but requires several installs in Ruby).

The last thing that table doesn't encompass is the version situation. That's a fairly major one from my perspective, but I'm not sure how serious it actually is. Python 3 has been out for quite a while, but it's not uncommon to see "Supports Python 2.7" on various frameworks/utilities. Squeeze still provides 2.6.6, Django still requires 2.[4-7] and Google app-engine is still asking for 2.5 (with 2.7 being supported as an "experimental" feature). That's less than encouraging. By contrast, Ruby 1.9 is fairly widely supported (though the Debian repos are still at 1.8.7). That just doesn't seem to bode well for the next version, regardless of how enthusiastic Rossum is about it.

---

Footnotes

1 - [back] - Though, to be clear, my opinion is that Ruby gets a damn sight closer than Python.

2 - [back] - Thank you Brendan Miller for pointing me to the with statement (documented here, here and here) which does emulate blocks well enough for my purposes.

Heart Ruby

I really do have to update that header more frequently. It's been a good year and a half since I did anything other than some light make scripting in Python, JavaScript may as well be jQuery as far as my recent use of it is concerned, and I haven't done much more than some very lightweight playing in Erlang. Most of my time at work has been getting spent ass-deep in PHP which wasn't very pleasant even back when it was one of two languages I knew. The rest of it is tilted heavily towards the lisps (Common Lisp, Elisp, Scheme, in that order), and I'm still trying to get my head around Haskell through some tutorial and semi-involved practice. The practice will actually increase soon; a friend of mine wants some help with his website, and he's convinced that having it written in a lesser-known language will make it less likely to get hacked (he's had some security troubles lately). I tried to explain that this isn't actually how cryptography works, but he's having none of it. His instructions were "I don't care what you use, as long as it's not PHP". Score.

The last piece up there is Ruby, which I've had an odd relationship with. I tried out Rails a while back, but didn't like the amount of magic involved (and the various "convention vs. configuration"/security exploit stories I keep hearing about through friends aren't exactly tempting me back). I also tried out some Windows automation back when "Windows" was a thing I used for work rather than just for playing 10 year old video games. We also run Redmine at the office, so I've had to spend a very little bit of time making cosmetic UI changes. The point is, I've yet to write more than maybe 200 lines of Ruby in one sitting, but I still like it. It's clean somehow. Simple. In a way that Python never felt, even though the syntactic whitespace forces more visual consistency onto it. Despite my low line-count, ruby-full is still firmly wedged in the ## languages section of my installation shell-script, and the only reason my installation shell-script isn't itself written in Ruby is that the language isn't bundled with Debian.

I'm musing on this now, because I recently received a reminder of how beautiful it can be for simple scripting purposes. I had a problem with my XFCE4 setup. Actually, not a problem, just something that wasn't going quite as smoothly as it might have. I use multiple monitors on each of my multiple machines, you see. My desktop has two, my laptops share an external, and my work machine travels with me so it actually has two different monitors to interface with depending on where it is. The key is, no matter where I am, the situation is the same; I just want my monitors arranged left to right, each at the highest possible resolution. XFCE doesn't seem to have an option for that, so my initial approach was just to manually check xrandr output and type out the appropriate combination of --output, --mode and --right-of to get it working. It dawned on me the other day that this is pretty inefficient given how consistent the pattern is, and since I occasionally profess to know how to program, I should be able to do something about it. The problem is that step one of the process is parsing the output from a shell command, which surprisingly few languages care to do. Luckily, Ruby is one of them. My initial pass worked, but it was ugly (and I won't inflict it upon you here). After consulting codereview.SE, it was whittled down to

#!/usr/bin/ruby

def xrandr_pairs (xrandr_output)
## Returns [[<display name>, <max-resolution>] ...]
  display = /^(\S+)/
  option = /^\s+(\S+)/
  xrandr_output.scan(/#{display}.*\n#{option}/)
end

def xrandr_string (x_pairs)
## Takes [[<display name>, <max-resolution>] ...] and returns an xrandr command string
  cmd = "xrandr --output #{x_pairs[0][0]} --mode #{x_pairs[0][1]}"
  args = x_pairs.each_cons(2).map do |(previous_output, previous_mode), (output, mode)|
      "--output #{output} --mode #{mode} --right-of #{previous_output}"
  end
  [cmd, *args].join(" ")
end

exec xrandr_string(xrandr_pairs(`xrandr`))

which is pretty beautiful, as far as I'm concerned.

It's elegant for shell-scripting for two reasons;

First, Ruby has a wide range of options for calling the shell. exec seems tailor-made for the purpose above (it replaces the current Ruby process with a call to the command you pass it), spawn is useful if you want to do things in parallel and ` delimits a special string type that gets executed synchronously as a shell command and returns that commands' output.

Second, any string can be a template. This includes special strings like regexes and backticks, which is why you can compose larger regular expressions from simpler pieces as in xrandr_pairs above. You can stitch #{ } into any string you like, and the contents can be any expression, not necessarily a string. A minor, but important part is that I didn't have to call a function in order to make a string a template (there's no call to printf or format), and the contents are inlined (I'm doing "Foo #{bar} #{baz}" as opposed to "Foo #{1} #{2}" bar baz) which makes the result that much more obvious. Neither would matter much proportionally in a big project, but when I'm working on a 16 line script, I'll knock out every bit of cognitive overhead I can.

That's why I still use it. I never liked Ruby for big stuff, or even medium stuff, but I instantly reach for it to do anything under 100 lines that needs to talk to the shell.