Quicklisp, Linode, Hacking in the pejorative and other notes...

This week has been kind of a mixed bag for me; I've been thinking about a bunch of stuff, but not enough about each thing to justify a whole blog post. So here's the result of my mental mastication. It's not pretty, but perhaps it will be nourishing.

On serving data

I've been administering my own server for the last little while. First for DieCast (which is on hold for the moment 'cause the server was needed for something else) then to host some ASDF files (which are actually back up now; you should be able to asdf-install:install 'cl-css or 'formlets without serious problems), and now for a couple of websites I'm doing work on. The experience has taught me three things.

1. Common Lisp webapp deployment sucked balls before Quicklisp

My first nginx+Hunchentoot setup took hours. Some of this was for lack of familiarity with the apps because my second deployment took hours (fewer of them though). Which is an excellent improvement, but still not good in the absolute sense. The main problem was actually setting up Hunchentoot; it has many dependencies (many of which have several recursive dependencies of their own), each of which need to be downloaded and evaluated separately, each of which has at least one compiler warning, and one of which usually fails to download. The worst deployment after the first involved a key ASDF-package hosting site going down. That meant I had to go out and download + install + load all of Hunchentoots' dependencies recursively by hand in order to get them running. Sadly, lacking encyclopedic knowledge of Hunchentoot, this meant I had to try (asdf-install:install 'hunchentoot), wait for it to error out, get the piece it errored on, install it and try again. Once the server was up it was awesome, but getting it to that state was a pain in the ass the likes of which I'm having trouble analogizing properly. Quicklisp does it in 10 minutes, while simultaneously massaging my aching shoulders. I really hope zach doesn't start charging, because I get the feeling many Common Lispers would end up owing him their house (he welcomes donations, of course).

2. System setup sucked balls before Linode

I used to use Cirrus Hosting. And actually still do at work; we had them before I came in, and they're pretty good so I don't have a burning need to switch over, but we'll see what's possible once our subscription is up. Basically, I was used to a VPS being more or less just a regular server, except virtual. You have to spend a bit of time installing the distro, reboot, and install. It turns out that if you put thought into the process, a lot of that startup time can be done away with behind the scenes. Linode has put a lot of thought into the process. Going from one linux distro to another takes something like 5 minutes. I found this out bouncing between different linuxes (linuxen? linuces?); the process was initiated and that's typically a cue for sandwiches, but I didn't have enough time. Needless to say, it was a pleasant surprise the first time a deployment from bare metal to a running Common Lisp server took less than half an hour.

3. Break-in attempts are surprisingly common

If I'm to believe my auth.log, a concerted effort at hacking is made by some jackass roughly every two days. Needless to say, my iptables have been modified. It's different IPs, but always the same MO; they try some random common usernames, fail and go away. Apparently it's escaped their notice that I switched to RSA keys and disabled password/PAM authentication. To be fair, checking the logs, it seems that before the change to key-based auth, the situation regularly looked like

Feb 10 07:40:04 Invalid user abc from 61.240.36.1
Feb 10 07:40:07 Invalid user abc123 from 61.240.36.1
Feb 10 07:40:10 Invalid user benjamin from 61.240.36.1
Feb 10 07:40:12 Invalid user lstiburekz from 61.240.36.1
Feb 10 07:40:15 Invalid user kent from 61.240.36.1
Feb 10 07:40:18 Invalid user jabber from 61.240.36.1
Feb 10 07:40:20 Invalid user andres from 61.240.36.1
Feb 10 07:40:23 Invalid user dovecot from 61.240.36.1
Feb 10 07:40:26 Invalid user magda from 61.240.36.1
Feb 10 07:40:28 Invalid user alex from 61.240.36.1
Feb 10 07:40:31 Invalid user stefan from 61.240.36.1
Feb 10 07:40:34 Invalid user stefano from 61.240.36.1
Feb 10 07:40:36 Invalid user cristi from 61.240.36.1
Feb 10 07:40:39 Invalid user claudi from 61.240.36.1
Feb 10 07:40:42 Invalid user sarah from 61.240.36.1
Feb 10 07:40:44 Invalid user smokeping from 61.240.36.1
Feb 10 07:40:47 Invalid user fetchmail from 61.240.36.1
Feb 10 07:40:50 Invalid user backinfo from 61.240.36.1
Feb 10 07:40:53 Invalid user umberto from 61.240.36.1
Feb 10 07:40:55 Invalid user mauro from 61.240.36.1
Feb 10 07:40:58 Invalid user jana from 61.240.36.1
Feb 10 07:41:01 Invalid user adriano from 61.240.36.1
Feb 10 07:41:03 Invalid user xenie from 61.240.36.1
Feb 10 07:41:06 Invalid user lea from 61.240.36.1
Feb 10 07:41:09 Invalid user joule from 61.240.36.1
Feb 10 07:41:11 Invalid user Debian-exim from 61.240.36.1
Feb 10 07:41:14 Invalid user unbunutu from 61.240.36.1
Feb 10 07:41:17 Invalid user cacti from 61.240.36.1
Feb 10 07:41:19 Invalid user polycom from 61.240.36.1
Feb 10 07:41:23 Invalid user payala from 61.240.36.1
Feb 10 07:41:26 Invalid user nicola from 61.240.36.1
Feb 10 07:41:28 Invalid user melo from 61.240.36.1
Feb 10 07:41:31 Invalid user axfrdns from 61.240.36.1
Feb 10 07:41:34 Invalid user tinydns from 61.240.36.1
Feb 10 07:41:36 Invalid user dnslog from 61.240.36.1
Feb 10 07:41:39 Invalid user dnscache from 61.240.36.1
Feb 10 07:41:42 Invalid user qmails from 61.240.36.1
Feb 10 07:41:45 Invalid user qmailr from 61.240.36.1
Feb 10 07:41:47 Invalid user qmailq from 61.240.36.1
Feb 10 07:41:50 Invalid user qmailp from 61.240.36.1
Feb 10 07:41:53 Invalid user qmaill from 61.240.36.1
Feb 10 07:41:55 Invalid user qmaild from 61.240.36.1
Feb 10 07:41:58 Invalid user alias from 61.240.36.1
Feb 10 07:42:01 Invalid user vpopmail from 61.240.36.1
Feb 10 07:42:03 Invalid user ldap from 61.240.36.1
Feb 10 07:42:06 Invalid user gica from 61.240.36.1
Feb 10 07:42:09 Invalid user sympa from 61.240.36.1
Feb 10 07:42:11 Invalid user snort from 61.240.36.1
Feb 10 07:42:14 Invalid user hsqldb from 61.240.36.1
Feb 10 07:42:17 Invalid user member from 61.240.36.1
Feb 10 07:42:20 Invalid user chizai from 61.240.36.1
Feb 10 07:42:22 Invalid user yakuji from 61.240.36.1
Feb 10 07:42:25 Invalid user gijyutsu from 61.240.36.1
Feb 10 07:42:28 Invalid user kaihatsu from 61.240.36.1
Feb 10 07:42:30 Invalid user iwafune from 61.240.36.1
Feb 10 07:42:33 Invalid user oomiya from 61.240.36.1
Feb 10 07:42:36 Invalid user seizou from 61.240.36.1
Feb 10 07:42:38 Invalid user gyoumu from 61.240.36.1
Feb 10 07:42:41 Invalid user boueki from 61.240.36.1
Feb 10 07:42:44 Invalid user eigyou from 61.240.36.1
Feb 10 07:42:46 Invalid user soumu from 61.240.36.1
Feb 10 07:42:49 Invalid user hanaco_admin from 61.240.36.1
Feb 10 07:42:52 Invalid user hanaco from 61.240.36.1
Feb 10 07:42:54 Invalid user system from 61.240.36.1
Feb 10 07:42:57 Invalid user tenshin from 61.240.36.1
Feb 10 07:43:00 Invalid user avahi from 61.240.36.1
Feb 10 07:43:02 Invalid user beaglidx from 61.240.36.1
Feb 10 07:43:05 Invalid user wwwuser from 61.240.36.1
Feb 10 07:43:08 Invalid user savona from 61.240.36.1
Feb 10 07:43:10 Invalid user trthaber from 61.240.36.1
Feb 10 07:43:13 Invalid user proftpd from 61.240.36.1
Feb 10 07:43:16 Invalid user bind from 61.240.36.1
Feb 10 07:43:19 Invalid user wwwrun from 61.240.36.1
Feb 10 07:43:21 Invalid user ales from 61.240.36.1

whereas I now merely get the occasional

Feb 12 11:53:12 Invalid user oracle from 212.78.238.237
Feb 12 11:53:13 Invalid user test from 212.78.238.237
Feb 12 12:03:59 Invalid user apache from 79.174.78.179
Feb 12 20:16:59 Invalid user postgres from 79.174.78.179

So it helps, but the regularity of these attacks is still surprising to me. It seems a bit odd that a script would keep trying if it got the refused (publickey) error, so I'm forced to conclude that there are one or two spammers out there manually looking for servers they can break into. That's ... odd. And I can't shake this picture of a 12 year old in some spamming sweatshop somewhere failing to break into my server and missing his quota as a result.

On starting up

So remember back in the prehistoric ninties, when the likes of this strange creature walked the earth? When the Playstation first introduced the idea of CD-based games to the console market, a friend of mine flatly said he preferred his SNES. When questioned about it, his reasoning boiled down to one word.

"Loading..."

For the youth who never experienced this; a Super Nintendo had no loading screens anywhere. You put the cartridge in, hit the power button, and it would go straight to the logo screen. While in-game, moving between areas was instantaneous. It seems like most people working in the consumer electronics industry today have either forgotten that instant usage is really good, or they never thought so to begin with. The latest generation of consoles has loading screens friggin everywhere. A different friend of mine purchased a TV recently that has a 30 second boot cycle, and comes with a network connection for the purpose of getting firmware updates. A fucking teevee. It's hilarious that between the TV boot time and the console boot time (and I won't even mention the install time on the console because it's really unfair), it actually takes longer to start a game of whatever in his living room than it does on my computer. Weird, because I thought the whole point of consoles was that they were special-purpose devices specifically designed to run games. Entertainment isn't the end of this trend though; my phone now also takes about a minute to start up (which is fair I guess, since it basically is a computer now, complete with a flavor of Linux and a web browser). Finally, my parents recently renovated their kitchen and procured for it a, I shit you not, dishwasher that needs to boot before it starts pulling in water.

At what point did this start happening? When the hell did the decision get made in the bowels of Sony corporate HQ that it was ok for my display to have a configuration cycle? If this is where the future of TVs is going, I may very well have already bought my last non-monitor display. But beyond entertainment, my greater concern is the trend of ephemeralization (as elaborated by Graham to mean "...the increasing tendency of physical machinery to be replaced by what we would now call software.") combined with the new human habit of sticking computers into things means that we are likely to soon have shoes, lip-balm and kitchen cutlery that come with their own fabulously designed and meticulously polished loading screens.

Somehow, I'm not enraptured by this prospect.

On data moving

It's come to my attention that the Canadian government has recently had a nontrivial (and ongoing) tussle with the CRTC and the major Canadian ISPs about whether or not they should be allowed to charge arms and legs for data overages. That last link was actually to the Open Media site, which is organizing a petition against the CRTCs move. If you're in Canada, you should probably sign it. My position is basically that I don't care, because the way I use the internet, 40GB is essentially unlimited. I'm not a netflix user (though I'm constantly told I should be) I don't torrent the games like the kids these days, and downloading Linux packages is a joke if you're running the minimal system I've got over here. The single largest component I install is haskell-platform, which takes something like 600Mb. With an M. Even with my fiancee being perhaps the worlds' biggest YouTube makeup video fiend, we've never actually approached the limit of our plan. My interest in this fight is purely in the interest of a theoretically unfettered future; one where data is as free as it could possibly be, and that world includes no limits on how much it's allowed to move per month (incidentally, that's also why I frown when I see things like this happening; freedom of information includes the right for said information to exist). So I'm against the CRTC here, but seemingly not for the same reason as anyone in a 100 km radius of me.

Heart Ruby

I really do have to update that header more frequently. It's been a good year and a half since I did anything other than some light make scripting in Python, JavaScript may as well be jQuery as far as my recent use of it is concerned, and I haven't done much more than some very lightweight playing in Erlang. Most of my time at work has been getting spent ass-deep in PHP which wasn't very pleasant even back when it was one of two languages I knew. The rest of it is tilted heavily towards the lisps (Common Lisp, Elisp, Scheme, in that order), and I'm still trying to get my head around Haskell through some tutorial and semi-involved practice. The practice will actually increase soon; a friend of mine wants some help with his website, and he's convinced that having it written in a lesser-known language will make it less likely to get hacked (he's had some security troubles lately). I tried to explain that this isn't actually how cryptography works, but he's having none of it. His instructions were "I don't care what you use, as long as it's not PHP". Score.

The last piece up there is Ruby, which I've had an odd relationship with. I tried out Rails a while back, but didn't like the amount of magic involved (and the various "convention vs. configuration"/security exploit stories I keep hearing about through friends aren't exactly tempting me back). I also tried out some Windows automation back when "Windows" was a thing I used for work rather than just for playing 10 year old video games. We also run Redmine at the office, so I've had to spend a very little bit of time making cosmetic UI changes. The point is, I've yet to write more than maybe 200 lines of Ruby in one sitting, but I still like it. It's clean somehow. Simple. In a way that Python never felt, even though the syntactic whitespace forces more visual consistency onto it. Despite my low line-count, ruby-full is still firmly wedged in the ## languages section of my installation shell-script, and the only reason my installation shell-script isn't itself written in Ruby is that the language isn't bundled with Debian.

I'm musing on this now, because I recently received a reminder of how beautiful it can be for simple scripting purposes. I had a problem with my XFCE4 setup. Actually, not a problem, just something that wasn't going quite as smoothly as it might have. I use multiple monitors on each of my multiple machines, you see. My desktop has two, my laptops share an external, and my work machine travels with me so it actually has two different monitors to interface with depending on where it is. The key is, no matter where I am, the situation is the same; I just want my monitors arranged left to right, each at the highest possible resolution. XFCE doesn't seem to have an option for that, so my initial approach was just to manually check xrandr output and type out the appropriate combination of --output, --mode and --right-of to get it working. It dawned on me the other day that this is pretty inefficient given how consistent the pattern is, and since I occasionally profess to know how to program, I should be able to do something about it. The problem is that step one of the process is parsing the output from a shell command, which surprisingly few languages care to do. Luckily, Ruby is one of them. My initial pass worked, but it was ugly (and I won't inflict it upon you here). After consulting codereview.SE, it was whittled down to

#!/usr/bin/ruby

def xrandr_pairs (xrandr_output)
## Returns [[<display name>, <max-resolution>] ...]
  display = /^(\S+)/
  option = /^\s+(\S+)/
  xrandr_output.scan(/#{display}.*\n#{option}/)
end

def xrandr_string (x_pairs)
## Takes [[<display name>, <max-resolution>] ...] and returns an xrandr command string
  cmd = "xrandr --output #{x_pairs[0][0]} --mode #{x_pairs[0][1]}"
  args = x_pairs.each_cons(2).map do |(previous_output, previous_mode), (output, mode)|
      "--output #{output} --mode #{mode} --right-of #{previous_output}"
  end
  [cmd, *args].join(" ")
end

exec xrandr_string(xrandr_pairs(`xrandr`))

which is pretty beautiful, as far as I'm concerned.

It's elegant for shell-scripting for two reasons;

First, Ruby has a wide range of options for calling the shell. exec seems tailor-made for the purpose above (it replaces the current Ruby process with a call to the command you pass it), spawn is useful if you want to do things in parallel and ` delimits a special string type that gets executed synchronously as a shell command and returns that commands' output.

Second, any string can be a template. This includes special strings like regexes and backticks, which is why you can compose larger regular expressions from simpler pieces as in xrandr_pairs above. You can stitch #{ } into any string you like, and the contents can be any expression, not necessarily a string. A minor, but important part is that I didn't have to call a function in order to make a string a template (there's no call to printf or format), and the contents are inlined (I'm doing "Foo #{bar} #{baz}" as opposed to "Foo #{1} #{2}" bar baz) which makes the result that much more obvious. Neither would matter much proportionally in a big project, but when I'm working on a 16 line script, I'll knock out every bit of cognitive overhead I can.

That's why I still use it. I never liked Ruby for big stuff, or even medium stuff, but I instantly reach for it to do anything under 100 lines that needs to talk to the shell.

Old Machines

I'm not attached to things.

My grandfather was; whenever we'd do some carpentry or light construction, he'd insist that we save old screws and nails we found. It's always seemed weird to me because even in the old country, nails and screws are things you can get at the hardware store for $20 per 5 lbs, so there never seemed to be much of a point in saving old ones. Whenever I'd point this out, he'd reply "You never know when a nail will come in handy", and proceed to stash stray nails in variously sized glass jars. Ok, yes, this was across the atlantic, so what he actually said was "Nikad neznaš kad ćeš trebati čavlića", but you get the point. He had a musty attic full of clothes he wore decades ago, books he read once, games he played when he was a kid, travelling cases that had only been used once, and a thousand other treasures that I never got to see, but that he likely also could have discarded with no disadvantages. I'm sure there was some socio-political reason for this, but I'm digressing.

I'm not attached to things, and I can say this as someone who has observed humans attached to things. But I still feel a bit perturbed when someone throws a computer away. I'm sure my descendants will have the same reaction I had to the old nails. "Grandpa, you can buy computers for 2¢ per core, they come with ten free petabytes of memory and a lifetime supply of storage on the Googazon servers. Why are you keeping those old things?" I can already feel the urge to tell those smarmy cunts to get the hell off my lan...

For the moment though, I'm slightly less than insane for putting old machines to use. Last week I stumbled upon an HP Pavillion circa 1998 (well, I assume it was thereabouts since it still had Windows 98 installed). With a roaring 566 MHz processor, a truly awe-inspiring 64 MB of SDRAM, and a massive 15GB hard drive. I've been meaning to set up a backup server for my setup here anyway. I still had to spend some money on a couple of hard drives (about $50 each for 160GB IDE drives, I had one lying around, but the rest of my spares are all SATA. Could have saved some money by getting a couple of adapters instead, but I didn't think soon enough. I'll get some of these if the drives ever fail) and an ethernet card ($4.99).

The first thing I had to do was remove a few unwanted items.

As comitted as I am to reusing old machines, I've still got to admit that there's very little use today for a phone modem or floppy drive. They were fairly easy to remove; just a couple of mounting screws internally. What was slightly tougher was this plastic faceplate that covered the area next to the front-facing USB port; it was held in by a small, springy metallic assembly that I had to lever out with a swiss army knife (I wanted another hard drive to go there).

Next up, I ripped out the 15GB drive it came with, popped in one of my 160GB ones and threw in that ethernet card for good measure. Then I installed Ubuntu Server 10.10. It could have been Debian, but I wanted to try out the latest Ubuntu release, and there are some things I'd like to do with pacpl that don't seem to work on my debian machine. The tradeoff is that Emacs seems to misbehave out of the box on Ubuntu, but this isn't exactly going to be a development machine so that's ok. The only stuff that went on was SSH server, GIT and Ruby (my language of choice for quick and dirty scripting).

Once the system was installed, the CD drive could come out (not about to install MS Word or any such nonsense; any other software that goes on this machine will come in through the network). That turned out to be easier said than done though; it was secured by screws on both sides, so I had to completely disassemble the box to get at it.

The hard drive destined for the position was going to rattle in a slot that size, and while I don't plan to race this machine around the block or anything, it's probably better to be safe. A couple of drive brackets made sure it would stay in place. Shop around if you plan on buying some, incidentally, I just put that link up because it was the first I found; there were actually a couple of braces lying around from my last case so I didn't need to order any. It also seems like you could improvise a set if you didn't feel like buying them.

With everything hooked up, it was time to boot back into the machine.

That incongruent looking mesh plate covering the top drive is a spare from the same case that had the extra brackets. And yes, I named the machine "orphan". It seemed appropriate. Here's ls /dev, showing the new drives (still haven't formatted them, that'll be for next weekend).

And that's it. I dropped it into a little wheel assembly that's been going unused since I got that mammoth tower for my main machine. It gives it a somewhat R2-D2 feel (this may be the start of an art project).

I'll put together some scripts to copy out key directories from my other machines and that'll be that. I guess I could also use it as a full-out NAS (ok, I technically am, but you know what I mean) or streaming server, but I'm not sure how far those 566MHz and 64MB of RAM are going to stretch. In any case, even with the slightly higher price/GB I had to pay for IDE drives, converting this old machine was much cheaper than shelling out for a pre-built.

The Microsoft Arc keyboard came in quite handy with this project. It's fairly ergonomic, the arrow oddity isn't as annoying as it seems it should be, and the transmitter is easy enough to move around. It's definitely a step up from wrangling USB cables from my main machine about three feet to the side. My only complaint is that it friggin devours batteries, compelled like some primal beast, always growling for more. That's easy enough to solve, I guess, just remove the batteries when it's not in use, but that's a small annoyance on an otherwise perfect spare keyboard.

Confusion of Ideas

On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - Charles Babbage Babbages' response was sufficiently condescending in the finest grumpy-IT-guy tradition (or perhaps this was just a reflection in his journal and he merely rolled his eyes at the time, boiling with internal rage). Holding the implicit assumption that, at least while operating a computer, a technician must forsake their humanity and act with perfect precision. The quote usually gets trucked out to ridicule someone who doesn't understand how computers work. You're supposed to be thinking of the CS manager who downloads and runs every .exe attachment, then wonders aloud (perhaps in your direction) why the damn machine doesn't work. "How the hell", we are expected to reflect, "would a computer getting the wrong inputs still produce the correct output? It's ridiculous!" Well the notion wasn't ridiculous. It was just ahead of its time by a few hundred years.