Joel Test Patch

Dear Mr. Spolsky,

I've been a big fan of your blog while you were updating it regularly. It was a very good collection of essays on managing a software enterprise from the Windows/shrinkwrap perspective (the second one is actually a distinction I learned from reading your excellent Five Worlds piece which still rings true for me).

During a recent spare moment, I re-read The Joel Test, and seriously considered how closely an organization that scores between 8 and 12 would resemble a place where I'd like to work. A few bugs emerged;

• I would not enjoy working at a place that still insists on CVS (SVN would be acceptable if for no other reason than git-svn exists and is bi-directional)
• "The Best Tools Money Can Buy", in the sense of hardware, isn't really necessary (in other circumstances, I'd argue that it's unnecessary in the software sense too, because I'm unsure money can buy the best software tools, but I'll put that down to my bias as a Linux user and say no more about it)
• Specs and schedules aren't necessarily the best way to do things. Both words also have a lot of baggage and history, and as a result, some organizations may claim a point or two here even if they're doing it wrong, while others might be penalized even though they adhere to the principles explained in the associated other articles you've written.

That said, I humbly submit the following patch for your consideration. Hopefully, I'm not wrong about all of it (if nothing else, I welcome a counter-argument and explanation of why I'm wrong).

Yours

-Inaimathi

========================================

diff -u joel-test.html joel-test-edits.html

--- joel-test.html 2011-07-20 10:28:40.000000000 -0400

+++ joel-test-edits.html 2011-07-20 11:05:06.000000000 -0400

@@ -5,14 +5,14 @@

The Joel Test

-Do you use source control?

+Do you use distributed source control?

Can you make a build in one step?

Do you make daily builds?

Do you have a bug database?

Do you fix bugs before writing new code?

-Do you have an up-to-date schedule?

+Do you have an up-to-date schedule or work queue?

-Do you have a spec?

+Do you have a minimal spec?

-Do you use the best tools money can buy?

Do programmers have quiet working conditions?

Do you have testers?

Do new candidates write code during their interview?

Do you do hallway usability testing?

@@ -24,8 +24,8 @@

Of course, these are not the only factors that determine success or failure: in particular, if you have a great software team working on a product that nobody wants, well, people aren't going to want it. And it's possible to imagine a team of "gunslingers" that doesn't do any of this stuff that still manages to produce incredible software that changes the world. But, all else being equal, if you get these 12 things right, you'll have a disciplined team that can consistently deliver.

-1. Do you use source control?

-I've used commercial source control packages, and I've used CVS, which is free, and let me tell you, CVS is fine. But if you don't have source control, you're going to stress out trying to get programmers to work together. Programmers have no way to know what other people did. Mistakes can't be rolled back easily. The other neat thing about source control systems is that the source code itself is checked out on every programmer's hard drive -- I've never heard of a project using source control that lost a lot of code.

+1. Do you use distributed source control?

+I've used commercial source control packages, and I've used Mercurial, which is free, and let me tell you, any of the free tools (Mercurial, Git, Bazaar) are fine. But if you don't have source control, you're going to stress out trying to get programmers to work together. Mistakes can't be rolled back easily. The other neat thing about distributed source control systems is that the source code itself is checked out on every programmer's hard drive -- I've never heard of a project using distributed source control that lost any code, or lost any time as a result of a poor network connection to the office.

2. Can you make a build in one step?

By this I mean: how many steps does it take to make a shipping build from the latest source snapshot? On good teams, there's a single script you can run that does a full checkout from scratch, rebuilds every line of code, makes the EXEs, in all their various versions, languages, and #ifdef combinations, creates the installation package, and creates the final media -- CDROM layout, download website, whatever.

@@ -82,7 +82,7 @@

Another great thing about keeping the bug count at zero is that you can respond much faster to competition. Some programmers think of this as keeping the product ready to ship at all times. Then if your competitor introduces a killer new feature that is stealing your customers, you can implement just that feature and ship on the spot, without having to fix a large number of accumulated bugs.

-6. Do you have an up-to-date schedule?

+6. Do you have an <b>up-to-date</b> schedule or work queue?

Which brings us to schedules. If your code is at all important to the business, there are lots of reasons why it's important to the business to know when the code is going to be done. Programmers are notoriously crabby about making schedules. "It will be done when it's done!" they scream at the business people.

Unfortunately, that just doesn't cut it. There are too many planning decisions that the business needs to make well in advance of shipping the code: demos, trade shows, advertising, etc. And the only way to do this is to have a schedule, and to keep it up to date.

@@ -91,7 +91,9 @@

Keeping schedules does not have to be hard. Read my article Painless Software Schedules, which describes a simple way to make great schedules.

+A work queue is the exact same thing, but without the mental baggage that people bring to the table when discussing scheduling (I prefer to ask this one, because it's inherently up-to-date, and people who make a schedule at the beginning of the project then never look at it again tend to mentally skip over "up-to-date" and just answer "yes"). Please read the Painless Software Schedules linked above. In fact, here, have another link. The point of the exercise, whether it's a schedule or work queue is that it's something you update each day to reflect the current state as honestly and recently as possible.

-7. Do you have a spec?

+7. Do you have a minimal spec?

Writing specs is like flossing: everybody agrees that it's a good thing, but nobody does it.

I'm not sure why this is, but it's probably because most programmers hate writing documents. As a result, when teams consisting solely of programmers attack a problem, they prefer to express their solution in code, rather than in documents. They would much rather dive in and write code than produce a spec first.

@@ -100,6 +102,10 @@

My pet theory is that this problem can be fixed by teaching programmers to be less reluctant writers by sending them off to take an intensive course in writing. Another solution is to hire smart program managers who produce the written spec. In either case, you should enforce the simple rule "no code without spec".

+Don't take this to mean that you should mandate several hundred page documents before any code is done; the spec needs to be just detailed enough that someone other than the writer can correctly understand what's being built. If you keep it minimal, you'll also keep yourself out of the temptation to use the spec as a club against your customers (whether internal or external).

+

+An alternative to the spec is a requirements database to go with that bug tracking database that you need. The point here is to make sure your spec can grow and be read by anyone who needs it any time, avoid the "spec-as-club" and "feature-vs-bug" arguments while still collapsing the waveform. You do not want to work on Schrödinger's project. Ever.

+

Learn all about writing specs by reading my 4-part series.

8. Do programmers have quiet working conditions?

@@ -117,25 +123,14 @@

Now let's move them into separate offices with walls and doors. Now when Mutt can't remember the name of that function, he could look it up, which still takes 30 seconds, or he could ask Jeff, which now takes 45 seconds and involves standing up (not an easy task given the average physical fitness of programmers!). So he looks it up. So now Mutt loses 30 seconds of productivity, but we save 15 minutes for Jeff. Ahhh!

-9. Do you use the best tools money can buy?

-Writing code in a compiled language is one of the last things that still can't be done instantly on a garden variety home computer. If your compilation process takes more than a few seconds, getting the latest and greatest computer is going to save you time. If compiling takes even 15 seconds, programmers will get bored while the compiler runs and switch over to reading The Onion, which will suck them in and kill hours of productivity.

-

-Debugging GUI code with a single monitor system is painful if not impossible. If you're writing GUI code, two monitors will make things much easier.

-

-Most programmers eventually have to manipulate bitmaps for icons or toolbars, and most programmers don't have a good bitmap editor available. Trying to use Microsoft Paint to manipulate bitmaps is a joke, but that's what most programmers have to do.

-At my last job, the system administrator kept sending me automated spam complaining that I was using more than ... get this ... 220 megabytes of hard drive space on the server. I pointed out that given the price of hard drives these days, the cost of this space was significantly less than the cost of the toilet paper I used. Spending even 10 minutes cleaning up my directory would be a fabulous waste of productivity.

-

-Top notch development teams don't torture their programmers. Even minor frustrations caused by using underpowered tools add up, making programmers grumpy and unhappy. And a grumpy programmer is an unproductive programmer.

-

-To add to all this... programmers are easily bribed by giving them the coolest, latest stuff. This is a far cheaper way to get them to work for you than actually paying competitive salaries!

-

+<!-- I've been doing research on what I want for my next development machine, and come to the conclusion that I don't need anywhere near "The Best Tools Money Can Buy" to do development work anymore. Give bonus points if each of the developers actually have 16 cores, 64GB of ram, and multiple terabytes worth of SATA3 SSDs in their desktops, but you can get a damn fine machine for well under $3000 (less if you want to hand your devs desktops), which wasn't exactly true back in ye olde yeare 2000e. If you have a choice between sending annoying emails to your developers or buying $400 worth of additional hard drive space, you should obviously save your fellows the trouble, but don't ding your organization for failing to allocate an annual $14 000 per developer budget for machines. -->

-10. Do you have testers?

+9. Do you have testers?

If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products, or you're wasting money by having $100/hour programmers do work that can be done by $30/hour testers. Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't recognize it.

Read Top Five (Wrong) Reasons You Don't Have Testers, an article I wrote about this subject.

-11. Do new candidates write code during their interview?

+10. Do new candidates write code during their interview?

Would you hire a magician without asking them to show you some magic tricks? Of course not.

Would you hire a caterer for your wedding without tasting their food? I doubt it. (Unless it's Aunt Marge, and she would hate you forever if you didn't let her make her "famous" chopped liver cake).

@@ -144,9 +139,11 @@

Please, just stop doing this. Do whatever you want during interviews, but make the candidate write some code. (For more advice, read my Guerrilla Guide to Interviewing.)

-12. Do you do hallway usability testing?

+11. Do you do hallway usability testing?

A hallway usability test is where you grab the next person that passes by in the hallway and force them to try to use the code you just wrote. If you do this to five people, you will learn 95% of what there is to learn about usability problems in your code.

+If you want the quickest possible primer on minimal testing techniques, take a look at the second half of Steve Krug's 2008 Business of Software talk. If you're out to do the least amount of user testing you can get away with, just take his advice.

+

Good user interface design is not as hard as you would think, and it's crucial if you want customers to love and buy your product. You can read my free online book on UI design, a short primer for programmers.

But the most important thing about user interfaces is that if you show your program to a handful of people, (in fact, five or six is enough) you will quickly discover the biggest problems people are having. Read Jakob Nielsen's article explaining why. Even if your UI design skills are lacking, as long as you force yourself to do hallway usability tests, which cost nothing, your UI will be much, much better.

Diff finished. Wed Jul 20 11:09:46 2011

Kindle and Thoughts about Work

So I got something in the mail a week ago.

Ok, not in the mail, UPS delivered it in their usual, careless manner, but I'm not talking about that right now.

There were a few alternatives I could have picked from, including the awesome looking Nook Simple Touch[1], but I went with the closed option this time. I feel like a traitor already, subscribing to this stupid, proprietary, walled-garden of a device clearly meant as a leash to Amazon's book store.

There's a lot obviously wrong with it that's wrong with much of the closed software world. Things have been engineered properly, or at least as properly as they can be given the constraints, but actually using the device runs you up against some small yet annoying limits pretty quickly. That you can't get custom wallpapers or arrange your books into categories from your computer without jailbreaking. That there's no such thing as a standalone notepad app, and you can't build it because the Kindle SDK is in beta, and signing up requires you to submit an application proposal for review. That the browser usability isn't exactly stellar, but you can't really do anything about it because you don't have the source, or the ability to make it run client-side scripts.

The reason that made me purchase this, over any other ereader out there, is the free (as in beer) 3G wireless attached to it. This is a taste of the future for someone living in Canada (or, from what I hear, in most places outside of Japan and South Korea) and I wasn't about to wait the several years it's going to take for other device providers to get their shit together.

It's still not my ideal device[2], but about as close as I can get with consumer-grade electronics at the moment. The biggest differences between it and the other devices I've tried variously[3] are the size, battery life and global 3G. Small size is obviously an advantage, but this thing is ridiculously light; at about 9 ounces it's actually competing with my cellphone in terms of portability. As an aside, it's also rather sturdy. Not that I'd try playing catch with it or anything, but it's definitely not as fragile as the stuff coming out of Cupertino. The battery-life is hands down best among portables; between the ARM processor and e-ink screen, power consumption is low enough to give it "two months" between charges. Realistically, it's two weeks or so, but that's still a damn sight better than competition. Like I said, the 3G is the winner here; it lets me do things like check various entertaining reddits on my way home, or check my email. Technically, I could do this with my phone too, but the screen is tiny enough to make it annoying, and the battery life is flaky enough that I have to charge the thing every day if I want to do any semi-serious browsing on it.

That's it for my new toys (for now anyway).

Work-wise. Man, I dunno. Non-free development has some pretty serious drawbacks. Not the least of which is that you can't really talk about implementation specifics without an NDA, and that's annoying as all shit. I'm working on an implementation of an online proofing tool (which you can actually see running over here; don't talk to me about the name, that's all the marketing guys) built in Common Lisp. The work is a lot of fun, and I get to very slowly nudge my limits in terms of solid user interface development and flexible coding, but I can't actually show it to anyone. It's improving at the glacial pace necessitated by the fact that I'm the only coder on it full time, and the rest of the people looking at it don't really know Lisp very well. I'm hoping to turn that around soon; arguing with marketing and sales people about why we should put something out into the copyleft ecology is...difficult. I haven't tried mentioning "Free software" because these are guys that'll respond according Raymond's predictions of business people.

The way they see it, we can sell it if we own it, and damn the consequences. Granted, the pool of programmers available for a Common Lisp web project is kind of narrow, but we could really do some good here, I get the feeling. Both in the sense that having such a project might make it easier to identify the next few people we want to hire (and we will at some point), and in the sense that opening it up will make it a better, more widely used product. And in the sense that free use of our tool would likely benefit the users. And, perhaps most significantly, I'll be able to put snippets up here or at codereview.SE without worrying about the people helping me getting sued at some undetermined point in the future.

Not much of a segway there, granted, but I'm starting to get pissed off. I have seven or eight different blog posts floating around in my head, half finished and in the process of construction, so I had to get something out here, even if it's not polished. There are differing opinions about how you should write, and I tend to hold with the brain crack angle. Seven ideas in the air and none on a hard drive sounds like I'm going too far in the perpetual-polishing direction.

---

Footnotes

1 - [back] - Which is a real, no-fooling, rootable Android device with an e-ink touchscreen, a micro-SD card reader, and a month-long battery life.

2 - [back] - That would be a color unit whose screen projects directly onto my retinas, has removable media/USB slots and connects directly to my thenarm, hypothenarm, interossei and lumbrical muscles for input.

3 - [back] - The i-family, laptops and netbooks.

Ugh.

I'm officially taking a break from thinking. And I'm doing it by taking in some contract work. NDAs are signed, and contracts exchanged, so while I can't really tell you specifics, it's a WordPress site. Nothing so nourishing as my usual fare; this is the proverbial greasy, bacon and cheese pizza for the soul.

The situation has blown me away, by the way. Did you know that there's a staggering amount of money out there for enterprising young hacker/graphic-designer dual classes willing to tweak around with CSS and PHP? It's bizarre because there's nothing inherently difficult about this. It's a fairly well documented, intuitively-named, neatly-packaged pile of security-errors-waiting-to-happen and all you really need to do is poke at a couple of places to make the colors show up right and the logo line up with the menu. It feels strange. I know I struggled with this same shit back at school, but it doesn't feel tough any more. It feels a bit like getting past Cinnabar Island with your pack of level 45 motherfuckers and finding yourself back in idyllic Pallet Town tearing through now-helpless level ~3 Pidgeys. That analogy may brand me as an irredeemable nerd, so feel free to substitute a more mainstream level-grinding RPG if you like. My point is, I used to be helpless in this situation, and I am now arranging reality to suit my whim.

The odd part given my recent thoughts, the really odd part, is that it pays at all. Let alone well. This is a simple task that anyone with sufficient time, interest and an internet connection (broadband optional) can learn how to do. It's the ultimate expression of freedom brought about by the GPL style of licensing and application design I talked about earlier. Why the ever-loving fuck aren't there more non-moron contractors taking advantage of the situation? Why aren't these jobs being shipped overseas like so many people seem to think all IT should be? If I had to pick one type of development that a language barrier wouldn't cock up, it would be CMS tweakery. Oddly, it's the one type of IT work that doesn't seem to be going anywhere, at least for the short term, because every place I've worked at so far and every client I've contracted with is having some local WordPress/Joomla/Drupal work done[1].

Back to the Free thing though, since that's what I was thinking about prior to my break. How does PHP+WordPress support those freedoms? First off, it actually is GPL2[2] but second off, it hits all of my added requirements, seemingly by accident.

0. components must be hot-swappable

1. required code must be terse (but not past that threshold that takes it to line-noise levels)

2. it must be simple and consistent enough that it doesn't take all of your brainpower over several hours to get into it and make changes (or, you must be able to more or less ignore the rest of the system while making changes to a specific piece)

3. An error can't bring the whole thing crashing down. It needs to toss you an exception gracefully, let you try some stuff to fix the problem dynamically and then continue on its merry way without a restart.-me, last time

Components are hot-swappable thanks to how it interacts with Apache[3]. WordPress is terse yet readable, it's consistent enough that I can get into the codebase easily and I can ignore pieces outside the specific widget/CSS component/theme I'm dealing with at whatever granularity. Finally, an error nukes a page, but doesn't crash the entire site (unless it's made in a template, but that's to be expected). Take that, every language I have at least a vague interest in. I read somewhere that the winner of a solution race frequently isn't the best, rather the worst that's still Good Enough™. This ... may sadly be that. Much as I wish a better language had a stronger web presence, PHP seems to be it outside of "actual" programmers[4].

So yeah. I'd prefer the dominating position to be held by a functional, consistent, macro-enabled (or lazy), assertion-capable (or strong-type-inferencing), namespace-enabled language without a deprecated list that includes half its functions, but PHP has floated to the top. And I'm kind of happy about that. Between its pervasiveness[5], GPLv2 licensing of key applications and inherently open code, PHP may be doing more to promote Freedom in the Stallman sense than perhaps any other server-side language. Of course I should probably ding it for also being behind Facebook, but maybe it's time to put a new logo up in the title bar...

---

Footnotes

1 - [back] - I've also seen exactly one SharePoint, which I'm told I should be thankful that I didn't have to maintain

2 - [back] - so it supports the four freedoms

3 - [back] - or whatever server you use

4 - [back] - who are a minority as evidenced by the numbers; WordPress, Joomla and Drupal between them have truly intimidating market share

5 - [back] - in terms of deployment and number of "native speakers"

Freedom

The discussions around communities I frequent has recently touched on the idea of freedom; specifically, what it means in the context of software. We've moved past the gratis vs libre confusion, thankfully, but there are still arguments about what "free" really means. Some are under the impression that in this context it means "I should be free to get my work done". That's a noble goal, but it misses the point somewhat. The sentiment behind that statement was in defending PhotoShop ("It shouldn't matter that I'm using proprietary software, the point is that it can do the job"). A fellow redditor puts it better than I could;

Software freedom is as much about choice as Lincoln's Emancipation Proclamation was about giving people the choice as to whether they wanted to retain slavery or not. Software freedom is about ending certain restrictive forms of software licensing and distribution which are harmful to the public and to technological progress in general. -spiceweasel

Laced with the standard internet forum hyperbole, but it gets the point across. It's not about "freedom of choice", it's about some freedoms that are fundamental to the creation of software

• The freedom to run the program, for any purpose (freedom 0).
• The freedom to study how the program works, and change it to make it do what you wish (freedom 1). Access to the source code is a precondition for this.
• The freedom to redistribute copies so you can help your neighbor (freedom 2).
• The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.-GNU Free Software Definition

These are important freedoms to programmers and to users who understand programs. I certainly wouldn't trade them for anything in my private life, which is why I've been trying like hell to make this a free software household with as few compromises as possible.

Freedom?

Now lets be honest from the other perspective.

GIMP contains about 230 000 lines of C code. Inkscape was at 132 134 lines of C/C++ back in 2003 (pretty sure they've racked up a few 10Ks in the past 8 years or so). I have good cause to believe that PhotoShop is much larger[2]. I get that it would be morally better for PhotoShop to become a piece of libre software, but I'm having trouble convincing myself that there would be a practical benefit for the end user.

Here's an example near and dear to my heart. Emacs. It's extensible and reasonably simple to tweak. If you know Elisp, you can push it further than I've seen any other editor go. Aside from the fact that I've hooked in quite a number[3] of components released by other Emacs users, I've also written a few of my own, and lightly amended some that I include. As the GNU page says, "Access to the source code is a precondition for this". When I say "simple to tweak", by the way, I mean "I can put together a new mode in an hour or two". In fact, as I was writing this article, I decided to take a poke at blog-mode to change how a couple things work, and I didn't have to so much as restart Emacs. Hypothetically, if I had to restart (or god forbid, re-compile it) every time I tweaked something, there would be a little less incentive for me to engage in this kind of tool-building myself. There's some pre-conditions to building a system that behaves this way:

0. components must be hot-swappable

1. required code must be terse (but not past that threshold that takes it to line-noise levels)

2. it must be simple and consistent enough that it doesn't take all of your brainpower over several hours to get into it and make changes (or, you must be able to more or less ignore the rest of the system while making changes to a specific piece)

3. An error can't bring the whole thing crashing down. It needs to toss you an exception gracefully, let you try some stuff to fix the problem dynamically and then continue on its merry way without a restart.

The reason I bring this up is

1. The freedom to study how the program works, and change it to make it do what you wish

Exactly how does Freedom One work if the program in question is larger than you can fit into your head? What if you can fit it into your head, but it takes several days or weeks of study? What if you can edit it but every time you tweak anything, you need to go through a 20 minute compilation step? Or even if you only had to go through that if you made some mistake in your tweaks? What if you didn't know the language it was written in, or didn't know how to program in the first place? At what point, exactly, can we say that whether the piece of software you've got cracked open is Free or not, you are not free to change it? It surprised me to no end to hear that there are actually people out there using Emacs who have neither the ability nor the desire to program in any language, let alone a quirky dialect of Lisp that's used exclusively for Emacs extension and development, but they apparently exist. In fact, outside the Emacs users community, non-programmers are the vast majority. Free Software must seem like the craziest catch 22 ever to them.

You have the right and freedom to study and change this immeasurably complex machine that you can't possibly understand without devoting your life to (and which will barf at you with the slightest provocation).

If you line this up with the argument about Freedom and PhotoShop, it's pretty obvious what's happening. Hell, Stallman knew exactly what's happening a while ago.

The public traded nominal freedoms that it was not in a position to exercise and in exchange got some benefit (the benefit of more books being written). So if you have something that you can't use at all and you trade it for something of some value to you, you have gained. Whether or not it's the best possible deal you could have made, that's another question, but at least it was a beneficial deal.-Richard Stallman (in his talk at University of Calgary)

(The above was in reference to the establishment of copyright, but the principles still apply)

The people using Adobe's software don't have a hope of studying and changing it even if they had access to the complete source code, so why the hell should they care that they're also deprived of that source code? Or legally barred from studying and changing it? From the outside, it must seem like passing a law against unassisted breathing on the fucking moon. The user-facing documentation of PhotoShop is thick enough to beat someone to death with (or it used to be before it was digitized), so the sheer volume of developer-facing docs that comes out of this monstrosity is probably enough to drive someone to madness. If you provide a tool to aid in the understanding of an immeasurably complex machine, but that tool is itself immeasurably complex, you've done no good.

Freedom One does matter to me, but it's only because the software I use is (for the most part) simple and open enough to actually change myself. It matters in that situation whether I'm allowed to.

Is it enough that some awe-inspiring genius exists out there somewhere that can dive in and re-write PS in short order? Or that if the source was opened, there could possibly be a group of people that could fork it and make steady progress? Turning this over and over in my mind, I can only see a moral distinction and not a practical one. If the end user has no way of exercising their Freedom One, then I'm doubtful that the majority of them will ever take up the Free Software banner, and opening up something like PhotoShop runs into some fairly obvious walls. First, the sheer size of the thing means that a theoretical fork of the codebase would go literally nowhere for a very long time as people ramped up. Second, the people who tend to use it aren't programmers, and that's actually three big potential problems, take your pick.

Second a) the size of the team is going to be small. Probably smaller than is useful on a task like this.

Second b) the people you get aren't going to have the vested interest that comes with building a tool that they themselves want to use. They'll have to rely on the time tested method of gathering requirements if they're going to grow the program at all.

Second c) the team is going to be made up mostly of non-programmers, which sounds like almost-certain-fail from the get-go.

There's an entirely other discussion to be had about the middling UI quality in projects that aren't of direct use to programmers, but I'm not getting into that one with this little sleep.

Freedom!

It's beginning to dawn on me that if you really support the idea of Free Software, you are also logically opposed to the idea of monolithic systems. That if your goal is to maximize the Freedom of users, you can not hand them many hundred thousand lines of code (with an optional several thousand pages of documentation) and be content that you did your job. That suggests some basic principles by which to build Free Software

• it needs to be built in small, individually understandable pieces (a few hundred lines across one file, not a few hundred thousand across hundreds)
• large systems need to be built more or less by combining smaller systems
• a large system shouldn't fail because one or more components failed
• a system should be modifiable without restarting

Which is more or less how the *nix culture likes to build things from what I've seen. The trouble is how to reconcile this with the needs of end-users, and more specifically the moral/practical ramifications of doing so. Making things as easy as possible for non-programmers[4] implies monoliths. They need as much of their computing experience handled by the computer as possible. It's really not acceptable, for example, to have the end user specify which audio/video codec and resolution to use for converting an mp4 to an avi. They likely don't know what a codec is, for starters.

The question I'm going to wrestle with for the next little while is what to do about those people. On the one hand, I really don't think a rational approach is "Well, they should just learn how to program". That's a deathtrap, and if you disagree with me, I invite you to try talking about it to a medical receptionist sometime. The thing is, unless the end user is properly engaged and shown how their own freedoms are being infringed upon by proprietary software, they'll never jump ship. The next question is: is it morally right to grudge them the choice of trading nominal freedoms they are not in a position to exercise in exchange for some benefit? If a graphic designer chooses to use PhotoShop, should I be telling them to stop and use GIMP instead because proprietary software is a social problem, or is that fair play until they try to get me to switch back to Adobes' steaming pile?

I have no answers for now, and I'm heading to sleep before I fall over onto my keyboard, but the thoughts are still bugging me in an odd way. Hopefully this piece released some pressure for the short term, at least.

---

Footnotes

1 - [back] - I'm making the assumption since Adobe arguably has more man-hours to throw at the thing, releases new versions fairly regularly, has to implement a lot more compatibility with legacy/third-party file formats and supports color modes that GIMP doesn't. Granted the combined SLOCs of Inkscape/GIMP/Scribus/Synfig probably outweigh the Adobe Suite because the latter group shares some code, but comparing one to one programs seems like it would tilt the scales towards the proprietary offering.

2 - [back] - 17 to be precise.

3 - [back] - Who, again, are actually the majority of humans, and will probably remain so for a long time yet. Though I agree with Sussman that there's immense value in teaching programming to non-programmers.

Quite possibly The Best Line Ever

“I don’t believe in magic that works whether or not you believe in it!” -Markos Sophisticus Maximus

Game Jam

The first part is a journal of thoughts as they were happening, so try not to laugh at me too hard. The second part is a bit of reflection on the project so far. This is a summary/record of my week participating in the 2011 Spring Lisp Game Jam. Hopefully, it's useful to someone (the very act of writing it was very useful to me).

---

April 23

I did the first public facing push of this code-base at about 4:00 am my time today, and then curled up into bed. When I got up at 10:30, there was already a bug report waiting for me. Not off to a spectacular start...

The problem turned out to be how cl-css handles file compilation. Specifically, it fails if the directory it's trying to compile to doesn't exist. At this point, I'm ready to load up cl-fad to whip up a simple fix, except that cl-fad doesn't have a function to create directories. Okay... maybe it's a Lisp primitive then? A quick search for "directory" over at the hyperspec symbol index turns up nothing. So...huh? I spend 10 minutes checking the cookbook and googling fruitlessly[1]. That doesn't discourage me though. I commonly run into situations where I have a naive mental model of a given problem that some Lisper has already built an elegant solution to. So I google the more general case common lisp making directory, which turns up a lispworks doc page abut ensure-directories-exist.

This is what I meant. When I'm making a directory, I really just want a place to put some specific file. Abstracting the basic directory creation means that I don't have to special-case the situation where multiple nested directories are required, and I also don't need to worry about checking whether the directory exists before creating it. Elegant. Maybe this is a good start. Day 1 and I'm already learning something.

April 24

I went pretty batshit on the code-base today, removing anything that looked like an unnecessary additional step. I'd been keeping the model and view as separate as possible because of an assumption that there would be some sort of SQL database involved in the game eventually. I may still need to switch to one at some point, so the move might come back to bite me in the ass, but the barriers got broken down. Mostly it was intermediate functions all over the place that slightly simplified the next set of intermediate functions[2]. Things like planet-info and inventory which took model structs and returned plists instead. To be fair, that actually did help at some point; since there doesn't seem to be a way to map over a struct, it was very easy to dump an entire plist into the view with a loop.

Anyway, I made the decision during my shower that I'm keeping the struct approach since I think it might help with the concurrency issues I'll be facing next. There's also one decision from earlier that's sort of coming back to haunt me there. The way that a given captain's current planet is stored is as a string of that planets' name. That meant that I needed a function called planet-name->planet early on, and it meant that any manipulation of the current planet happened directly on the *galaxy*. It's fine for a single player, but more would cause some odd errors that wouldn't be particularly fun. The way I'm thinking of solving that is by having each captain copy out the current planet, and keep track of transactions they conduct. The total changes would then get applied when they travel. A given market could still dip into the negatives in some circumstances, but it should serve to make it much harder to force an error on an opponent sharing your planet.

Finally, markets behave differently now. Instead of static prices, each market generates a new price from the *tradegoods* and averages (By which I mean "gets the arithmetic mean of") that with its current price. It just makes sure that markets tend to stabilize over time, while hopefully keeping enough fluctuations to make profit possible.

April 25

Changed up a lot more of the system. Today, I focused on the purchase!/convey! end of things rather than the basic model. Basically, captains now have a copy of the planet they take with them and merge transactions back later. As I mentioned yesterday, this isn't a measure to make sure that no more product leaves a planet than absolutely should. In fact, I'm tempted to make deficit exports an explicit mechanic, upping the price of a good fairly substantially if it slides into the negatives. That would also play off a tweak I want to make to the market-produce! function. Namely, generating (+ 4d20-30 (/ productivity tech-level)) instead of the current (+ 2d20 (/ productivity tech-level)) to allow goods to be consumed as well as created by planets[3].

Today's lesson[4] was that state is fucking hard. Introducing side-effects to a primarily functional system played more hell with my code than I thought it would. I still don't know if I squashed every possible bug with that transaction system. It's enough to make me go paranoid and start sprinkling asserts in the vicinity of all setfs and !s. I won't yet; I'll wait 'till something unexpected blows up, but I can see an argument for it here. The end result is that this game should now be playable by more than one human at a time. There's no explicit goals yet, I'm going to save that for later, I think, but you can still get a few friends together on a lan[5] and play economic hell with a small galaxy with the game as it stands.

[Break time to remodel my kitchen. Did a bit of light view coding, but nothing interesting to report.]

April 28

Spent today mainly on the UI end. Making it pretty-ish and squashing a few display bugs I came across. Still not happy with the default theme, but it's all I'll have time for in a week. I did make an effort to completely disconnect it from all other code, so others can theoretically just push a folder of images+theme.css to add a new one. The entire experience was a bit surreal; in the middle of using GIMP to do some graphic design work, I found myself embroiled in a conversation on Reddit about how GIMP is not good enough to do design work. Since the best argument I read was "But, I like Photoshop!" I'll stick with the free[6] version, thank you.

Part of that UI tweaking mentioned above was adding jQuery and jQuery UI to the codebase. I didn't want to have to tell people to go download it themselves, and there's really nothing that needs to be done other than putting them in the correct directories. The downside is that this is now considered a Javascript project by GitHub. Which, ok, I guess is true by character count, but it still feels inaccurate.

April 29

I realized earlier today that my time's officially up for this little project[7]. Technically, time ran out about an hour ago, but I kinda got into the swing of things and decided to finish one last TODO before doing a final check-in for the event and collecting my thoughts. That ended up taking more time than I thought it would, but it really was my own fault for giving into indecision for so long. The game looks passable, it plays nicely, doesn't seem to blow up[8] and is actually pretty fun[9]. I really shouldn't have picked out something this ambitious for a week I knew I'd be occupied, but hey. Such is life, I guess. I'll keep working on it for as long as I can stand to look at the code-base, but for the moment, it's done.

Anyway, lets get to the juice; here's the distillation of what I learned in trying to put together a small-ish game in Lisp over the course of a week.

Incremental beats planned...

It was honestly surprising how true this turned out to be. I started out trying to plan out as much stuff as possible into the future, going so far as to put several levels of indirection into the model in case of a switch to a relational database later. That... was a tremendous waste of time. Both in the sense that useless code was produced as a result, and that I then had to spend time going through call-trees to figure out what a given function was actually doing. I got fed up with that fairly early on in the week and went on an already noted rampage through the model, deleting everything that I wasn't actively calling right now, preferably in multiple places.

Overall, the time spent just diving in and writing code resulted in a lot more of the final code-base than carefully plotting out where I needed to go[10]. That's probably partly a result of the plans not being comprehensive enough, and partly a result of the plans changing mid-way. I'd argue that's healthy though; there are some specific interaction points that I'd have been hard pressed to predict in advance, but that became perfectly obvious when I tried to implement them properly.

Don't read this as "don't plan", because it does save a bit of time, but not as much as I thought[11]. My best guess is that there's some point of diminishing returns with planning. You want to do about that much because doing less causes some pretty serious headaches in terms of direction, but if you do more, you're not helping (and are likely hurting). That's just a hypothesis based on my experience, of course, I'd love to see some experimental data about it.

...but remember to fix it up

The other side of the incremental approach is that it ends up producing a lot more code. The main catch is that you need to remember to stay flexible, since you might be changing large portions of the system at any given point. Even so, prospective coding still turned out to be a very powerful tool. Producing more code does mean that you have to periodically measure and cut again. There are already two places where I can see a lack of planning costing me efficiency, but I'm not sure I would have been able to predict that in advance.

Distractions hurt. A lot.

That really shouldn't even need saying, but there you have it. Trying to code up a storm doesn't work out very well when you're also re-modeling your kitchen. I'll know for next time, I guess. The one upshot is that it resulted in a system that's easy to fit in my head basically by necessity, because I had no hope whatsoever of getting an entire 8 hour stretch of time devoted to it.

Deadlines help. A lot. No, more than that.

I've had this project on the back-burner for close to four months. In those four months, I've managed to put together a half-way decent name generator and comb over the code-base for Elite for Emacs a dozen times or so. In the past five days or so, I did the rest of it[12]. The deadline helped get me in gear where I probably would have sat indecisively for a further few months. Even with all the distractions, I easily did four times as much work since the beginning of the game jam as I did since starting on this.

Reflection helps. A lot. No, more than that. Keep going. A bit more. Ok, there.

Even more than the deadline, the fact that I had to look at this from different perspectives helped a lot. It wasn't just writing code, there was also a lot of brain power spent on thinking about how I would explain the code, what it meant in the grander scheme of the project, and whether there was a better way of doing it. It's a bit counter-intuitive, but it seems that reflection (thinking about things you've just done) is a lot more useful and a lot more productive than planning (thinking about things you're going to be doing soon). I didn't end up explaining it very well regardless, but the process of thinking up said explanations seemed to help in the construction regardless.

I don't know enough

Probably the biggest one. I'm not sure if any hypothetical reader can apply this, but I sure as hell will. I've got a lot of learning left to go in pretty much every direction. First, I get the feeling that most of the model could have been put together much better using CLOS than my current approach of structs and functions. For example, planet-produce!, add-to-market!, banned? and local?[13] would have made more sense as methods. I can say that without actually knowing much about CLOS other than the name, and I'll be going through documentation and tutorials as soon as I can squeeze it into my day. The 3D math is also something I'm rusty on. I could swear I used to know this stuff back in high school, but got surprisingly little use out of it since. It would have helped a lot. Specifically, having a clearer understanding of 3D transformation would have let me create an actual 3D interface instead of faking it poorly with layers. I've ... gotta work on that. Lastly[14], I still have no clue how to use loop properly. I gather I'm not alone since it gets its own chapter in both PCL and the CL Cookbook, but it still caught me by surprise how deep the construct is.

So there. I knew I knew little, but it turns out I knew less than I knew.

That's certainly something to fix for next time.

---

Footnotes

1 - [back] - "I guess Lisp can't create directories." Is what I would have thought about two years ago

2 - [back] - And by "slightly", I mean "almost not at all"; the functions that did do necessary abstraction are still there

3 - [back] - Produced, on balance, but it opens up that scarcity mechanic

4 - [back] - Or, rather, reminder.

5 - [back] - Or an open server, if you're feeling especially frisky

6 - [back] - Libre, not gratis (although GIMP is that too).

7 - [back] - leastwise, my time for the Game Jam is up

8 - [back] - even when there are multiple people playing at once

9 - [back] - though at the moment, it's really a lot of economic activity for its own sake because I didn't get to TODO: Goals or TODO: Ship upgrades yet

10 - [back] - which is to say, most of the code resulting from careful planning was later replaced by code that had an immediate, unplanned need

11 - [back] - and probably not as much as you think, if you're a Joel fan

12 - [back] - which is to say, ported it from Elisp to CL, a working market system, GUI with faux-3D interface, debugging and a copious amount of testing

13 - [back] - and others besides, I'm sure

14 - [back] - or, at least, the last specific thing I was surprised to find myself ignorant of

Writing Less C in Lisp

It seems that I only ever get around to working on this pet project when I'm sick (which I was earlier this week). It's taken almost 5 months at this point, but the hours counter is really closer to ~15, which means that I could have done the work during a single, particularly slow, weekend.

Anyway, moving on, I've been plaing around with the codebase for Elite for Emacs (and there's a post around here somewhere that details some of the blunders it contains). Today, I'm dealing with the next level up; not pointing out where primitives are being misused, but pointing out needless patterns where they don't belong and showing one way of composing them properly. Actually, now that I look at it, I'd better take a single pattern out and deconstruct it lest I bore the ever-living shit out of everyone, including me. I'm also not eliding anything this time, this is going to deal with specifics from the Elite for Emacs 0.1 codebase and how I'm thinking about re-implementing them.

Describing Things

Actually, before I get to that one,

Random Numbers

At a cursory examination, I've found myrand, randbyte, rand1 and gen_rnd_number (and no uses of the the built-in rand function). They may or may not do similar things. The author also insists on tracking his own random number seed in a global variable (and re-generating it with a function named mysrand). Here's a sample

(defun gen_rnd_number ()
  (let ((a)
        (x))        
    (setq x (logand (* (fastseedtype-a rnd_seed) 2) #xFF));
    (setq a (+ x (fastseedtype-c rnd_seed)))
    (if (> (fastseedtype-a rnd_seed) 127)
        (setq a (1+ a)))
    (setf (fastseedtype-a rnd_seed) (logand a #xFF))
    (setf (fastseedtype-c rnd_seed) x)
    (setq a (/ a 256)); /* a = any carry left from above */
    (setq x (fastseedtype-b rnd_seed))

    (setq a (logand (+ a x (fastseedtype-d rnd_seed)) #xFF))
    
    (setf (fastseedtype-b rnd_seed) a)
    (setf (fastseedtype-d rnd_seed) x)
    a))

I'm not sure why Lisp coders get stick for re-implementing infrastructure if this is reasonably common in the outside world. Building your own byte-oriented random number generator is something a Lisp can do, but[1] you really shouldn't. If you were in the middle of writing your own implementation of rand in Elisp, Common Lisp or Scheme before you started reading this, please just do us both a favor and stop.

Now then.

Describing Things

Here's how Elite for Emacs generates planet descriptions.

(defun elite-for-emacs-planet-description (galaxy-index system-index)
  "Return planet description"
  (let ((planet-sys)
        (rnd_seed))
    (setq planet-sys (aref (aref elite-for-emacs-galaxies-in-universe galaxy-index) system-index))
    (setq rnd_seed (copy-fastseedtype (plansys-goatsoupseed planet-sys)))
    (setq elite-for-emacs-planet-description "")
    (goat_soup "\x8F is \x97." planet-sys)
    elite-for-emacs-planet-description))

Which actually lulled me into a false sense of security the first time around because it seemed

1. functional-ish
2. short
3. simple

There's one thing there that should have set alarms off though. What kind of name is goat_soup?

(defun goat_soup (source planet-sys)
  (let ((c)
        (rnd)
        (source-list nil)
        (tmp)
        (i)
        (len)
        (x))
      (setq tmp (split-string source ""))
      (setq source-list nil)
      (while tmp
        (setq c (car tmp))
        (setq source-list (append source-list (list (string-to-char c))))
        (setq tmp (cdr tmp)))
      (while source-list
        (setq c (car source-list))
            (if (< c #x80)
                (setq elite-for-emacs-planet-description (concat elite-for-emacs-planet-description (list c)))
              (progn
                (if (<= c #xa4)
                    (progn (setq rnd (gen_rnd_number))
                      (setq tmp 0);;true: non-zero, zer=false
                      (if (>= rnd #x33)
                          (setq tmp (1+ tmp)))
                      (if (>= rnd #x66)
                          (setq tmp (1+ tmp)))
                      (if (>= rnd #x99)
                          (setq tmp (1+ tmp)))
                      (if (>= rnd #xCC)
                          (setq tmp (1+ tmp)))
                      (goat_soup (nth tmp (nth (- c #x81) desc_list)) planet-sys); .option[()+(rnd >= 0x66)+(rnd >= 0x99)+(rnd >= 0xCC)] planet-sys))
                  (progn ;;switch...
                    (cond ((= c #xB0);;planet name
                           (setq elite-for-emacs-planet-description 
                                 (concat elite-for-emacs-planet-description 
                                         (capitalize (plansys-name planet-sys))))
                           ;;(insert (capitalize (plansys-name planet-sys)))
                      )
                     ((= c #xB1);; /* <planet name>ian */
                      (setq tmp (capitalize (plansys-name planet-sys)))
                      (if (and (not (string-match "e$" tmp)) (not (string-match "i$" tmp)))
                          (setq elite-for-emacs-planet-description (concat elite-for-emacs-planet-description tmp))
                        (progn ;;(setq tmp "helleinooio")
                          (setq elite-for-emacs-planet-description (concat elite-for-emacs-planet-description (substring tmp 0 (1- (length tmp))) "ian" ));;(insert (substring tmp 0 (1- (length tmp))))
                          )))
                     ((= c #xB2);;/* random name */
                      (setq i 0)
                      (setq len (logand (gen_rnd_number) 3))
                      (while (<= i len)
                        (setq x (logand (gen_rnd_number) #x3e))
                        (if (/= (aref pairs x) 46);;46='.' (string-to-char ".")
                            (setq elite-for-emacs-planet-description (concat elite-for-emacs-planet-description (char-to-string (aref pairs x))))
                             )
                         (if (and (> i 0) (/= (aref pairs (1+ x)) 46))
                             (setq elite-for-emacs-planet-description (concat elite-for-emacs-planet-description (char-to-string (aref pairs (1+ x)))))
                             )
                         (setq i (1+ i)))
;;                                              case 0xB2: /* random name */
;;                              {       int i;
;;                                      int len = gen_rnd_number() & 3;
;;                                      for(i=0;i<=len;i++)
;;                                      {       int x = gen_rnd_number() & 0x3e;
;;                                              if(pairs0[x]!='.') printf("%c",pairs0[x]);
;;                                              if(i && (pairs0[x+1]!='.')) printf("%c",pairs0[x+1]);
;;                                      }
;;                              }       break;

                       ))))))
            (setq source-list (cdr source-list)))))

The kind that designates a procedure built out of dead things, most of which you'd really rather not think about.

I've removed the irrelevant comments[2], but the above is still considerably longer than what I consider good style for a single function. Later on, there's a snippet of code that looks like

(defconst desc_list
  (list
;; 81 */
        (list "fabled" "notable" "well known" "famous" "noted")
;; 82 */
        (list "very" "mildly" "most" "reasonably" "")
;; 83 */
        (list "ancient" "\x95" "great" "vast" "pink")
;; 84 */
        (list "\x9E \x9D plantations" "mountains" "\x9C" "\x94 forests" "oceans")
;; ... continues for a further 69 lines

What it does in context, basically, is take the string "\x8F is \x97." and expand it out recursively until all the "byte" references are gone and it ends up with a little semi-sensical, explanatory description like "The planet is reasonably famous for its inhabitants' ingrained shyness but scourged by deadly edible wolfs." or "The planet is famous for its pink parking meters.".

The non-lisp thing I hinted at last time, and wanted to discuss this thime out, is this idea of byte-orientation. This is an architecture built by someone used to assembly or C, that then tried to shoehorn the same way of looking at the world into Lisp. I wouldn't mind so much, but it's far too easy to imagine someone hacking together a system like this and thinking to themselves "Wow, this really sucks. I could have done it MUCH more efficiently in C, and I wouldn't have had to deal with all this 'list' and 'setq' nonsense. I guess Lisp is just a language for masochists...". Going against the grain of any language creates the impression that it's less powerful than it really is, and this is a prime example.

I took a minute out at the beginning of this post to point out how this codebase re-implements random number generation at a very low level. Well, the reason I consider it an egregious mistake here is that the main place I found that particular generator used is in goat_soup above. In other words, the coder set up a list (and associated various bytes with each element), then used gen_rnd_number to generate an appropriate byte and then pick out an element from the resulting list. The really funny part is that I could see this being implemented as a performance optimization in a C version of this game, but when you're dealing with string representations of bytes that you have to split and convert before operating on, any performance gains fly directly out the window.

It's beside the point, though. Remember, Lisp is a symbolic language. So here's a Lispier way of generating some planet descriptions.

(defparameter *planet-desc-grammar*
  (list :root '((" is " :reputation " for " :subject) 
                (" is " :reputation " for " :subject " and " :subject) 
                (" is " :reputation " for " :subject 
                 " but " :adj-opposing-force " by " :historic-event)
                (" is " :adj-opposing-force " by " :historic-event) 
                (", a " :adj-negative " " :syn-planet))
        :subject '(("its " :adjective " " :place) 
                   ("its " :adjective " " :passtime) 
                   ("the " :adj-fauna " " :fauna) 
                   ("its inhabitants' " :adj-local-custom 
                    " " :inhabitant-property) 
                   :passtime) 
        :passtime '((:fauna " " :drink) (:fauna " " :food) 
                    ("its " :adjective " " :fauna " " :food) 
                    (:adj-activity " " :sport) 
                    "cuisine" "night-life" "casinos" "sit-coms") 
        :historic-event '((:adj-disaster " civil war") 
                          (:adj-threat " " :adj-fauna " " :fauna "s") 
                          ("a " :adj-threat " disease") 
                          (:adj-disaster " earthquakes") 
                          (:adj-disaster " solar activity")) 
        :place '((:fauna :flora " plantations") (:adj-forest " forests") 
                 :scenery "forests" "mountains" "oceans")
        :technology '(:passtime "food blenders" "tourists" "poetry" "discos") 
        :inhabitant-property '(("loathing of " :technology) 
                               ("love for " :technology) 
                               "shyness" "silliness" "mating traditions") 
        :fauna '("talking tree" "crab" "bat" "lobster" "shrew" "beast" "bison" 
                 "snake" "wolf" "yak" "leopard" "cat" "monkey" "goat" "fish" 
                 "snail" "slug" "asp" "moth" "grub" "ant") 
        :flora '((:fauna "-weed") "plant" "tulip" "banana" "corn" "carrot") 
        :scenery '("parking meters" "dust clouds" "ice bergs" 
                   "rock formations" "volcanoes") 
        :reputation '((:emphasis " " :reputation) 
                      "fabled" "notable" "well known" "famous" "noted") 
        :emphasis '("very" "mildly" "most" "reasonably") 
        :drink '("juice" "brandy" "water" "brew" "gargle blasters") 
        :sport '("hockey" "cricket" "karate" "polo" "tennis" "quiddich") 
        :food '("meat" "cutlet" "steak" "burgers" "soup") 
        :adjective '((:emphasis " " :adjective) 
                     :adj-local-custom :adj-fauna :adj-forest :adj-disaster 
                     "great" "pink" "fabulous" "hoopy" 
                     "funny" "wierd" "strange" "peculiar") 
        :adj-fauna '(:adj-threat "mountain" "edible" "tree" "spotted" "exotic") 
        :adj-negative '((:adj-negative ", " :adj-negative) 
                        "boring" "dull" "tedious" "revolting") 
        :adj-local-custom '("ancient" "exceptional" "eccentric" "ingrained" "unusual") 
        :adj-forest '("tropical" "vast" "dense" "rain" "impenetrable" "exuberant") 
        :adj-disaster '("frequent" "occasional" "unpredictable" "dreadful" :adj-threat) 
        :adj-threat '("killer" "deadly" "evil" "lethal" "vicious") 
        :adj-activity '("ice" "mud" "zero-g" "virtual" "vacuum" "Australian, indoor-rules") 
        :adj-opposing-force '("beset" "plagued" "ravaged" "cursed" "scourged") 
        :syn-planet '("planet" "world" "place" "little planet" "dump")))

That's the data, at any rate. The above uses all the original words and combinations from the Elisp codebase (I did add "Australian, indoor-rules" to the activity adjective list, but that's all), so the descriptions popping out of it should be the same as those coming out of goat_soup and friends. Note that the * surrounding the variable name denote a global variable[3]. Note also that instead of using byte relations, the plist approach lets me avoid splitting strings in intermediate steps. A terminal is a string and a non-terminal is an atom. The way to unfold these is

(defun expand-production (production grammar)
  (cond ((stringp production) production)
        ((symbolp production) 
         (expand-production (pick-g production grammar) grammar))
        ((listp production) 
         (reduce (lambda (a b) 
                   (concatenate 'string a (expand-production b grammar))) 
                 (cons "" production)))))

pick-g is a function that takes a key and grammar, and returns a random expansion of that key in that grammar.

(defun pick-g (key grammar) 
  (let ((choices (getf grammar key)))
    (nth (random (length choices)) choices)))

In other words,

• a string gets returned
• an atom gets expanded (by looking it up in the grammar and picking a random possible expansion)
• a list gets expanded (by expanding each of its elements)

* (expand-production :root *planet-desc-grammar*)

" is fabled for its inhabitants' ancient love for ice tennis"
* (expand-production :root *planet-desc-grammar*)

", a revolting little planet"

All I have to make sure is that these get displayed along with the planet name and we're golden. This is the sort of elegance that Lisp is capable of when you go with the grain. ~140 lines of flaming death and side effects replaced by two recursive functions and a plist that succinctly and accurately signal the intent of the programmer. I wouldn't be particularly surprised if there's an even simpler way to accomplish the same thing, actually.

Naming Things

Planet names seem like they should be implemented the same way, given what they really are. In fact, I did just implement them as another grammar that depends on the same functions to unfold, but the original takes a different approach.

;; buried at line ~103 of a single function that generates a planet
  ;set name
  ;init alphabet pairs
  (setq pair1 (* (logand (lsh (seedtype-w2 s) -8) 31) 2))
  (tweakseed s)
  (setq pair2 (* (logand (lsh (seedtype-w2 s) -8) 31) 2))
  (tweakseed s)
  (setq pair3 (* (logand (lsh (seedtype-w2 s) -8) 31) 2))
  (tweakseed s)
  (setq pair4 (* (logand (lsh (seedtype-w2 s) -8) 31) 2))
  (tweakseed s)
  ;Always four iterations of random number
  (setq planet-name
        (concat 
         (code-to-char (aref pairs pair1))
         (code-to-char (aref pairs (1+ pair1)))
         (code-to-char (aref pairs pair2))
         (code-to-char (aref pairs (1+ pair2)))
         (code-to-char (aref pairs pair3))
         (code-to-char (aref pairs (1+ pair3)))))
  (if (/= longnameflag 0)
      (progn
        (setq planet-name 
              (concat
               planet-name
               (code-to-char (aref pairs pair4))
               (code-to-char (aref pairs (1+ pair4)))))))
  (setf (plansys-name thissys) (stripout planet-name "."))

pairs showed up in goat_soup too, and it's defined as

(defconst pairs 
  "..LEXEGEZACEBISOUSESARMAINDIREA.ERATENBERALAVETIEDORQUANTEISRION"
  "Characters for planet names.")

s is passed as an argument to the planet-generator, the rest of which I won't inflict upon you, I think you may have gotten the idea already. In other words, planets are restricted to 4 syllable names put together by side-effect in the procedure that creates planets. They're put together by using byte operations on a string that represents the valid pair combinations contained in a planet name.

Well, seeing as I already put together a convention for unfolding components to strings by using a recursive function and a plist, I figured I'd do the same for this. Code reuse is good, I hear.

(defparameter *planet-name-grammar*
  ;be mindful of name probabilities if you try to reduce duplication here
  (list :root '((:starter :link :ender) (:starter :partition :ender) 
                (:starter :partition :link :ender) 
                (:starter :partition :root) 
                (:starter :link :link :ender) (:starter :ender))
        :starter '((:starter :link)
                   "aa" "ae" "al" "an" "ao" "ar" "at" "az" "be" 
                   "bi" "ce" "di" "ed" "en" "er" "es" "ge" "in" 
                   "is" "la" "le" "ma" "on" "or" "qu" "ra" "re" 
                   "ri" "so" "te" "ti" "us" "ve" "xe" "za")
        :ender '((:link :ender) 
                 "aa" "al" "at" "di" "ti" "so" "ce" "re" "za" 
                 "in" "ed" "or" "an" "ma" "ab" "ge" "aq" "en" 
                 "ri" "ve" "ag" "qu" "us" "es" "ex" "ae" "on" 
                 "bi" "xe" "le" "is" "er" "be" "la" "ar" "az" 
                 "io" "sb" "te" "ra" "ia" "nb")
        :link '((:link :link) (:link :link)
                "at" "an" "ri" "es" "ed" "bi" "ce" "us" "on" 
                "er" "ti" "ve" "ra" "la" "le" "ge" "i" "u" 
                "xe" "in" "di" "so" "ar" "e" "s" "na" "is" 
                "za" "re" "ma" "or" "be" "en" "qu" "a" "n" 
                "r" "te" "t")
        :partition '("-" "'" " ")))

[4]And that's that. It unfolds into planet names with the same mechanisms

* (string-capitalize (expand-production :root *planet-name-grammar*))

"Ri Orleio"
* (string-capitalize (expand-production :root *planet-name-grammar*))

"Xenain"

* (string-capitalize (expand-production :root *planet-name-grammar*))

"Es'Ae"

string-capitalize just makes sure it looks like a proper name (it's a lisp primitive, so I won't define it here). The important part, which I'll likely cover in a future post, is that making things functional aids in composeability. The setq sequence from the original codebase has no hope of being reused anywhere because it intentionally grubs about in the surrounding state. If nothing else, the expand-production approach ensures that if I ever need a planet name in some other context, I can easily generate it. Also, as we've seen already, abstracting out the general pattern of "compose strings from a given pattern of components" easily pays for itself with even one instance of reuse.

My Sinister Purpose

The reason I've been picking away at this codebase isn't idle fancy[5], or an intense hatred of Sami Salkosuo[6]. It's that I've been putting together a little web game based on it. It's still not done, mind you, but I'm going to try to put something together fairly soon for you to poke at (even if it's ugly as sin from the visual perspective to start with). If nothing else, I'm tossing the ported Common Lisp codebase up onto my GitHub this weekend so that some other pedantic bore can pick apart a project I was just doing in my spare time. After two articles full of bitching about poor style and inelegant expression, it seems like it's only fair.

---

Footnotes

1 - [back] - Especially considering the situation.

2 - [back] - Which weren't particularly horrible, most of them were upholding the time-honored tradition of testing by printf

3 - [back] - It's not enforced except in one or two places, but it is a style convention. SBCL will bitch at you if you try to do something like (let ((*foo* '(bar baz))) ...), for example. Not refuse to run it of course, but it'll warn you that it's using a lexical binding for *foo*.

4 - [back] - That's the first time that my blog-mode highligher chugged for a second before coming back with the result, by the by, take a look at the source code and see if you can see why.

5 - [back] - Or at least, not just idle fancy.

6 - [back] - Which I don't have. I've never met the guy, and he couldn't possibly have known that I'd be going through his hobby-horse with a sledgehammer at some point in the distant future of 2011.

Switching

This won't be a long entry, but I still have to get it down out of my head before long.

I had this 24 inch iMac around for a few years. Got it back when 2GB of ram was a lot and a 2.4 Ghz single core was blazingly fast. Played around with it for a while and it served well for various design tasks. When I switched gears to more development than design a couple of years ago, I also put together a modest Linux machine. It actually started out with worse specs than the Mac. A much smaller hard drive, 1GB of ram and I forget how fast the processor was. Definitely not very; I remember getting the cheapest AMD I could find. It could grow though. I've always liked tinkering and building machines, but having used Macs through my university years made me forget how much fun it was for a little while. Before long, I was tuning and tweaking again. A slightly better processor here, a bit more ram there. Eventually, SSDs got cheap enough that I could afford a small one. The Linux machine began to rival, and eventually surpass the shiny giant.

I kept the Mac as a design machine, just to run Photoshop/Illustrator and (occasionally) Flash. After about half a year of this, I realized that my time was spent primarily in Emacs on the Linux machine and secondarily in a browser/terminal (on either machine). The miniscule remainder was actually using Photoshop/Illustrator. I had also taken up GIMP for smaller jobs, just so I wouldn't have to switch back and forth between computers. It became obvious that as shiny as it was, the Apple desktop wasn't doing much for me, so I gave it to my fiancee. She fell in love with it, not that she would admit that as a former Windows user. And I mean "user" in the sense of "end user". A computer is a tool that lets her do the stuff she's really interested in. She doesn't care how it functions on the inside, as long as it does what she wants it to. In any case, she got quite comfortable with the Apple setup in short order.

Well, earlier today, the Mac died.

She was doing something random with her Kindle when it shut down randomly. It wouldn't come back up, or respond to any of the start-up keys, and it wouldn't boot from its installation DVD either. So we were staring down the OS X equivalent of the blue screen of death. The problem potentially wasn't as simple as the hard drive getting borked. Given that the mere process of replacing a hard drive for one of these units involved supplies we'd have to go shopping for, she decided she'd just switch[1]. She wouldn't go back to Windows, so I got her onto Ubuntu without much convincing.

Ok, that was the background. Here's the story.

I mentioned being a graphic designer. Actually that should probably say "Graphic Designer" because I have the degree to back up those capitals. So when I say that I underestimated the importance of UI on the decision making of end-users, understand the implications.

She was intensely disappointed by pretty much everything she's seen of other Linux machines, so I more or less gave up on turning this into a house that respects the four freedoms [2]. The thing that ended up placating her is, and I shit you not, cairo-dock set to auto-start at login with the spaces widget removed and the trash icon enabled. OpenOffice is close enough to MS Word for the stuff she does, Rhythmbox syncs with her iPod and her browser of choice is available. She still wants Photoshop, so I may need to resort to some VirtualBox shenanigans, but I'm hoping to get her using GIMP instead. Those are nitpicks though. It turns out that dock and the iPod syncing were the deal-breakers.

This isn't meant to be funny, by the way. It's here to serve as a reminder of how small a change it takes for an end user to willingly switch.

---

Footnotes

1 - [back] - I've got some extra supplies lying around. Nothing terribly impressive, but still enough to put together a half-way decent backup machine in an hour or so if I need to.

2 - [back] - It's still not incidentally. As I said, she owns a Kindle, and an iPod from a while back, but our desktops are now all open source software and commodity hardware.