Direct Manipulation, and Text Editors

Hat-tip to Giles Bowkett for mentioning Bret Victor’s talk, Stop Drawing Dead Fish, about direct manipulation and computer interaction as a tool for illustrators and animators.

I paused this video part-way through, so I could Get Down to Work. I paused it around 34:30, right after Bret Victor mentioned David Hestenes‘ idea that algebra and geometry help us model the world in similar ways, but that algebra uses our linguistic ability, while geometry uses our visual/spatial perception. Victor went on to say that tools for visual artists should use our visual/spatial abilities (direct manipulation), rather than our linguistic ones (code).

Like I said, it was time to Get Down to Work. I flipped over to Sublime Text 2, where I happened to have a block of text selected. Warming up, I idly hit command-I to search for a bit of text, only to realize that Sublime was only searching in that selected text. This is handy! I’ve been wanting something like this lately, when a method or variable name shows up all over the file, but I’m only working on one method.

Using the trackpad to select a bunch of text, and then working on it, feels a lot like I’m holding the text in one hand, and working on it with Sublime in the other. I discovered this accidentally, too – I’ve felt pretty productive with Sublime after the (tiny) initial bump, and I occasionally, gradually, get better as I learn new tricks.

I switched to Sublime after trying to learn Vim for a month or so. I’d been an emacs user for a few years, but I was under-using it, and didn’t need all the extra machinery. Vim seemed lighter, so I tried it out. But it felt like learning emacs all over again, but with different incantations: everything forced into the keyboard. And I get that! I’m a home-row Dvorak typist! But I still felt like emacs and vim were a step back from humbler tools like Programmer’s Notepad.

Bret Victor’s talk suggests an interesting explanation for that. He points out that some animations are a pain to create manually, and some behaviors are hard to code into the tool: so you divide and allocate tasks to the computer and the artist according to their abilities, and their cooperation produces the best effect.

Maybe this explains the appeal of these less-hardcore text editors. Sure, using the mouse and buttons for everything, a la Microsoft Word, is tedious, but so is forcing all interaction through the keyboard. Maybe a better allocation of tasks, a better balance of responsibilities between typist and tool, is what’s needed.

Chaos, Order, and Software Development

Zach Dennis gave a very interesting, but not terribly well-received talk at RailsConf 2012, called “Sand Piles and Software.” (It’s on the schedule on Tuesday in Salon J, if you want to check it out.) Here are the slides (which are more suggestion than information), and here’s the synopsis:

This talk applies the concepts of chaos theory to software development using the Bak–Tang–Wiesenfeld sand pile model [PDF link] as the vehicle for exploration. The sand pile model, which is used to show how a complex system is attracted to living on the edge of chaos, will be used as a both a powerful metaphor and analogy for building software. Software, it turns out, has its own natural attraction to living in its own edge of chaos. In this talk, we’ll explore what this means and entertain questions for what to do about it.

The TL;DR of the talk was: as you build your software system, as you add features, you add complexity, and when it’s too complex, you won’t be able to add anything more, until you clean something up. So you clean a bit up, and add more complexity, until it falls over again. Like dropping grains of sand onto a sand pile, each grain is tiny, hardly worth noting, but one of them will cause a slide.

That much rang very true with me.

Zach’s advice, then, was to “fall in love with simplicity,” and “loathe unnecessary complication,” and there are some more slides about practices and values and refactoring, but I can’t remember the ideas for them; I’ll have to check my notes.

To me, that part sounded virtuous.

This morning, I turned again, for other reasons, to Dick Gabriel’s Mob Software: The Erotic Life of Code. (I’ll say it until I stop meeting programmers who haven’t read him: you are missing out.) I got to the part where he talks about swarms (he’s preparing to introduce us to the Mob, the open-source hackers), and complexity emerging from local actors with simple rules, and this part reminded me of Zach Dennis’ talk:

Chaos is unpredictability: Combinations that might have lasting value or interest don’t last—the energy for change is too high. Order is total predictability: The only combinations that exist are the ones that always have—the energy for stability is too high.

He goes on to quote Stuart Kauffman from “At Home in the Universe”:

It is a lovely hypothesis, with considerable supporting data, that genomic systems lie in the ordered regime near the phase transition to chaos. Were such systems too deeply into the frozen ordered regime, they would be too rigid to coordinate the complex sequences of genetic activities necessary for development. Were they too far into the gaseous chaotic regime, they would not be orderly enough.

…cell networks achieve both stability and flexibility…by achieving a kind of poised state balanced on the edge of chaos.

Is Zach telling us to stay where it’s safe and ordered? Are we stuck on this edge between chaos and order, if we want to write interesting software? I’d like my software to be both stable and flexible. If, to achieve this stability and flexibility, its behavior must be emergent, not guided by my brain, is that ok? Or is there a way for me to still specify requirements, and get this stability and flexibility? Is emergent-design only able to produce certain kinds of software?

One of Zach's slides: reaching your software's critical point

Thanks to Ren for reviewing this!

Book Review: Pragmatic Thinking and Learning

I was planning on reading Andy Hunt’s Pragmatic Thinking and Learning: Refactor Your Wetware for a while, but kept putting it off, thinking it wasn’t really central to what I’m interested in, what I’m doing.  I was wrong.

People have different skill levels, from novice to expert.  One sign of expertise is the ability to intuit solutions not available through linear thinking.  Intuition happens on the right side of the brain, so we need to use the right side more effectively.  Rather than “left brain” and “right brain”, he calls them “L-mode” and “R-mode”, for “linear” and “rich”, to emphasize that it’s not literally two halves of your brain working differently, but two different modes of thinking.  Most of the book is about using R-mode more, and using L-mode appropriately, to fact-check your R-mode.

It draws on psychology, neurobiology, cognition studies, managing, teaching, the arts, product design, and math.  The bibliography is ten pages long, and I’ve got a bunch of them highlighted for my next evening at the bookstore.

The most valuable changes I’ve made since finishing the book are:

- Be ready when your R-mode strikes. Since “querying” the R-mode can take a long time, we run it asynchronously, and we never know when the results will come in.  (This explains the “ah-ha!” moments in the shower, on the drive home, and when you’re falling asleep.)  So keep a notebook handy for writing things down.  I’m writing in a notebook almost daily (I’ve joined the cult of moleskine), writing down ideas as they come up.  It’s also a tider home for all the scraps of paper that have lived in my wallet for years, ideas scrawled and smudged into them.  And I’m on my 4th or 5th pocket mod.

- Maintain your ‘exo-cortex,’ your external memory. I wrote a mini-wiki engine with Sinatra, and so far, I’ve used it for everything from clojure, to saving quotes, to robot rights.  I wrote it for practice, but it’s nice to have a searchable, non-linear notebook.

The book mentions Gerald Weinberg’s fieldstone method of writing: to build a stone wall, you gradually gather stones from the field as you clear it, and pile them up.  When you have enough, build your wall.  Both the wiki and the notebook serve this purpose.

- Map out your thinking, and doodle. Drawing pictures engages your R-mode, making it more active, helping you make connections, and understand more clearly.  Besides, it’s fun. This is my 3rd time learning emacs, and it’s finally sticking — I think it’s mostly because I drew up a cheat-sheet, pictorially explaining each command’s effect.  I also map out my learning and career goals this way.

emacs cheat sheet

One of my favorite parts is, after you’ve mapped something out, and the drawing looks like a pile of yarn, re-draw it on a clean sheet, fixing up all the awkward placements and messy bits.  The benefit of this second version is in its creation — by considering how to re-organize it, you’re thinking more about the problem, in ways you might not have before.  This works for note-taking, too.

- Take advantage of your brain’s plasticity. You can change how your brain works by believing it works differently: make it smarter, more creative, cheerier.  This is a pretty incredible claim, but he cites two books I’d like to check out.  It sounds impossible, but I’ve seen it work on me, to some extent.  My biggest problem is continuing to believe it works.

Some things I have yet to try:

- Take a walk.  A meditative walk, like around a labyrinth.

- Write morning pages: for three weeks, as soon as you wake, write down three pages of text.  It doesn’t have to be anything specific.  The idea is, right after we wake, our R-mode is more active, so the gems will be more accessible, and they’ll pop out onto the page.  I’m more curious to see what I come up with.

- I have to get better at setting SMART goals: specific, measurable, achievable, relevant, and time-boxed.  I’ve never been much for this kind of discipline — I made a few, but didn’t stick to them, and haven’t gotten into the habit yet.  But like the book says, change is hard:

Change is always harder than it looks — that’s a physical reality, not just an aphorism. An old, ingrained habit makes the equivalent of a neural highway in your brain. These old habits don’t go away. You can make new neural highways alongside, going a different route and making short-cuts, but the old highways remain. They are always there for you to revert to—to fall back on. Practice may not make perfect, but it sure makes permanent.

Heading to the Int’l Lisp Conf, 2009

My next step on the road to lisp is at the International Lisp Conference in Cambridge, MA, this March.

A lot of people are surprised to hear I’m interested in Lisp, but I’ve been studying scheme informally for about 3 years, reading SICP, and finishing The Little Schemer last year. I’ve been interested in clojure for about 18 months, especially after seeing Rich Hickey talk about it last March.  Now I’m studying it in earnest with Stu Halloway’s Programming Clojure.

I don’t use lisps of any sort at work, so it’s not a “practical” language to study like, say, C# is.  But that’s short-term thinking.  Learning lisp is plenty practical, if you’re looking longer-term — even if it’s only three years out.  Learning lisp teaches you about some of the core issues of programming.

I like learning lisp because it’s a window into the workings of programming languages, being so close to the AST.  I want to learn more about meta-programming with macros, hygienic or not.

And for the record, the functional techniques in The Little Schemer have improved my Ruby, JavaScript, and C# in some really solid ways. Functional programming is making its way to the rest of the programming world, and lisp does functional programming.

If you’ll be at the conference, I’d love to hear from you.

Why “Less Code” Matters

…being able to do task X with 50 lines of code is preferable to needing 500 lines of code to do task X. Less code takes longer to write, but the real benefits are around maintenance: less code means less of a chance of bugs, less to keep in your head, less for someone else (or yourself 6 months later) to read through and learn, less to test, and less to modify when you change the rest of the system.

- Alan Keefer, Syntax Matters

I’d like to expand on that. I don’t think it’s clear how important “less code” is, or how harmful more code is. So let’s take an example written in a Blub-y language, and see how well we can refactor it.

(I know this post is kind of long, but it’s mostly Blub code, and it should scan quickly.)

Let’s make a sandwich.

routine makeSandwich
    look for the peanut butter in the cabinet
    if it's not there, look for it in the other cabinet
    put the peanut butter on the counter

    look for the jelly in the fridge
    if it's not there, look for it in the cabinet
    if it's not there, look for it in the other cabinet
    put the jelly on the counter

    find a napkin
    put the napkin on the counter

    find the bread in the bread drawer
    untie the bread bag
    take two pieces of bread from the bag
    close the bread bag
    put the bread back in the bread drawer
    put the two pieces of bread on the napkin

    find a butter knife
    put the butter knife on the napkin

    open the peanut butter jar
    stick the butter knife into the peanut butter jar
    with the butter knife, scoop out some peanut butter
    spread the peanut butter on one piece of bread
    close the peanut butter jar
    put the peanut butter back in the cabinet

    wipe the butter knife on the other piece of bread

    open the jelly jar
    stick the butter knife into the jelly jar
    with the butter knife, scoop out some jelly
    spread the jelly on one the other piece of bread
    close the jelly jar
    put the jelly back in the fridge

    put the knife in the sink

So much work! No wonder I seldom cook. Can we improve that at all? Well, the “looking for in 2 cabinets” seems to be a pattern, so let’s Extract Method:

routine lookForInTwoCabinets (lookFor)
    look for the lookFor in the cabinet
    if it's not there, look in the other cabinet
    return it

routine makeSandwich
    lookForInTwoCabinets (peanut butter)
    put the peanut butter on the counter

    look for the jelly in the fridge
    if it's not there, lookForInTwoCabinets(jelly)
    put the jelly on the counter
    ...

Can we move the “put it on the counter” inside lookForInTwoCabinets? I don’t know…it would work for the peanut butter, but what if we find the jelly in the fridge? In that case, we wouldn’t call lookForInTwoCabinets(jelly), so we might never put the jelly on the counter. Besides, the name doesn’t really imply anything about what we do after we find the thing. We should probably leave it outside. Yeah, it’s not so DRY, but let’s move on.

That big block where we look for bread, we can’t really compress it at all…but we can extract it, just to wrap the whole sequence of steps up with a name.

...
routine getBread
    find the bread in the bread drawer
    untie the bread bag
    take two pieces of bread from the bag
    close the bread bag
    put the bread back in the bread drawer
    put the two pieces of bread on the napkin

routine makeSandwich
    ...
    find a napkin
    put the napkin on the counter

    getBread

    find a butter knife
    put the butter knife on the napkin
    ...

Ok, we’re making progress. What about spreading the peanut butter & jelly on the bread? Can we extract another method?

routine spread (topping, breadSlice)
    open the topping jar
    stick the butter knife into the topping jar
    with the butter knife, scoop out some topping
    spread the topping on breadSlice
    close the topping jar
    put the topping back in the cabinet

routine makeSandwich
    ...
    find a butter knife
    put the butter knife on the napkin

    spread (peanut butter, one piece of bread)

    wipe the butter knife on the other piece of bread

    spread (jelly, the other piece of bread)

    put the knife in the sink

Great! Except we just introduced a bug: after closing the topping jar, spread always puts the topping back in the cabinet, and the jelly goes in the fridge (moldy jelly is a Bad Thing). Introduce Parameter:

routine spread (topping, breadSlice, returnToppingTo)
    open the topping jar
    stick the butter knife into the topping jar
    with the butter knife, scoop out some topping
    spread the topping on breadSlice
    close the topping jar
    put the topping back in returnToppingTo

routine makeSandwich
    ...
    find a butter knife
    put the butter knife on the napkin

    spread (peanut butter, one piece of bread, the cabinet)

    wipe the butter knife on the other piece of bread

    spread (jelly, the other piece of bread, the fridge)

    put the knife in the sink

Ok, I think we’re done. (Does it make sense to send a “return topping to” parameter to a method that’s just spreading? Not now, we’re almost ready to commit…) Let’s step back and admire our craft:

routine lookForInTwoCabinets (lookFor)
    look for the lookFor in the cabinet
    if it's not there, look in the other cabinet
    return it

routine getBread
    find the bread in the bread drawer
    untie the bread bag
    take two pieces of bread from the bag
    close the bread bag
    put the bread back in the bread drawer
    put the two pieces of bread on the napkin

routine spread (topping, breadSlice, returnToppingTo)
    open the topping jar
    stick the butter knife into the topping jar
    with the butter knife, scoop out some topping
    spread the topping on breadSlice
    close the topping jar
    put the topping back in returnToppingTo

routine makeSandwich
    lookForInTwoCabinets (peanut butter)
    put the peanut butter on the counter

    look for the jelly in the fridge
    if it's not there, lookForInTwoCabinets(jelly)
    put the jelly on the counter

    find a napkin
    put the napkin on the counter

    getBread

    find a butter knife
    put the butter knife on the napkin

    spread (peanut butter, one piece of bread, the cabinet)

    wipe the butter knife on the other piece of bread

    spread (jelly, the other piece of bread, the fridge)

    put the knife in the sink

31 lines down to…32 lines. Ok, well, even if it’s longer, is it better? makeSandwich is shorter, that’s good. But it doesn’t feel like we’ve really made the job any easier — we moved stuff around, but it’s still all there. There’s no semantic compression. It’s still 3 + 3 + 3 + 3 + 3 + 3, instead of 3 * 6.

What did we think about? We had to ask ourselves whether to move “put it on the counter” into lookForInTwoCabinets. The value of getBread is questionable. We had the bug with spread putting the jelly in the cabinet, and we had to wonder about its “return topping to” parameter. Every time we consider refactoring, we risk introducing a crappy abstraction that confuses, when it should clarify. Every decision point, we have to think about it, and we might get it wrong. But that’s why they pay us the big bucks, right? Software development is hard, after all!

No. We’re looking at accidental complexity, not essential complexity. Here’s the same code, in a higher-level language, that removes some of the accidental complexity:

put peanut butter on a piece of bread
put jelly on another piece of bread
stick the peanut butter to the jelly

Essential complexity is when you start thinking, why jelly? Why not cinnamon and raisins with the peanut butter? Or currants? What kind of bread? Let’s use multigrain. Would peanut butter with jelly and banana be overkill? What to drink? Essential complexity looks at the problem, not the solution. Accidental complexity is when you say “I really want to do THIS but dammit, my language just won’t let me.” Or, “Gosh, we have so much code to move around, I can barely see what it does.” Or when you just can’t figure out where to put that parameter, or method, or class.

So what does this have to do with “less code”?

This is why we say YAGNI. If you add that method on a hunch that it’ll be helpful, you have more stuff to move around, more accidental complexity, more decisions to make about your housekeeping, all for a speculative benefit. It’s like playing lotto – you pay up front, and if you’re really lucky, you’ll win. But if you lose, you’ve wasted resources, and now you have something you need to throw away.

Each of the possible ways to code and refactor that sandwich code is pretty valid…any of them could be in our source control repository. A new hire is going to have to read through whichever one we coded, and try to mentally get from there, to the 3-liner at the end, before he can really be effective. Why don’t we just start him there?

Let’s take that 3 + 3 + 3 + 3 + 3 + 3 example again. What if we don’t use multiplication? We could still refactor it. The first two threes are kind of together, let’s group them: 6 + 3 + 3 + 3 + 3. And the last one looks kind of bulky, so let’s decompose it: 6 + 3 + 3 + 3 + 1 + 1 + 1. Could we move some of the numericality from the middle 3 to an earlier one? 6 + 3 + 4 + 2 + 1 + 1 + 1. Oh, and let’s sort, so it’s easier to find the numbers you want: 1 + 1 + 1 + 2 + 3 + 4 + 6. There! Is it immediately obvious to you that this is the same as 3 * 6? Of course not. Ralph Johnson calls refactoring “wiping dirt off a window,” and you just put more dirt on.

Passing by reference, and dog leashes

Pass-by-reference and pass-by-value are pretty confusing when you start learning to code. When I first saw them, I know I ignored the distinction (until I got tired of my code not doing what I expected). Throwing collections into the mix just makes it worse.

Today, though, we stumbled on a pretty decent analogy for passing-by-reference: a reference to an object is like a leash to a dog. Let’s take our dog Dagwood for a walk.

Dog dagwood = new Dog(“Dagwood”);

new Dog() creates, of course, a new Dog object. Dog dagwood creates a reference that can point to any Dog object — it’s really the leash, but we name our references for what they point to, rather than what they are: a reference, a handle, a leash. The equals sign takes the leash, and hooks it to Dagwood’s collar. Now we can take Dagwood for a walk.

dagwood.walk();

To tell Dagwood it’s time to walk, we tug on the leash. He feels the tug, and gets the message, so he starts following us. We come to a busy road, and wait for the crossing signal, but Dagwood’s oblivious, and tries to cross anyway.

dagwood.halt();

Since we’re stopped, he feels the tug of the leash again, gets the message, and stops. We’re sending messages to Dagwood through his leash. In OO terms, sending a message to an object means calling one of its methods. We’re calling methods on our Dagwood through our reference to him, through the leash.

Storing a reference in an array

In the park, we find a snack shop. We’re getting hungry, but the snack shop doesn’t let dogs inside. Luckily, there’s a chain link fence, and in our eyes, a chain link fence is nothing but a big row of places for us to attach a dog leash. We tie a spare leash to the end of the fence, and attach it to Dagwood’s collar.

Dog[] fence = new Dog[10]; // only room for 10 dogs
fence[0] = dagwood;

What’s happening here in OO terms is that our reference to Dagwood, our leash, is copied into the zeroth slot on the fence. It’s not our leash, but it’s one just like it. So now there are two leashes on Dagwood: one in our hand, and one on the fence. We’ll take our leash off Dagwood, since we can’t very well hold it while we’re in the store.

dagwood = null;


Don’t worry, he’s fine…he’s still tied to the fence, by that other leash. Let’s go buy cashews.

When we come out of the store, we want to re-attach our leash to Dagwood.

dagwood = fence[0];

Now let’s untie him from the fence, and head over to the lake.

fence[0] = null;

Passing by reference

Passing references to methods works in much the same way. Dagwood got kind of stinky, swimming in the lake, so let’s bring him to the groomer for a bath.

DogGroomer.shampoo(dagwood);

When you pass a reference to a method, your reference is copied into that method. Again, it’s like a new leash, one just like ours, springs into the groomer’s hand — now Dagwood’s attached to us, and the groomer. He gets fidgety when he’s getting bathed, so it’s just as well.

From the groomer’s perspective, it might look like this:

void shampoo(Dog doggie) {
wet(doggie);
apply(shampoo, doggie);
rinse(doggie);
towelDry(doggie);
}

The groomer doesn’t care what Dagwood’s name is, she just keeps calling him “doggie.” That’s ok, she must see a lot of dogs during the day…names aren’t that important to her. The interesting thing is, even though it’s the groomer who’s shampooing our dog, since we still have a leash on him, we can observe him getting cleaner.

When she’s done, the procedure ends, the method returns, and her leash to Dagwood disappears. Which is fine, because he’s stopped fidgeting, now that he’s dry.

Garbage collection

We head back home through the park. Dagwood’s itching to run around, but we’re tired, so we just unleash him. Hopefully we can find him before it gets dark…

dagwood = null;


Unfortunately, the dog catcher spots him running around without a leash, which is illegal in these parts — a stray dog will hang around forever, eating up resources. The dog catcher carries off our poor Dagwood, and destroys him. We take it in stride, and try to keep the whole circle of life allocation-deallocation in mind.

So…

So that’s how references work. It’s why code like this (C#) will ensure the balloon bouquet has at least one balloon that says “Happy Birthday!”:

List balloons = GetBalloons();
Balloon printed = balloons.Find(Balloon.IsPrinted);
if (printed == null) {
printed = new Balloon();
printed.PrintMessage(“Happy Birthday!”);
balloons.Add(printed);
}
return balloons;

Why We Abstract, and What To Do When We Can’t

Whenever you see yourself writing the same thing down more than once, there’s something wrong and you shouldn’t be doing it, and the reason is not because it’s a waste of time to write something down more than once. It’s because there’s some idea here, a very simple idea, which has to do with the Sigma notation…not depending upon what it is I’m adding up. And I would like to be able to always…divide the things up into as many pieces as I can, each of which I understand separately. I would like to understand the way of adding things up, independently of what it is I’m adding up.

- Gerald Sussman, SICP Lecture 2a, “Higher-order Procedures” (emphasis added)

The purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.

- Edsger W. Dijkstra, The Humble Programmer

What Larry Wall said about Perl holds true: “When you say something in a small language, it comes out big. When you say something in a big language, it comes out small.” The same is true for English. The reason that biologist Ernst Haeckel could say “Ontogeny recapitulates phylogeny” in only three words was that he had these powerful words with highly specific meanings at his disposal. We allow inner complexity of the language because it enables us to shift the complexity away from the individual utterance.

- Hal Fulton, The Ruby Way, Introduction (emphasis added)

Programming is our thoughts, and with better ways to express them, we can spend more time thinking them, and less time expressing them.

3 + 3 + 3 + 3 + 3 + 3 is hard…hard to read (how many threes?), hard to get right (I lost count!), hard to reason about (piles of operations!). 3 x 6 is easy, once you learn multiplication. This is a good trade-off. We should look for ways to add abstractions, new semantic levels, to our programs.

If you’re doing the same thing twice, stop, and look for the common idea. Peel the idea away from the context, from the details. Grasp the idea, and then use it over and over. As a bonus, you’ll type less, re-use code, and debug less.

“But I can’t find ways to do that!”

When you look at similar bits of code, and can’t find a good way to remove the duplication, you’re hitting the limits of either your language, or your knowledge.

Programming languages put up very real walls, they force you down their paths, often by leaving out features. A language without recursion puts up a wall in front of recursive solutions; a language without first-class functions makes it tough to write higher-order functions. Language limitations are the cause of Greenspun’s Tenth Rule.

Sometimes, the language is not the problem. Sometimes you just can’t find your way through. This is why you read Refactoring, and Design Patterns, but really, this is why you learn other programming languages. Think about the right way to factor the problem.

If you can’t remove the duplication, you need to work around your language, or learn some new tricks.

Fear, Uncertainty, and Doubt

Of all the monsters that fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, one seeks bullets of silver that can magically lay them to rest.

The familiar software project, at least as seen by the nontechnical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet…

- Fred Brooks, No Silver Bullet

Software is full of failure, and will be until we can learn how truly to build it. Fear of failure is fear of death. In fear of failure, we seek order.

- Richard Gabriel, Mob Software

Eventually, people begin to realize that dealing with fear is not good; instead we need to be adults, taking calculated risks and living with the consequences. We need to drive out fear in the organization (think Deming) and instead live with uncertainty. This brings us to what we called (again, lack of a better term) “second generation” methodologies that are a backlash to the first. XP is a great example – the whole point is simplest process that could possible work, combined with just enough safety nets that you can live with a little risk. Stop living in fear that the developers won’t hit some meaningless deadline for a huge piece of work, and instead have a simple working system up in weeks and add features as you go. Duh.

As I said before, these second-generation methodologies are, for the most part, just a backlash to the first. They are saying “well, heck, too much process didn’t work. Let’s take away and take away and take away process untill we have something workable.”

So, second generation languages are still caught up in fear – that is, driving it out. That’s half the sales pitch. If we just take those gates down, maybe pave the road, we will get home faster.

- Pothole Methodologies

Perhaps the persistent tendency to harden insight into unquestioned doctrine arises from our need to feel secure, to gain a hold on our situation and stave off the fear that things are dangerous and beyond our control. It’s an attitude that naturally appears when dealing with illness, aging, and mortality, the greatest of fears; and the fact that we really aren’t in control, that we can’t wield power over such awesome eventualities, just makes us crave answers and solutions all the more.

- Jeff Wilson, White Ashes at Killing the Buddha

To truly learn a new language, you have to learn the different things. When you find something difficult, something that doesn’t make sense, you don’t avoid it, you embrace it. This is where the actual learning takes place.

- Raganwald, The challenge of teaching yourself a programming language

Ordinarily we are swept away by habitual momentum. We don’t interrupt our patterns even slightly. With practice, however, we learn to stay with a broken heart, with a nameless fear, with the desire for revenge. Sticking with uncertainty is how we learn to relax in the midst of chaos, how we learn to be cool when the ground beneath us suddenly disappears. We can bring ourselves back to the spiritual path countless times every day simply by exercising our willingness to rest in the uncertainty of the present moment — over and over again.

- Pema Chödrön, Comfortable with Uncertainty: 108 Teachings, pg 8

Doubt is not a pleasant condition, but certainty is absurd.

- Voltaire

How to Design A Domain Specific Language, with David Pollak

I just finished watching David Pollack’s presentation at Google, How to Design A Domain Specific Language. It’s only an hour, and it’s got some interesting ideas. A nice jog, good for your mental health.

His main theme is bringing computing closer to the business users. Computers exist to help us solve problems, but most people can’t program, so we need all these programmers to translate for the business people, but the translation is often imperfect, so we go back and forth like couriers between the business and the machine. Don’t we have better things to be doing? This idea of letting business users program often riles programmers who fear being automated out of a job, but I think there’s enough hard stuff out there for anyone who’s up for cracking some hard nuts. I say, let’s make the drudge work easier, so we can get on with the real work.

The essence of a DSL is building the semantics of the business into a mini-language, so the business people can read the code. [Sure, they could probably write it themselves, but most don't want to.] And if they can read the code, they can sign off on the code. Why translate specs from business-speak into Java? Why not make the code so clear, it is the spec?

Pollack points out that if you try this, you have to convince the business that the translation works. I’ve seen this before…when your code is readable by the business, they’ll forget that it’s code. “How do we know the software does what the requirements say?” Because the requirements basically are the software. Once they grok it, though, you can almost remove yourself from the equation. Pollack says of his DSL SiteMap:

We were able to go through with…the business users themselves, to figure out [the navigation rules] for each page…We never had a failure where the business user didn’t get something they were expecting to get. We had to demonstrate early…that the implementation of the code matched the semantics of the DSL. Once we proved that, the code reviews went really really quick.

That’s a great place to be.

BTW, thanks to Peter Cooper for sharing that video. He also shared Code Generation With Ruby with Jack Herrington…Herrington’s book Code Generation in Action has been on my wishlist ever since I read the sample chapters Manning let me download, so I’m really looking forward to the video.

Programming, Zen, and…Origami

I’ve started re-reading Zen and the Art of Motorcycle Maintenance. There are so many gems in it that I’ll probably start a page in my backpack for them. Here’s one I like:

”…I’ve a set of instructions at home which open up great realms for the improvement of technical writing. They begin, `Assembly of Japanese bicycle require great peace of mind.’”

This produces more laughter, but Sylvia and Gennie and the sculptor give sharp looks of recognition.

“That’s a good instruction,” the sculptor says. Gennie nods too.

“That’s kind of why I saved it,” I say. “At first I laughed because of memories of bicycles I’d put together and, of course, the unintended slur on Japanese manufacture. But there’s a lot of wisdom in that statement.” … “Peace of mind isn’t at all superficial, really,” I expound. “It’s the whole thing. That which produces it is good maintenance; that which disturbs it is poor maintenance. What we call workability of the machine is just an objectification of this peace of mind. The ultimate test’s always your own serenity. If you don’t have this when you start and maintain it while you’re working you’re likely to build your personal problems right into the machine itself.”

I think this peace of mind is needed if you want to achieve that mental state called flow. I’ll bet that “workability of the machine” translates nicely into the design of the [programming language, platform, library] you’re using – I like Ruby because it gives me greater peace of mind! – but that’s another discussion. For now, I want to talk about two ways I have of getting back that peace of mind, when I lose it.

Origami!

I learned all the origami I know from a book I got in Disney World when I was 10, and I’ve been folding on and off since then. I’m good enough to impress people who either have never tried it, or are very polite. Man, did I get tired of doing those dollar-bill bow-ties for relatives to leave as fancy restaurant tips.

Folding origami is a really interesting way to spend time (no joking!). You line up edges and folds as perfectly as you can, because the first few times, when you weren’t so careful, it came out like crap, and you learned that origami is a precise art. So you check twice before creasing the paper, and get it as perfect as you can. But the paper gets in the way! If you fold paper over itself enough times, you find that its thickness only seems negligible. You can get really thin paper and try to ignore this, but eventually, it’s always a factor. If that’s not enough, try remaking a fold that didn’t come out right—remake it enough times, and that crease starts to remind you of the maps in the car. Balancing geometric precision against the paper’s physical limits, the abstract ideal against the actual reality, is a lesson that applies beyond origami. How about when applying software design principles? Hiding your object’s data is a good idea, but 50 private variables with 50 getters and 50 setters is decidedly not. If your code reads like an illustrated tour of the GoF Design Patterns, you should chew on this.

Origami philosophy aside, it’s relaxing to do, once your fingers are accustomed to it. After folding a few thousand pieces, finding the sweet spot between the geometry and the paper is almost meditative. How well I achieve that balance is my measure of a piece’s quality.

Origami requires a light focus—just enough to let your mental peripheral vision wander. It’s an easy jog for your brain, somewhere bewteen sitting on the couch and running up a mountain. The same way some people need constraints to be creative, my mind needs something tiny to do, for it to really relax. All this sounds very zen to me: beginner’s mind, nothingness, mindfulness…I have to think about this some more.

Juggling!

I started juggling two weeks ago. Ok, started trying to juggle two weeks ago, and started something-almost-like-juggling in the last few days. Maybe I’ll start juggling this week…I can feel myself getting close. At any rate, I’m at the point where it’s not so frustrating, so I’m even more apt to practice (be warned, my cats).

Juggling also requires a light focus. It’s also kind of medidative. Until I drop a ball, which always happens, but up to that point, it’s really pleasant.

In Closing, Back to Zen

There’s something that connects movement and brainpower, I’m sure. I don’t know nearly enough about it to say anything conclusive or probably even convincing, so I’ll just end with some words from ESR, and if that doesn’t convince you, well…

Train in a martial-arts form. The kind of mental discipline required for martial arts seems to be similar in important ways to what hackers do. …The most hackerly martial arts are those which emphasize mental discipline, relaxed awareness, and control, rather than raw strength, athleticism, or physical toughness.

[The emphasis is of course mine.]