Reflections on the boundary between users and programmers

It appears to me that there is, in the minds of many people, a brick wall between "users" and "programmers," a wide boundary between those who "just use computers" and those who create programs by writing code, which is often described as "arcane," "cryptic," "complicated," or some such. I wonder whether this barrier is an essential part of computer nature, or whether maybe it is an accidental consequence of the way that our currently most popular systems are built.

A graph of the amount of skill required to perform tasks of varying sophistication in Windows or on a Macintosh might look something like this.

Typical Graphical User Interface Environment

That huge jump is, of course, the place where we divide the programmers from the users. Users live to the left of that jump, performing tasks which they already have buttons and preferences for. Programmers live to the right of that jump, creating new programs for other people, trying to anticipate the user's every need and validate every piece of data the user provides, doing interface design and interaction design and such.

So far, so what? Well, the problem is, a whole bunch of "normal" users are constantly wishing that they could do things which they don't have buttons for. Brian Harvey describes this very well in his essay Is Programming Obsolete?

If you ask most computer users, they'll tell you that they don't want to program. The word "programming" has come to connote something horribly complicated, beyond the ability of most people. But the people who say they don't want to program are wrong. The proof is that they buy "macro" languages, such as QuickKeys. These are abysmally poor programming languages, in which the notation is based on keystrokes and mouse clicks rather than on the desired program functions, and yet people put up with their awfulness because they are so desperate to program their computers! They don't express their desperation in those terms; they say that they merely want the computer to carry out their tasks, which aren't exactly the ones built into the commercial software. But teaching a computer to carry out a desired task is precisely what "programming" really means.

In The Mythical Man-Month, Fred Brooks argues that writing a program for external distribution is around 9 times more difficult than writing a program for internal use. He argues that "productizing" a program incurs a factor of three, which includes making it work in multiple environments, making it respond sanely to all possible inputs, and documenting it. He also argues that "componentizing" a program incurs another factor of three, which includes making it easy to use as a component in a larger system.

If Brooks is correct, then couldn't we turn it around and argue that it should be nine times easier for someone to create a program that just meets their immediate needs, rather than a programming "product"? How much of that wall between users and programmers comes from an essential difference between the activities of "using" and "programming" and how much comes from all of the extra effort required to create a "product"?

Most of the time, when I need to do something to a text file, I write a Unix one-liner and I'm done. I don't usually worry about handling every text file of a certain kind that might ever be thrown at me. I don't worry about dealing with malformed lines if there aren't any in the file that I need to deal with at this moment. I solve the problem that I have right here and now. Doing this makes the programming so simple and easy that I usually don't even bother to save the program--I know that it'll be easy to recreate if I ever need it again. (Of course, there are plenty of times when I do write scripts which will be reused many times and I do worry about error handling and such, but they aren't my focus right now.)

Unix shell users need to acquire a little bit of skill to start out with, but after that the learning curve is much more gradual. First you learn a few shell commands (ls, cp, mv, rm, mkdir, rmdir, cat/more/less). Then you write simple aliases and shell scripts which are just a list of commands. Maybe you write a script that takes an argument, and later on one that processes a bunch of files with a loop, and the whole time you're learning how to create useful pipelines as you pick up a few of the text processing commands, and before you know it you're doing all sorts of "programmer" stuff. I would guess that the learning curve for a typical Unix shell user would look something like this.

Typical Command Shell Environment (Unix)

Of course, the problem with this model is that it requires some up front investment, and we all know that people hate up front investment. Is it possible to create a system that doesn't have the wall, but which doesn't require any up front investment, something that looks like this?

Ideal Environment

Many years ago, it was popular to imagine a world where people and computers worked together, the people doing the creative work and the computers doing the drudge work, as instructed by the humans. For example, this vision appears in Licklider's writings on Man-Computer Symbiosis (which were influential in the popularizing of timesharing and networking) and books like Man and the Computer by John G. Kemeny (one of the creators of Basic).

What happened to this vision? I don't know if anyone has actually achieved the ideal, but I think there are some environments that are close enough to be interesting.

Emacs, for example, has a fairly gradual learning curve. You can start with simple editing, and then when you want to make some customizations you tweak a few configuration variables and your key mappings. Most people probably stop there, but the incremental effort is fairly small in each step if you move on to some short one or two line functions, longer functions, tweaking existing major and minor modes (first with hooks, then by making and editing copies) and eventually writing your own major and minor modes. At every step along the way, you just have to learn one or two new things. There is no single point where you need to stop and "learn elisp" before you can progress any farther.

In Emacs: The Extensible, Customizable Self-Documenting Display Editor, Richard Stallman says

The programmable editor is an outstanding opportunity to learn to program! A beginner can see the effect of his simple program on the text he is editing; this feedback is fast and in an easily understood form. Educators have found display programming to be very suited for children
experimenting with programming, for just this reason (see LOGO).

Programming editor commands has the additional advantage that a program need not be very large to be tangibly useful in editing. A first project can be very simple. One can thus slide very smoothly from using the editor to edit into learning to program with it.

I suspect that what Stallman describes here, the combination of receiving feedback quickly and being able to get useful results from short programs, is the key creating the ideal environment. Other environments that have shared these qualities to some extent include spreadsheets (though there is a bit of a wall between using formulas and writing macros, it is made easier to climb with "record my actions" type features), HyperCard, Logo, many Basic environments, and Squeak. Technically APL also possesses both of these qualities.

Back when ircII was the client for irc, many people learned to write some fairly sophisticated ircII scripts despite having no particular interest in programming as a subject. The same thing happened with many mud clients like pmf, tinyfugue and tintin.

Quite a few people over the years have learned some programming in order to create areas in programmable muds, like LPMuds and LambdaMOO. First you learn how to create a new room and add some exits, then you learn how to do things like doors with a variable and a two-line function, and so on up a fairly gentle learning curve.

Most of these environments are text based. I don't know if that's just a reflection of my own personal history, or if there's something about the GUI that makes it more difficult to give users an easy to climb learning curve that includes programming power (although Smalltalk and Squeak seem to pull it off pretty well), or if it has something to do with text based environments being older than GUI environments.

Have you seen non-programmers slide into programming-like activity in any other environments? Does this sort of thing ever have a chance at going mainstream?