The very first item in Dan Ingall’s esteemed Smalltalk introduction “Design Principles Behind Smalltalk” is this:
Personal Mastery: If a system is to serve the creative spirit, it must be entirely comprehensible to a single individual.
Most of the other points Dan makes in this essay are well-served by modern Smalltalks, and apart from a few glaring outliers, even by other modern systems that are not remotely connected to Smalltalk, but this one item in particular stands out to me as the most criminally absent.
In order to place it first, I would imagine Dan felt that this item is crucial. He didn’t want it lost in the shuffle in the middle of the essay. And yet, even small, modern Smalltalk implementations contain thousands of classes. That pasty abomination Morphic is by itself a great demonstration of how not to build software: one class with several hundred methods. (I think Smalltalk suffers from not showing all the methods in one file; at least in Java when a class exceeds a couple thousand lines you know there is a problem, but there’s no way to know that about a class in Smalltalk).
Earlier today I was looking at the source code for J and generally thinking about J and how odd it is that I like two languages so radically different that they almost have no similarities at all. In Smalltalk, readability is paramount; in J, readability is something that “will come in time.” In Smalltalk, the goal of the code is to communicate intent; in J, it is simply to process the data. In Smalltalk, one talks about objects and their responsibilities. In J, one talks about arrays and the functions that manipulate them.
As an aside, J is object-oriented, but I have trouble imagining this faculty is heavily used, if for no other reason than to use advanced features of J one must first use J, and there isn’t a lot of that going on either.
In any case, the source code for J is famous for being written in a C that has been heavily contorted to look and act like J itself. For example, drawing a paragraph of code from a random file in the distribution, you see this kind of thing:
I saw this and I thought to myself, damn, that’s totally unreadable, but then upon reflection a couple other things occurred to me.
Users who Program, and Other Yetis
Proposition 1 of “Design Principles Behind Smalltalk” is predicated on the notion that if we make programming easy enough, users will take it up. And yet, as we have seen, users do not and Smalltalk remains obscure, heavily used in a handful of industries but mostly kept alive by a medium-sized group of passionate, die-hard fans. Interestingly, APL and J are known for having a clutch of equally die-hard fans, many of whom know no other languages.:
If APL were a specialist, complex language, it would only attract the “Boy Wonders” of IT, those with A-Grades in everything, whose horizons are limited by bits and bytes.
So it is paradoxical that the great majority of IT people have never really understood APL. Those who have used it successfully have very often not been computer-literate, or have only a slight knowledge … and they have frequently learned APL in isolation. That is to say, the language serves anyone prepared to explore it without prejudice.
This got me thinking. In addition to not knowing non-programmers who are interested in programming in any capacity, I also know of no users who use Unix with proficiency and have no interest in programming. And the crux of Plan 9 was to return Unix to its pure roots, where everything is a file and everything is done in the classic Unix way by stitching together small programs and leaving the heavy lifting to C programs. So it seems interesting that these systems with an emphasis on simplicity—in the name of hypothetical casual users—seem to enter obscurity without having been wildly adopted by any of these casual users.
Programmers Like To Program
It’s widely bemoaned that programmers like writing programs, even when doing so is unnecessary busy work. When Tom Lord discovered distributed version control, we didn’t just use arch, we all had to write our own DVCS, and today we live in a world with six or seven that have essentially the same featuresets. Back in the late 1990s, everybody had to write their own window manager. Now, in the early 2010s it seems like everyone is writing their own “NoSQL” database.
Since programmers like to program and there weren’t really any non-programmer users of Smalltalk, it makes sense that Smalltalk got complex and forgot its roots. This is because a programmer is not all that intimidated by being presented with a huge system of astonishing complexity if we can just dive in and use it. If you have to understand everything, you’ll never write any code. So keeping things comprehensible never occurred to anyone; we were too busy writing abstractions to keep things simple for us!
The Questionable Importance of Readability
Let’s stick with J and Smalltalk even though Perl and Python would probably be better examples. Ignoring the rounding errors, say J and Smalltalk have similarly-sized user communities. Obviously there must exist a category of people for whom the initial readability is not a key concern. Indeed, looking at Haskell and Ruby web frameworks, I find that even if you have a system of aesthetics based on simplicity, your advanced users will bend them to increase power. This is why non-Yesod users balk at Yesod’s use of Template Haskell. The fact that Template Haskell is well-understood and well-supported doesn’t make it aesthetically pleasing. Snoyman can put all the work he wants into making it simple and easy and pleasant, but until Template Haskell is core-curriculum stuff it’s going to be off-putting to new users.
So looking at the J source code, it’s easy for me to hold my nose and say, that’s totally unreadable garbage; how can that be maintained? But at the same time, it’s not my place to maintain it. Imagine if it were written in the most clean, beautiful C code possible. I might be able to dupe myself into thinking I could maintain it, but it would be a lie! Is it so bad that complex projects like J have complex code? If it were a complex Java program instead, I’d still need substantial time to learn it before I would stand a chance at modifying it. Making it J-like means I am required to understand J to change the source code. Wouldn’t I have to understand J to change it anyway?
Does readability give us a false sense of maintainability?
A Thought about Power
If I were to tell you you should learn J because it is powerful, mind-bending, and high-performance, most programmers would say, yeah, maybe. But what if instead I told you, I have this language which is extremely terse, in which the output in some ways resembles and in some ways completely contradicts the input, but which is of high utility in a particular class of problems. At first it sounds like I’m trying to trick you into liking J, but I’m not—I’m describing regular expressions.
Think about it. Without batting an eye, I can write a regular expression to (say) parse an auth log line from this computer:
Nov 16 12:46:07 7gf sshd[20009]: Invalid user oracle from 200.216.162.113
A regex might look something like this:
^... \d\d \d\d:\d\d:\d\d 7gf sshd\[\d+\]: Invalid user ([^ ]+) from (.*)$
If you use regular expressions, it’s probably quite clear to you what that does, despite:
- control sequences mixed in with literal text
- the same character doing radically different things depending on escaping and context
What’s the difference between ^ at the beginning and ^ in brackets? What’s the difference between bracket and backslash bracket? Why do some things have to be escaped and not others? What’s the difference between + and *? These are all hurdles we have to get over to learn regular expressions, but once we know them, it’s not hard to remember how they work.
On top of all that, regular expressions do nothing to aid readability: they look nothing like English, nor do they look like instructions for how to match text. Despite this fact, nearly all of we programmers learned them, either on the job, in school or on the side, and we all use them with great frequency. We defend them, to the point that Larry Wall is possibly the first person in 30 or 40 years to suggest we could actually have a better syntax with which to do this. I know of just one program that matches text in a radically different fashion (I’m talking about txr) and even it lets you fall back on them.
Let’s consider a Smalltalk-like API for defining that same regular expression. Would it look like this?
Does that seem like an improvement? Maybe if you don’t know regular expressions. But you do. This is the crux of the APL/J argument. Where’s the flaw?
Afternote
I just ran into this document about the proceedings of the APL 2010 conference discussing OOP. In it you see some real gems, like:
I tried to learn SmallTalk once, but it was not a success.
and:
J does not have instances [in the OOP sense] properly speaking. They are locales and locales are not assignable (like in APL) so you cannot have arrays of instances or classes (which are also locales).
Interestingly, there seems to be some truck with Haskell users. I have seen mention of Haskell both on this page and on the J Programming Forum, but not really of other FP languages.
Tentative Conclusions
- It is more honest to see programming paradigms, development tools and language diversity as an effect caused by programmers need to program, rather than technical or market needs.
- Technical decisions promoted in the name of “programming users” are lies.
- Readability as a concept may be a lie.
Addendum: Lack of Visi.ion
I just saw this Visi.io project linked from Reddit. I want to say it’s admirable what this guy is doing, and he has quite a pile of code already written. But even his version of history is quite damning: HyperCard is dead and Excel is rarely understood. Taken with the preponderence of other evidence, this project will likely go the same way as all the other well-intentioned user-liberation schemes. We’ve been trying to make programming accessible to non-programmers for the last fifty years. Has it worked? Are the missing ingredients really type inference, JSON and monoids? It wasn’t that long ago that HTML was invented specifically to be readable to humans and machines, and here he is dismissing it because it sucks compared to native apps. Well, spreadsheets also suck compared to native apps, and so did Hypercard stacks. Is the problem really that those tools lacked a sufficiently expressive language, and not that users would rather have higher-quality tools that do not require programming?