Thursday, April 2, 2009

Perl 6: Why So Long?

Over the years, many people have asked me when Perl 6 is going to happen. Now, I've only been peripherally involved, and I don't think I can speak for any of the current developers, but here's my take. First off, comparing development times of Perl 6 to, say, Python 3 (ne Python 3000) is entirely spurious. Python and Perl 5 relate to Perl 6 in roughly the way that classic Lisp 1.5 relates to Common Lisp. That is to say, Python and Python 3 are essentially the same language and exist at essentially the same order of complexity. In fact, they exist at about the same order of complexity as Perl 5, Ruby, PHP and many other high level languages.

Perl 6, meanwhile, is a massively ambitious effort that aims to meld the best features of nearly every style of language design into a single language. This means that the language must be able to represent programs as data (in order to implement true macros); it must provide a meta-object protocol (in order to support Ruby-style mixins and Smalltalk-style traits); it must implement function overloading (C++-style calling) and multi-method dispatch (CLOS-style methods) while also providing interface contracts (like Java) and generics (C++ at the low-level, Haskell at the high level); and it should provide native threading and parallel execution (ala Fortress). Now, take a language that has all of these features, and yet still manages to be familiar to a C-derived language programmer and also allow programs full access to the definition of the language at run-time in order to support multiple Common Lisp style domain-specific languages within a single program. To my knowledge no other language has ever taken on all of these goals and managed to get very far.

OK, so that's the excuse. What's the reality? Hmm... I'd say that Perl 6 is probably closing in on its .0 release within the next 2-3 years. Realistically, it probably would have taken a team of dedicated programmers who knew the finer points of self-hosted language design a couple of years to accomplish the Perl 6 design and implementation, given a clear idea of what they wanted from the onset. Perhaps a bit longer, but not much. As it was, many of the Perl developers have other things to do and Larry had some time that he wasn't able to do much for personal reasons in the middle of the whole thing. All of that combined with the fact that the Perl community was never entirely sure what the end-goal would be, and much of the design process has been a journey of discovery and... well, 10-12 years isn't a surprising number.

These days, there's an STD that describes the grammar and a fairly solid set of specs. That's a big improvement. There's also a virtual machine for running the language that's reached 1.0 status and deployed what can arguably be called the most advanced compiler-writing toolkit ever written. Again, a huge leg up.

That in turn has enabled the creation of a new prototype of the language that uses said virtual machine, taking a large portion of the burden of implementation off of the Perl 6 team, and the result has been a flood of development on the prototype, which has overtaken that of previous efforts in only a little over a year.

In the end, if you thought that Perl 6 was going to be Perl 5 with a few changes, you're in for a shock. If you thought that Perl 6 was basically a dead project because it has taken almost 10 years so far, then you're not aware of the history of such language design (the Common Lisp design and implementation process took nearly as long with DARPA and many commercial Lisp organizations backing it).

As we enter the final stretch and Perl 6 begins to become a platform upon which real work can be done, I just wanted to remind everyone that, while simpler languages (and I mean that in a non-pejorative way) rise and fall, there are some things that are worth waiting for.