Frameworks drive language adoption
October 24, 2011
Hypothesis: Frameworks are the drivers of adoption of new programming languages. Without frameworks the inertia of the existing languages are too great to overcome.
Others have explored this hypothesis from different angles, sometimes with actual data!
It is very difficult for a new language to overcome the cost of switching from an existing language. PHP never had much to offer in terms of great language features over Perl, which it was originally created from. But it was designed to be able to quickly create web pages as the web was taking off, so it was able to fill that specific niche by offering something more productive than alternatives. PHP was like a framework - it excelled at a specific problem domain. And there are incredibly popular PHP frameworks like WordPress that helped drive and assure its continued use today.
Ruby started with very small scripting use, argued by some as better than Perl or Python for that domain. But then Ruby offered Ruby on Rails, a much more productive way of doing web development than what existed at the time, and it began an exponential growth. And now Ruby is also getting increased usage in the system administration domain because it has two good frameworks for that: Puppet and Chef.
Scala is on the rise. One of the biggest reasons is because Java has now become a platform itself, and Scala can be used as a better Java. Scala offers many great language features that make for a much more productive programming language than Java. But as David Pollak points out, that alone is not enough to overcome the effort required to switch languages. Pollak makes the case that Scala needs to focus on specific industries and problem domains where it offers enormous benefits over Java.
A language being good on its own merits may not be enough to drive new adoption anymore unless it is highly compatible with an existing popular language. And in the case of Scala, that compatibility does not seem to be enough for explosive growth. I think Pollak is correct that Scala needs more focus, but I think it could focus in many more areas than he thinks, so long as it first develops great frameworks in those areas. The most impressive exponential growth of a platform independent language in recent times has been from Ruby, and it was driven almost entirely by the Ruby on Rails framework.
I am a big fan of Paul Graham's writings and doings. Graham writes in relation to startups. He has written about the importance of programming language, and he has the experience to back it up. Graham wrote the first web application using a Lisp programming language, and it was very successful for him and all that were involved. Graham attributes much of his success to the choice of Lisp, and asserts that it is a superior language. However, Graham was creating the first web application. If he had to compete against someone using a modern web framework today, it wouldn't be the same level playing field it was when he created his first startup. He would also be at a greater disadvantage in terms of available and easy to install libraries relative to a modern popular programming language user.
This entire line of thought follows from a broader principle: the key to creating software quickly is to re-use good existing code. Existing languages remain popular in large part because they build up a large collection of re-usable libraries, and a system for conveniently installing and using them. However, libraries alone are not enough any more. An even greater productivity boost are collections of libraries that work extremely well together: frameworks. A framework takes care of as much code common to the problem domain as possible and provides developers with re-usable patterns to develop their custom software. Popular languages build up frameworks also. But new languages can compete at the framework level, in part because the cost of switching frameworks can be very high. Switching to a new language is now a piece of the equation that also includes the benefits of the new framework.
If frameworks are the key, why aren't there a ton of great frameworks in every language? One reason is that not every problem domain lends itself as well to frameworks - hard constraints make for frameworks - server-side web application development most often involves using web frameworks because there will always be the constraints of HTTP.
But I suspect the biggest reason for a lack of great frameworks is that creating a great framework is hard. Hard in a design sense means that most frameworks won't be designed well. But the natural evolution of the problem space would still lead to a lot of well designed frameworks. The real difficulty in creating a framework comes from the amount of effort required for a long duration. A good programmer can write a useful library in a day to satisfy a need. Good frameworks do come from companies that invest programmer time in creating frameworks. However, many open source frameworks start with the unpaid efforts of a single programmer. That pace is very limited - the key to making something good is for the programmer to stick with it. Just the initial framework creation can easily consume a programmer's free-time for a month. Making a really good framework is a process of continual creation and adjustment from feedback of users - a process that can easily take a year of dedicated effort. It is difficult for a programmer to directly recoup the cost to their time that they have put into an open source framework. If a programmer is creating a framework in an unpopular language that they don't get to use in their day job, that is even more the case. Time and money are working against the framework creator, particularly in the case of open source and unpopular languages. Perhaps the most important aspect of a framework is the community of users that grow around it. I tend to think this of this as an organic process. However, supporting a new community of users adds on to the skillset and work required to create a great framework before the threshold of community contributions makes up for that effort. In the long-term, frameworks can gain a lot of advantage by making it easy for the community to contribute re-usable code that is not officially part of the framework (plugins).
Lets put on our Haskell hat!
The Haskell programming language has an amazing set of technical merits that are unmatched by other programming languages. The type system lets one declaratively model data and catches an amazing number of bugs. The GHC runtime has incredible concurrency and parallelism capabilities.
But technical merits do not translate directly to writing software quickly - libraries are required. Haskell is in a great position to share libraries - its installation infrastructure is starting to mature. The strong-typing means it is much easier to use a library correctly, and there is much more certainty that the code works.
Certainly Haskell is lacking in specific libraries, but there are now thousands of libraries available to help get you started on your task. Some of these libraries (Parsec, QuickCheck) are incredibly good in comparison to what popular languages normally offer. However, parsing and testing are usually just a portion of a larger problem domain. The largest effect these libraries are having with those outside the Haskell community is causing them to implement these libraries in the language they are using. Many Haskell libraries are nowhere near as remarkable, with many not of great quality or are at least lacking in documentation. But even a large set of good libraries is not enough to drive adoption. Haskell needs frameworks that make developers really productive in established problem domains.
Yesod is such a framework for web development. Warp, the preferred web server for Yesod (and now other frameworks), leverages the GHC runtime for concurrency performance not possible in other languages. Yesod uses the type system to catch errors that can't be caught in popular programming languages. For instance, all application URLs are known to be valid at compile time. There are other web frameworks for Haskell with their own merit, none of them have gone as far in type safety, levels of abastraction, and ease of use. Yesod is bringing new users into Haskell, or at least helping new users stick with Haskell by making them productive at web development.
A running joke of Haskell is to “Avoid Success at All Costs!”. If frameworks are necessary for success, then some members of the Haskell community seemed to be taking things too seriously. Particularly when Yesod first started, some in the community seemed to be criticizing Yesod essentially for being a framework (saying things like "Yesod is its own world"). On the one hand this is true to a certain extent of all frameworks, but on the other hand Yesod is highly modular and new web frameworks have already been created in Haskell largely by re-using its libraries. Other criticisms were made, but none of those offering negative opinions about Yesod had actually tried using it. I think this peculiar behavior reflects cultural issues with the Haskell community. I think Haskell is partly a reaction against the normal inadequate ways of programming. One of the normal ways of programming is to use frustrating frameworks, so anything ambitiously venturing into framework territory risks guilt by association. One of the reasons Haskell is great is because the community is amazingly knowledgeable. However, this amazingly smart group of people tends to know very little about web programming. Haskellers have not been doing much web development because there haven't been good libraries to support their effort. This lack of experience has not always stopped some from being opinionated. However, now that the library and framework situation is turning around in Haskell I expect this situation to change. Haskellers are learning web development and web developers are learning Haskell.
A good framework becomes a language dialect of its own, and pushes the host programming language to its limits. For Haskell to be successful, we need the community to be supportive of these kinds of efforts. But we also need more great frameworks! In every domain, Haskell has a chance to beat out existing popular frameworks because of its strong typing, efficient code, powerful runtime, and support for creating declarative code. Haskell is already very good at embedded domain specific languages, but you can also crank things up a notch and use Template Haskell and Quasi Quoting when the regular language features are not enough.
There are many great ways to help out the Haskell community. In particular, those that work on the compiler, core infrastructure, and quality libraries are all heroes. They help ensure Haskell stays a technically amazing and easy to use programming language. Haskell needs to keep this momentum going. But if the goal is to drive adoption, building and supporting great frameworks should be one of the greatest concerns.