The Great Yesod Reorganization of 2011
July 24, 2011
In the beginning, Yesod was a single repository written by a single developer (me). It included all the bells and whistles internally, from authentication to forms and static file serving. And all was well and simple.
Except this didn't scale very well. Making breaking changes in a single component meant bumping version numbers everywhere. Someone wanting to implement a small site with few features still needed to pull in on the dependencies of the greater Yesod. And as features continued to be added, this overhead continued to grow.
The solution was to split Yesod up into many smaller packages: yesod-core, yesod-auth, yesod-static, etc. And this was a very good solution (sans some Cabal annoyances).
But of course, no good solution is without its issues. In this case, the sheer number of repositories involved made it prohibitively difficult for people to test out beta code. Branching for a beta version also meant branching across 8 different repositories. Often, this meant that some packages weren't branched, and this made it even more confusing to build straight from the repos.
It also became very difficult to follow Yesod development. People tracking the yesod repo itself suddenly saw a significant drop in activity, since most development now occurs on yesod-core. And combined with my non-Yesod packages (yes, I do write code besides Yesod), my personal Github account was at about 80 repositories, making it very difficult to find Yesod code.
The solution is actually very simple. There is now a yesodweb organization on Github. Instead of an individual repository for each package, it has "megarepositories" grouping together common packages. The wai repo contains WAI, Warp, wai-handler-fastcgi, wai-test and others. Persistent contains the core persistent code, as well as all the backends and TH wrappers. (Actually, this was already the case, a fluke of our development history.) And yesod contains a bunch of yesod-* packages, plus the wrapping yesod package.
With the new setup, it should be significantly simpler for users to try out beta code: clone a few repos and run the convenient "install-all.sh" script. This script also leverages the new "cabal test" framework to automatically run unit tests. The goal is that by having more eyes on the code, we can do a better job. And by having more people testing beta release before they arrive an Hackage, we can catch bugs before they go into the wild, and particularly eliminate bad API decisions.
Well, why not? Actually, it's a very good question. We are quickly approaching the 0.9 release of Yesod. We're waiting on the release of a few libraries that we're very excited to be using (I'll leave that as a surprise for later), and hopefully within the next week or two we'll be ready to go live. 0.9 is a very important release: it's our last milestone before the 1.0 release.
Over this last development iteration, you'll be interested to know that there were no breaking changes in yesod-core. I consider this a huge milestone: the base Yesod API has stabilized to such an extent that we didn't feel the need to break people's code this time. There were still changes in the underlying libraries: removal of polymoprhic Hamlet, a greatly improved forms API, drastically cleaned up Persistent architecture, etc. But the core of our framework is stabilizing.
Now is when we're getting very serious about taking Yesod to the next level. We have all the raw features we want. Most of them have been there for over a year. We've had a number of iterations to clean things up. We've found high quality libraries we can rely on: text, blaze-builder, enumerator.
Our focus at this point is to refine Yesod. We're focusing on things like more helpful logging, better deployment strategies, increased performance. In the next two weeks, we'll be tying off a very high-quality release, and beginning on the last touch-ups before our 1.0 release.
How you can help
If you've made it this far, odds are you care about Yesod. So if you'd like to help, here are some ideas:
- We love the positive feedback, but the constructive criticism is what helps us grow. Keep it coming!
- Participate in the discussion on our new mailing list.
- Download the code now and test it with your current codebase.
- Keep up the comments on the book.