Recent update: database migrations and more

July 27, 2010 12:00 am

Michael Snoyman

I usually blog much less about Yesod when the changes going on aren't near finalization. But I wanted to get out at least one post letting people know what's in the works. Just remember: we're early in the cycle for the next release, so a lot of these features are really wild guesses.

Database migrations

Thanks to some motivation from Greg Weber, I've finally gotten around to working on database migrations. The code is currently only functional for PostgreSQL, but it's already being heavily battle tested: my main project relies very heavily on this feature to deal with the ever-changing specs that are thrown at me.

What I think is a relatively unique feature in persistent is database introspection. The migration code looks at your table structure, including default values, NULL/NOT NULL and type, and compares that to your entity definitions. In addition, some pretty cool features:

• There's a separation between "find the differences" and "apply changes." This allows you do store the migration information in a separate file, or use some external tool (such as dbmigrations) to manage the actual migrations.
• There are two different functions for applying the changes: the "safe" version never drops columns, whereas the unsafe one does. If you use the safe version (I do) you'll have to manually drop columns.

migrate will replace the initialize function in persistent 0.2. I still have to implement the code for Sqlite. Unfortunately, sqlite does not provide the same level of introspection and alteration abilities that postgresql does. Most likely, I will need to parse the CREATE TABLE statements and copy table contents for any migration, but it should be manageable. And when it comes to schema-less databases like Redis, migrations shouldn't be necessary.

Improved select function

Rehno Lindeque pointed out that we were missing two important features in persistent: getting the number of records and limiting/offsetting a result set. For the former feature, I was hoping to write something more generic, allowing any aggregate functions. Unfortunately, I haven't come up with a type-safe approach to this yet, so we'll just have a count function for the moment. If anyone has a brilliant insight, let me know.

The select function is now a bit more complicated. In addition to a list of filters and a list of orders, it now also takes an offset and limit. If the limit is 0, then no limit is imposed.

Finally, select no longer returns a list of records. For this usage, use the selectList function. select now has an enumerator interface; you provide it with a seed value and an iteratee. I'm debating whether to use the iteratee package for this interface; input is welcome.

Foreign keys and deleteCascade

For postgresql, we now support foreign key references. persistent figures these out in one of two ways:

• If you provide a references=tblName attribute, then we create such a reference to that table's primary key. If you provide a noreference attribute, then no reference is created.
• If neither of the above attributes exist, and the data type for a field is SomeNameId, than a reference is created to the primary key of tblsomename.

This makes it exceedingly difficult to actually delete data from a database; having any references to data you try deleting will result in an exception. So I've also added a DeleteCascade typeclass and a TH helper function to automatically create "deep deletes".

Some forms cleanup

Back to Greg Weber, we've been working together on cleaning up the Yesod.Form module a bit. Here are some notable changes:

• You can supply id=someId and name=someName attributes in the entity definition and mkToForm will set HTML attributes accordingly. This can be useful for creating separate CSS files.
• We're making the locations of Javascript and CSS files customizable. Now there is a YesodJquery typeclass that has functions you can override. This makes it much easier to use a local copy of the jQuery code, or switch a jQuery UI theme. This change precipitated some major reworkings of the ToForm typeclass, in particular it is now a MPTC.
• There are no longer special datatypes for javascript-enabled fields. For example, in order to create a jQuery UI date picker field, you would now specify in your entity definition day Day toFormField=YesodJquery.jqueryDayField. The YesodJquery prefix specifies that this function requires a YesodJquery instance.

nginx

I've added an nginx section to the deploying page of the Yesod docs site.

Questions for the community

• What kind of changes would people like to the layout of the yesod docs site?
• Do people prefer screencasts or text tutorials?
• Do you have any Yesod projects that are not listed in the powered by Yesod section?

One final note: I'm in the process of some big changes over here (my wife's expecting any day now, and we're probably moving apartments in the next few weeks), and I need to make sure I don't slip too far behind in my day job. Don't be surprised if I suddenly stop answering my email for a few days; it's probably because I'm either in the hospital with a new baby or between apartments without internet access.