Mixing Metaphors

Today at work, while dealing with Hibernate, we noticed a queer problem that comes up when using one-to-many mappings. Like everything else in Hibernate, the system is apparently self-consistent, until it isn't, and I usually find I can get about 90% through an explanation of why Hibernate requires something, but when I get to the last 10% I realize there's a missing step in my understanding, or an unsound leap of reasoning in the software.

At any rate, what we're trying to do is put ON DELETE CASCADE on all our foreign keys, but Hibernate only supports on-delete="cascade" on inverse mapped one-to-manys. This turns out to suck because in order to have this, your relationship must be bi-directional, and in order to be bi-directional your relationship must be between entities that refer to each other.

This reveals a mistaken world view in Hibernate, which is that you can somehow achieve all three of these goals simultaneously:

  1. You can support legacy databases
  2. You do not require changes to the Java code for persistence
  3. Every column you care about maps to a Java bean property or field.

In this case, the problem is revealed by the fact that Hibernate's on-delete="cascade" feature depends on inverse mapping because otherwise Hibernate manages the collection from the parent's side. But databases do not have parents and children nor do they have collections. A one-to-many relationship in a database is just a foreign key shared by many rows in one table, pointing to a primary key in another table with as few as one row. Looking at it as a collection of any kind is a mistake, but it's a mistake foisted upon you by Java.

Hibernate layers certain defined meanings onto several ways of using a one-to-many relationship by way of <list>, <set>, <bag> and so on. In reality, relationships between tables aren't so well-defined, but Hibernate needs this information to work its magic for some reason. At any rate, Hibernate achieves support for lists by, in general, first inserting the list items and updating them with their sort order and then applying their foreign keys, like so:

INSERT INTO parents (id, ...) VALUES (?, ...);

INSERT INTO children (id, ...) VALUES (?, ...);
INSERT INTO children (id, ...) VALUES (?, ...);

UPDATE children SET parent_id = ? WHERE id = ?;
UPDATE children SET parent_id = ? WHERE id = ?;

This works well enough in databases with poor relational integrity, but in a real database such as Oracle or PostgreSQL, these columns will often not be nullable, so you'll have a constraint violation on the first INSERT into the child table. Well, as it turns out there's a way to get Hibernate to do this the intuitive way. Change this:

 <key>
   <column name="parent_id"/>
 </key>

Into this:

<key not-null="true" update="false">
  <column not-null="true" name="parent_id"/>
</key>

This is confusing for one of the most annoying of reasons: there's an update attribute for many-to-one which means something completely different than it does on <key>, and the not-null attribute has similar but subtly different meanings on <key> and <column>, a fact exacerbated by the fact that you can often merge the two and further complicated by the fact that you almost always want to index this column, which you can't do by merging the elements because <key> does not take an index attribute. (Also note that there's an <index> element that goes inside a <one-to-many> as well. This chart summarizes the differences:

<column><key><many-to-one>
not-null="true"Add NOT NULL to column definitionMandates this primary key at insertion time--
update="false"Mark this foreign key column as non-updateable--Half of marking this relationship as "virtually inverse."

Now you may ask, what's this virtual inverse there? Well, as it happens, Hibernate doesn't support inverse="true" on many-to-one relationships, so instead you say <many-to-one update="false" insert="false" ...>, which is apparently considered the opposite of <one-to-many inverse="true"...>, because by disabling UPDATE and INSERT statements, you've effectively nullified the management problem for Hibernate (but it still counts for bi-directional mapping purposes).

Note that in that case, update="false" means something very different from what it means under <key>. The combination of update="false" and not-null="true" in <key> tells Hibernate that the parent key is needed at insertion time and that it cannot be fixed later with an UPDATE.

Another thing to note is that once you have <key> and <column> you can set not-null= to different values, yielding this chart of possibilities.

<key><column>Effect
not-null="true"not-null="true"Works fine.
not-null="false"not-null="true"Broken: Hibernate thinks it doesn't have to insert the row with the foreign key column, but the RDBMS is requiring it for data validity because Hibernate added the constraint.
not-null="true"not-null="false"Hibernate does an insert and no update, but doesn't make the database add a constraint to check for this situation.

Back to Philosophy: Mixed Metaphors

Zed Shaw wrote another beautiful manifesto this last week, calling for us to embrace Programming, Motherfucker. I tend to agree with it, insofar as methodology is never a silver bullet, there are no silver bullets, and so forth, but I can't accept it if its promulgated as a strain of anti-intellectualism. None of Hibernate's problems are due to a lack of programming. They're all due to a lack of philosophy.

Is Hibernate a framework for persisting an arbitrary object model into an arbitrary database? No, because if it were, Hibernate would not have opinions about the kinds of structures in relational databases they support. There would never be a dismissive post from Gavin King or one of his cronies, explaining to the poor RDBMS exile that Hibernate doesn't support this capability, fuck you for asking, and what kind of moron are you anyway? Looking through the hype, when you search for help on any particular topic with Hibernate, you will invariably find three or four borderline-relevant forum postings in which the diligent user is dismissed for doing something non-standard in their database and told to fix it or fix the object model. Hibernate is most certainly not a framework for supporting legacy databases.

At the same time, Hibernate is not particularly a framework for generating databases based on an object model. For one thing, you're required to specify, either in annotations or in XML, the precise structure of the database to which you want to connect. Hibernate has tools for generating the database from there, but they come with a strong warning to the effect of, "these are for development, not migrating production" and a pointer to Liquibase.

Hibernate pushes itself as being unintrusive into your domain model, but the inability to support the database feature of cascading deletes in the absense of bi-directional links in the model is a prime example of a situation where these two things that are supposed to be separate and invisible to each other are actually intertwined in a very complex way. It's more fodder for the ORM restatement of Jamie Zawinski's celebrated regular expression axiom: you see an object model and a relational database and say to yourself, "Oh, I'll use Hibernate." Now you have three problems, and each one of them is literally a world unto itself.

Not to cut Hibernate any slack, but really, how can you support all of these ideas at the same time? How can you abstract the structure of the database, the structure of the object model, database querying, and the database engine itself at the same time without making gross assumptions? I'm not sure you can, but doesn't most software live in this sticky realm between the hard and fast rules?

Doesn't that explain some of the oddities though? If Hibernate were mainly about mapping objects to legacy databases, wouldn't it mainly use database terminology? If it were mainly about mapping legacy objects to self-created databases, wouldn't it mainly use Java terminology? Instead the terminology is a weird hodge-podge, and you wind up with lots of cases like update= above, where the configuration isn't really using jargon from either domain or being particularly abstract or declarative. What it's doing is giving you control over some implementation detail, some piece of minutia in Hibernate's process. This, like all situations in application design where you rely on configuration or preferences too much, is an indication that the author didn't understand the user. And in this case, the reason is just that Hibernate sits at a particular intersection of the three variables above and each developer has their own slant, their own idea of what "abstract" would be, and this just turned out to be the only wording the devs could all agree on.

Another symptom of this is the "virtually inverse" debacle, where it takes two properties to achieve one effect. This one's made even funnier by the fact that Hibernate requires that both the insert and update properties have the same value on a <many-to-many>. So you have to set two settings to the same value whose names bear no resemblence to the effect you're trying to achieve, and better yet, Hibernate statically determines whether you've broken the rule and will reject your configuration!

I'd like to nail Hibernate for being shitty at mapping complex object models and legacy databases both, but the truth is that Hibernate does an acceptable, if not great, job at both, provided you invest enough resources into it. I just happen to believe that the amount of resources you need to invest far exceed the benefit, but not just for philosophical reasons. I believe it also for practical reasons:

  1. You never need to support a legacy database and a database generated by Hibernate at the same time. (If you think you do, what you actually have is a legacy database that you've allowed Hibernate to continue making changes to, which is not a great idea).

  2. If you do need to support a legacy database, you don't also need RDBMS portability. (Why would you need to be able to generate the legacy database's schema when porting to a new RDBMS?)

  3. Writing your own home-brew data storage wrapper is time consuming, but the profile of the work is a medium-sized amount up-front and then a small amount spread over the life of the project. Using Hibernate is more like a labor time-bomb: you get off the ground running, but eventually a day comes when you have to master Hibernate and fix your mapping. On that day, you are forced into spending as much or more time learning this complex system.

  4. Hibernate's mapping is brittle and depends on precise cooperation with the object model. For example, we had log messages being created and discarded in our object model that were causing Hibernate to load lists during object instantiation. It turns out, doing this can cause Hibernate to load the collections, but because it happens during another load, Hibernate ignores its configuration settings and loads them one-by-one instead of using outer joins or subselects per the configuration. In other words, we were having Hibernate performance problems due to unsuspicious looking log statements that weren't even printed.

Problems like #4 are easy enough to avoid if you know a little bit about how Hibernate works, but the whole point of Hibernate is that my boss doesn't have to learn anything about it, it will just handle the database and he won't have to mess up his data model. But this can't really be true, so you wind up having to learn "the right way" to write your object model, which amounts to breaking a lot of your preconceived notions.

Missing Context

Every bad decision Hibernate made was to solve some problem. It's just that in many cases the solutions that came out of facing those problems were constructed in a different context.

Here's a (haha) simpler example: JSF and Facelets. A Facelet file is an XML format that defines a component tree, in other words, a tree of UIComponents, which integrate into the JSF framework for processing web requests. It's a web framework, in short. A facelet looks a bit like this:

<h1>Facelet Example</h1>
<p>Here's an example of an input control:</p>
<h:inputText value="#{object.name}"/>

Now, what JSF does for you is build the UIComponent tree, and then whenever the form on this page is submitted, it makes sure that the value in the inputText component there gets propagated back to the name property on the object managed bean. In practice, it means that JSF is going to find an object, probably in your session but also possibly in the request or globally, and then it's going to use JavaBean-style reflection to get a setName(String name) method and call it with whatever text wound up in the inputText tag. It's also going to populate the field with the result of calling object.getName().

Seems pretty cool and flexible, right? And not too far off from Smalltalk, where you often see code that looks like this:

builder newTreeFor: self list: #elements selected: #selectedElement
changeSelected: #selectedElement:.

This tells some builder instance of UITheme to generate whatever the current theme's implementation of a Tree component would be, and tells it to get its data, it should (essentially) invoke self elements to get the list of elements, self selectedElement to get the currently selected element, and self selectedElement:anElement to set the currently selected element, of course self being a pointer not to the builder but to the containing class. That's the whole interface for this particular UI widget: three methods. So to use a tree widget in Polymorph, one doesn't even need to define methods with certain names or implement an interface to integrate with the widget. You can just tell the widget some callback methods.

So you can see JSF inherits some of this philosophy. Rather than making you build a component hierarchy in statically-typed Java code, you can just assemble the facelets in a facelet file and avoid recompilation. You can give the code to your designer, they can move stuff around, proof it in their own browser and give it back to you when it looks good. Sounds pretty fun, right?

The problem here is, again, philosophy: interpreting a string or symbol at runtime to find an object and invoke a method on it is extremely useful in Smalltalk, but if anything goes wrong, you're in the debugger already, you can fix the code and hit "Proceed" without stopping or restarting anything. In Java, if the facelet's calling code I haven't yet written, there's no compiler error, but to fix it will require recompiling. Deploying a web app necessitates a full recompile/redeploy cycle, which takes about a minute. By introducing late binding into Java with JSF at the view level, we're not adding a helpful bit of dynamism to stark statically-checked pain, we're driving a wedge between the view and the model/controller completely. We're nullifying the gain of static type checking without nullifying the cost. In other words, it would have been better to pay the price of static checking all the way through and not be able to compile an app with bad facelets views, or better to just jump ship altogether and use Smalltalk or some other dynamic language with a dynamic framework on top.

What's interesting to me about this is that, in a sense, the solution is both. There's no excluded middle here. Sometimes dynamic is better, other times static, but it would be easy to couch an argument for one or the other in logical-looking terms that would seem to "prove" that the other one is utter shit. And that's why I think perhaps philosophy ought to be looked at a little more closely in programming. It seems to me there's going to be room for Smalltalk and Haskell for a long time to come. If such wildly different things can exist simultaneously and provide usefulness to, in some cases, the same people solving the same kinds of problems, what use are the philosophical underpinnings?