Hibernate, Get Out of My POJO!
June 21, 2006
If you do a Google search on Hibernate, you are bound to find a rather large bag of posts on the infamous lazy loading exception that Hibernate produces as part of its lazy fetching strategy. The output of this exception is typically in the form:
org.hibernate.LazyInitializationException: failed to lazily initialize a collection - no session or session was closed
A majority of the time, the error is on the part of the programmer, who did not take the time to understand the Hibernate API and attempted to perform an illegal operation given the resources that were available to Hibernate at the time of the call. Most of the time, but not all of the time.
In fact, my colleagues and I have come across a number of instances in our current project when we need to purge all Hibernate-specific references from our POJO. In this particular case, our goal is to get back to the native collections and object references so that client code can walk the object, processing the current set of data that it offers. We don't want to risk the chance of bumping into this nagging exception when we no longer have a need for its services.
Before I continue to explain our use case, I want to quickly go over some background that will help to clarify our problem.
In order to provide a clean POJO programming model, Hibernate hides itself inside of your POJO by using either its own implementation of the JDK collections or by using a CGLIB proxy to surround an object reference, depending on the type of association it is managing. Since object graphs can be quite large, and in some cases infinite, it is mandatory to draw the line somewhere when loading an object and claim these associations as lazy. This deferral means that at some later point, it may be necessary to fetch the associated objects from the database when this line is crossed.
It would be impossible for a pure POJO to instrument this functionality since java relationships do not talk the language of database connections. Thus, what Hibernate does for you is to "enhance" your objects by using "persistent" collections and proxies to make them aware of the database, without tying them in any way to the Hibernate API (the object instances still implement the same interfaces). Of course, if one of these lazy lines is crossed (and was not previously initialized), it is mandatory that a Hibernate session be open or else that injected code is going to fail when it attempts to use the session it references. These programming errors are actually quite easy to solve by either redesigning the data access layer or settling for the OpenSessionInView solution.
Our use case is a bit different. Once we retrieve an object using the Hibernate API and initialize any lazy collections needed, we then want to be able to detach this object graph from Hibernate's management. To our excitement, Hibernate does offer a method in its API to detach an object, but the devil is in the details. The method that Hibernate offers merely informs the hidden code in the POJO that it should no longer consider itself a part of the current session, and thus allows the object to be reconnected with either the same or perhaps a different Hibernate session. However, the hidden code is still present all the while. Attempting to access uninitialized lazy collections once detached will still throw up the lazy initialization exception. This functionality is not quite what we want. In short, we need to completely cleanse Hibernate from our POJO so that we are left with an object that has absolutely no Hibernate signature left in it.
Let's look at why we need this functionality. In our application, we need to be able to pull data out of the database using Hibernate and then export that object's data so that it can be shipped off to another system. The encoded format we use is XML, so we choose to use a tool such as XMLEncoder or XStream. The trouble was, every time we attempted to use either of these tools, we either received lazy loading exceptions or ended up with a bunch of Hibernate-specific class names in the output (such as PersistentSet). Due to the complexity of our graph, and considering all the circular references, simply cloning was not enough since each object held nested references which themselves were Hibernate-specific implementations of the interfaces.
I am relieved to say that a solution has arrived! Finally, after many, many nights of banging my head against the screen, I experienced a breakthrough that would lead to the solution to this problem. I recognized that by working recursively through the object graph with a map of previously-seen references, it would be possible to reconstruct the graph as a clone of the original without any fingerprint of Hibernate left behind. I call this operation "exporting" the object. Once this export is complete, any serialization tool will be able to traverse the graph without fear of triggering a lazy loading exception.
public MyObject export( Map<String, Object> references ) {
String refId = ExportUtils.buildRefId( this );
if ( references.containsKey( refId ) ) {
return (MyObject) references.get( refId );
}
// perform a shallow clone (no references attached)
MyObject export = shallowClone();
references.put( refId, export );
// now we cleanse the references and collections
List<Child> childrenExport = new ArrayList<Child>();
for ( Child child : children ) {
childrenExport.add( child.export( references ) );
}
export.setChildren( childrenExport );
export.setRelatedObject( relatedObject.export( references ) );
// and so on...
}
Of course, this process could be done automatically using reflection, but this snippet is simply pseudocode that seeks to demonstrate the spirit of the process. I could have also used custom converters on XStream. In our case, we choose to step through the objects manually for other reasons. The point that I am trying to get across is that you should be aware of the presence of these custom collections and proxies and be ready to address them in the case that you need to get back to the pure POJO.
34 Comments from the Peanut Gallery
1 | Posted by john smith on June 21, 2006 at 04:22 PM EST
Or, you could get rid of Hibernate and use OJB. OJB doesn't muck with your POJO's. They are all detached. You have to call save when you want them to be persisted, but you can do whatever you want to them, then call save at the end and OJB will save them.
Problem solved.
2 | Posted by Gavin on June 21, 2006 at 08:48 PM EST
It is of course interesting to know that OJB successfully solves impossible problems. It must be that patented OJB Magic ;-)
LazyInitializationException (or equivalent) exists in all ORM solutions with a well-defined persistence context. (By the way, a process-scoped persistence context is NOT "well-defined".)
Indeed, OJB has proxies just like Hibernate, and would behave exactly the same way if you try to serialize an object graph to XML.
3 | Posted by Dan Allen on June 21, 2006 at 11:28 PM EST
I have no doubt that OJB has it's strengths, but as Gavin mentioned, it is more of a problem that is inherient in the ORM model and it isn't likely going to be solved simply by changing frameworks (even EJB3).
When writing this post, I made a sincere effort not to blame the tool, and encouraged developers to understand what the framework must do to offer the convenience and transparency promised by this model of programming. Sometimes, however, it is necessary to extract the underlying data from the managed object, such as when the data must be exported to XML. In this case, we simply have to do some extra work, but work that is really quite trivial.
No tool is a silver bullet, something Frederick Brooks taught us well. It is very important to understand both what the tool can do and how it accomplishes these tasks. The wrong approach is to go flaming a tool like Hibernate because you are hung up on a lazy loading exception. Take the time to understand why it is happening and determine if you need to adjust your program or your thinking.
4 | Posted by max on June 22, 2006 at 04:02 AM EST
XStream has the notion of Converters which would be a much more sensible thing to use in this context.
5 | Posted by Lari on June 22, 2006 at 06:40 AM EST
To get rid of hibernate collection classes in the output, you could also run the XStream serialisation to DOM (with Dom4JWriter and XStream.marshal) and then run the DOM through a Trax API / XSLT transformation which replaces hibernate collection classes with java.util.* ones (to StreamResult, if you want to get a xml string back)
6 | Posted by Lari on June 22, 2006 at 06:43 AM EST
Similar discussion: http://blog.murf.org/2005/04/06/lazy-loading-and-the-hibernate-cleanser/
7 | Posted by Gavin on June 22, 2006 at 06:43 AM EST
Yes, Dan your actual blog was all quite reasonable and nicely-explained. Though I agree with Max that a better way would be to utilize some kind of custom serializers in your XML serialization solution.
8 | Posted by Solomon on June 22, 2006 at 08:39 AM EST
Hibernate has some pretty neat XML serialization features; i.e. Hibernate can give you a representation of an "object graph" as XML.
Hibernate XML Docs DevX article
9 | Posted by Dan Maher on June 22, 2006 at 10:50 AM EST
With all respect, I'm not sure you have come to the ideal solution with regard to pure architecture. I am not aware of your other design motivations, but I have an opinion given the problem as presented.
"Due to the complexity of our graph, and considering all the circular references, simply cloning was not enough since each object held nested references which themselves were Hibernate-specific implementations of the interfaces."
It appears this is the nature of your problem. If at some point in your app, you cannot be even be sure an object has been fully 'hydrated' from your orm solution, therein is the issue. I urge you as an anonymous friend to reconsider this complexity.
I am not a jboss employee, just a guy that's been around the block with Hibernate.
10 | Posted by Dan Allen on June 22, 2006 at 02:11 PM EST
Thank you all for the great feedback! I elaborated on the snippet of code to clarify the mechanism used to perform the export.
I agree that I could have used custom converters or reflection to automate the process. However, this post is more to present the use case rather than propose a hard and fast solution. Our method of exporting worked for our domain model.
11 | Posted by Alex Popescu on June 22, 2006 at 04:37 PM EST
Hi!
Nice entry, but I wonder if Gavin can comment on this specific fragment:
IMO, if the collection is disconnected I guess it may be a way to inform it not to throw the LazyInitializationException. I am not aware of the implementation, but I guess it would be doable, so I am wondering if there are other reasons behind it.
TIA,
./alex -- .w( the_mindstorm )p. --- (http://themindstorms.blogspot.com)
12 | Posted by Casey Huggins on July 03, 2006 at 07:01 PM EST
@Alex
The collection may be disconnected, but it has still not beein initialized. If it didn't throw an exception, calling code would have no way to know if the class it is seeing was correctly loaded from the DB or is merely an empty proxy.
13 | Posted by trollswagen on August 04, 2006 at 01:37 PM EST
i'm surprised nobody's offered annotations as a way to decouple the dependencies between different components and the object graph. this is a good example:
http://entzi.blogspot.com/2005/08/xstream-jdk15-annotations.html
14 | Posted by Konstantin Pribluda on January 11, 2007 at 09:37 AM EST
I had similar problem and came up with the solution for XStream ( http://xstream.codehaus.org ) See: http://jira.codehaus.org/browse/XSTR-377
15 | Posted by Dan Allen on January 12, 2007 at 09:38 AM EST
Great! I am glad that this problem is starting to get more attention and that solutions are being documented and/or made available. I have to admit that the code sample I provided was a bit vague on providing a solution. I hope to get together a more detailed discussion if time allows.
Again, to be clear, this process is only required when you need to "export" a Hibernate managed POJO to another system. In this case, a "clean" version of the object graph is required.
16 | Posted by Maher Kilani on January 19, 2007 at 06:28 AM EST
Dear All,
I have created a method which would go through the member variables of any object and sets any uninitialized hibernate entities to null.
if you are interested i can post it once am done testing it.
17 | Posted by Tim O'Brien on January 28, 2007 at 10:48 AM EST
Great post, this is a recurring problem with Hibernate in particular, Lazy Loading has advantages, but I can't tell you how many times I've been bitten by the fact that my POJOs are surrounded by these finnicky, magic proxies.
Also, I find it entertaining the number of comments people left which were essentially "You are doing it wrong", or "Your architecture is wrong you have too many objects", or "You should address this in your serialization approach". You did us a service by posting this blog entry, and the usual suspects show up to tell you that you are, in fact, an idiot. :-)
18 | Posted by Sam Martens on March 01, 2007 at 09:32 AM EST
I am currently using container managed persistence so my EJB container will start a Hibernate session and close it after the session bean call has been finished. I have no control over this. Of course the data will be marshalled for an RMI call. This marshalling already gives:
org.hibernate.LazyInitializationException: failed to lazily initialize a collection - no session or session was closed
So, does anybody know if there is a way to use Hibernate lazy loading along with JTA or is this impossible?
19 | Posted by Maher Kilani on March 16, 2007 at 05:54 PM EST
The lazyloading has many advantages. but it is also a pain in the ass whenever you want to use with flex, or when yoiu want to unmarshall object then a lazy loading exception occurs.
The solution is to use a cleaner whenever you are done. if there was a way to upload the class i have implemented, i would have posted it here. if you need the class please contact me.
20 | Posted by Luxspes on July 25, 2007 at 09:14 PM EST
Hi! Great article, first it made me realize I had to build (or find) something to properly serialize Hibernate POJOs...and after finding Granite externalizers to do that, it helped me explain other why externalizers (POJO extractors?) are so important to send Hibernate Objects over the Flex side. bye
21 | Posted by Steve on September 10, 2007 at 04:03 PM EST
I implemented a similar method to what Maher describes in #16, but That only takes care of part of the problem.
There is still the issue of what happens when you try to save updates. Hibernate thinks there is a change to the child collection, so it tries to disassociate the children from the parent, which causes issues if its own.
I'm still trying to find a way to detach the object from Hibernate, remove the uninitialized collections, then properly re-hydrate the object when it comes back from the client.
I would think someone else has run across this before
22 | Posted by Maher Kilani on September 20, 2007 at 04:05 PM EST
What is the framework you are using? spring? struts?
23 | Posted by Phillip Rhodes on October 29, 2007 at 10:50 AM EST
I am solving the same problem but manually copying from the hibernate pojo to a new pojo that was not loaded by hibernate.
24 | Posted by Joe Trankon on November 05, 2007 at 09:31 PM EST
Has anyone written a class that cleans a pojo from any hibernate injections?
The other option for me would be to write a "deep clone" method on every pojo then use that "deep clone" method to retrieve a clean pojo for serialization. Of course this is ugly since I have over 100 pojo's. I guess I could also write a generic object cloner that uses reflection but I am betting performance-wise it isn't a great idea.
25 | Posted by Dan Allen on November 12, 2007 at 02:19 PM EST
I can say from experience, since I implemented a "deep clone" solution on a project, that it is very painful and naive. Avoid that solution because it is a rabbit hole. Do check out the beanlib library as it tries to accomplish a similar goal.
To clarify, Hibernate does not *inject* anything. Rather, it instantiates its own collection implementations and then assigns those collection objects to the appropriate properties on your POJO. Hibernate's collection implementations are "session aware".
26 | Posted by Larry Hughes on November 13, 2007 at 07:56 AM EST
Hi Dan,
I'm actually running into a similar situation as your original post (trying to get rid of the Hibernate proxy). Your last comment suggested of using beanlib. I took a look at it but am not sure how this gets rid of the proxy. I guess from your experience, what is the best way of cleansing hibernate from my POJOs-- XStream or beanlib. Thanks.
27 | Posted by Steve on December 03, 2007 at 01:01 PM EST
I wound up diving down the rabbit hole.
All of my POJOs extend a "DataObject" class. In that class, I wrote a dehydrate and rehydrate method.
The dehydrate method does 3 things, using reflection: 1) If a given property is a collection, determine if it is a hibernate PersistentSet. If it is, and it is uninitialized, replace it with a null. 2) If a given property is a collection, try to see if it is the child end of a bidirectional association. If it is, set the parent attribute to null 3) dehydrate DataObject attributes, and dehydrate each DataObject child in a collection.
The dehydrate method returns a copy of the original object, because my unit tests were having issues with object identity otherwise.
The rehydrate method reverses this process, reassociating children to their parents, and replacing null collections with new Hibernate Persistent Sets if the object has an ID and the collection is not Transient.
I can see that this is a very messy solution, it probably has performance issues, and I'd bet I have some pretty nasty bugs in the code, but it appears to work in my app so far. I could eliminate the performance problem by requiring each POJO to have its own dehydrate/rehydrate pair, which would eliminate the need for reflection, but I don't know if I want to do this for large numbers of POJOs. Having it use reflection in the base class has the added benefit that other POJO developers don't need to get into the details of the Hibernate-SOAP handoff. Only the Service endpoints need to worry about it, and they only need to call the dehydrate/rehydrate methods.
I did notice that there were JAXB annotations to mark the collection attributes as XMLTransient, and map Hibernate Transient to XML properties. that may help with serialization, but I'm not sure how to reassociate parents and children using that approach.
For reference, I'm using Spring/Hibernate on the server, and trying to return object via SOAP without having to maintain 2 object hierarchies (one for Hibernate, one for SOAP)
As for beanlib and XStrem, I haven't had the time to look into those, but remember that removing the Hibernate proxies is only half of the problem - you still have to deal with the circular reference problem.
The research continues...
28 | Posted by NeedInfo on January 29, 2008 at 05:35 AM EST
Can you please provide details on the actual implementation, i urgently need it
29 | Posted by John Seals on February 04, 2008 at 04:25 PM EST
I have yet to get this to work. Anybody got a working example I can use for a reference?
30 | Posted by Adnan on April 13, 2008 at 02:55 AM EST
Hi Dan, Well, i have went thru your pseudocode but what did I understand is that you are copying the Collection and that will be same as using EAGER mechanism? Is it possible that we use Lazy Loading without the LazyInitializationException??
31 | Posted by Dan Allen on April 13, 2008 at 04:40 AM EST
@Adnan, you are correct in saying that what I am proposing here has the effect of eager loading the relationships. The difference, though, is that I am also doing an export of the records, unbinding them from the Hibernate session entirely.
For anyone listening, the way to enabling lazy loading without bumping into a LazyInitializationException is extremely straightforward. The Hibernate session that loaded the entity must remain open.
In most cases, Spring gets this all wrong and closes the session when the transaction ends. You can extend it for the duration of the request using the Open Session in View pattern, but that only gets you so far and has other complications. Alas, this is exactly the problem JBoss Seam was designed to solve. Using a Seam conversation, you can extend the lifetime of a Hibernate session across any number of requests, saving you from having to fear lazy loading. You can read more about Seam's solution in my book, Seam in Action.
32 | Posted by Bryan Deter on May 02, 2008 at 09:41 AM EST
There is an even better solution, depending on your application: turn off lazy fetching by specifying in the association mapping.
There's a reference in Chapter 19 of the hibernate reference documents.
Another solution if that isn't practical (you don't want non-lazy loading for the entire application) would be to change the fetch strategy to JOIN.
33 | Posted by Roy on May 08, 2008 at 04:12 PM EST
Steve... (comm #27)
Can you please provide me details of your solution? "DataObject" class with dehydrate/rehydrate methods?
I'm using Flex, BlazeDS, Java, Hibernate (POJO) and i urgently need it. Thanks!
34 | Posted by Steve on September 04, 2008 at 05:23 PM EST
For those who are interested, I've updated my code as I've found bugs and changed the architecture.
I moved the dehydration code out of my base DataObject (now called BaseEntity), and into a standalone utility class. We've also changed from method level annotations to field level.
We now support accessing our services via RMI, SOAP, or AMF (via BlazeDS). We are using Hibernate in a GlassFish container.
The biggest bug I fixed had to do with the CGLib generated proxies that were present in my object graph - Flex clients couldn't convert them into Flex classes because the class returned was not the Java entity class we had mapped to the Flex entity, it was the CGLib enhanced class. I now strip out all the proxy classes and replace them with the actual entities they represent.
I'd be happy to send a copy of my code to anyone who needs it. I'm still surprised that with all the issues getting Hibernate managed entities across the client-server divide, that there are not more solutions out there to take care of these things.
Got Something to Say?