mojavelinux.com: Open Source Advocacy

Or, you could get rid of Hibernate and use OJB. OJB doesn't muck with your POJO's. They are all detached. You have to call save when you want them to be persisted, but you can do whatever you want to them, then call save at the end and OJB will save them.

Problem solved.

It is of course interesting to know that OJB successfully solves impossible problems. It must be that patented OJB Magic ;-)

LazyInitializationException (or equivalent) exists in all ORM solutions with a well-defined persistence context. (By the way, a process-scoped persistence context is NOT "well-defined".)

Indeed, OJB has proxies just like Hibernate, and would behave exactly the same way if you try to serialize an object graph to XML.

I have no doubt that OJB has it's strengths, but as Gavin mentioned, it is more of a problem that is inherient in the ORM model and it isn't likely going to be solved simply by changing frameworks (even EJB3).

When writing this post, I made a sincere effort not to blame the tool, and encouraged developers to understand what the framework must do to offer the convenience and transparency promised by this model of programming. Sometimes, however, it is necessary to extract the underlying data from the managed object, such as when the data must be exported to XML. In this case, we simply have to do some extra work, but work that is really quite trivial.

No tool is a silver bullet, something Frederick Brooks taught us well. It is very important to understand both what the tool can do and how it accomplishes these tasks. The wrong approach is to go flaming a tool like Hibernate because you are hung up on a lazy loading exception. Take the time to understand why it is happening and determine if you need to adjust your program or your thinking.

XStream has the notion of Converters which would be a much more sensible thing to use in this context.

To get rid of hibernate collection classes in the output, you could also run the XStream serialisation to DOM (with Dom4JWriter and XStream.marshal) and then run the DOM through a Trax API / XSLT transformation which replaces hibernate collection classes with java.util.* ones (to StreamResult, if you want to get a xml string back)

Similar discussion: http://blog.murf.org/2005/04/06/lazy-loading-and-the-hibernate-cleanser/

Yes, Dan your actual blog was all quite reasonable and nicely-explained. Though I agree with Max that a better way would be to utilize some kind of custom serializers in your XML serialization solution.

Hibernate has some pretty neat XML serialization features; i.e. Hibernate can give you a representation of an "object graph" as XML.

Hibernate XML Docs DevX article

With all respect, I'm not sure you have come to the ideal solution with regard to pure architecture. I am not aware of your other design motivations, but I have an opinion given the problem as presented.

"Due to the complexity of our graph, and considering all the circular references, simply cloning was not enough since each object held nested references which themselves were Hibernate-specific implementations of the interfaces."

It appears this is the nature of your problem. If at some point in your app, you cannot be even be sure an object has been fully 'hydrated' from your orm solution, therein is the issue. I urge you as an anonymous friend to reconsider this complexity.

I am not a jboss employee, just a guy that's been around the block with Hibernate.

Thank you all for the great feedback! I elaborated on the snippet of code to clarify the mechanism used to perform the export.

I agree that I could have used custom converters or reflection to automate the process. However, this post is more to present the use case rather than propose a hard and fast solution. Our method of exporting worked for our domain model.

Hi!

Nice entry, but I wonder if Gavin can comment on this specific fragment:

The method that Hibernate offers merely informs the hidden code in the POJO that it should no longer consider itself a part of the current session, and thus allows the object to be reconnected with either the same or perhaps a different Hibernate session. However, the hidden code is still present all the while. Attempting to access uninitialized lazy collections once detached will still throw up the lazy initialization exception.

IMO, if the collection is disconnected I guess it may be a way to inform it not to throw the LazyInitializationException. I am not aware of the implementation, but I guess it would be doable, so I am wondering if there are other reasons behind it.

TIA,

./alex -- .w( the_mindstorm )p. --- (http://themindstorms.blogspot.com)

@Alex

The collection may be disconnected, but it has still not beein initialized. If it didn't throw an exception, calling code would have no way to know if the class it is seeing was correctly loaded from the DB or is merely an empty proxy.

i'm surprised nobody's offered annotations as a way to decouple the dependencies between different components and the object graph. this is a good example:

http://entzi.blogspot.com/2005/08/xstream-jdk15-annotations.html

I had similar problem and came up with the solution for XStream ( http://xstream.codehaus.org ) See: http://jira.codehaus.org/browse/XSTR-377

Great! I am glad that this problem is starting to get more attention and that solutions are being documented and/or made available. I have to admit that the code sample I provided was a bit vague on providing a solution. I hope to get together a more detailed discussion if time allows.

Again, to be clear, this process is only required when you need to "export" a Hibernate managed POJO to another system. In this case, a "clean" version of the object graph is required.

Dear All,

I have created a method which would go through the member variables of any object and sets any uninitialized hibernate entities to null.

if you are interested i can post it once am done testing it.

Great post, this is a recurring problem with Hibernate in particular, Lazy Loading has advantages, but I can't tell you how many times I've been bitten by the fact that my POJOs are surrounded by these finnicky, magic proxies.

Also, I find it entertaining the number of comments people left which were essentially "You are doing it wrong", or "Your architecture is wrong you have too many objects", or "You should address this in your serialization approach". You did us a service by posting this blog entry, and the usual suspects show up to tell you that you are, in fact, an idiot. :-)

I am currently using container managed persistence so my EJB container will start a Hibernate session and close it after the session bean call has been finished. I have no control over this. Of course the data will be marshalled for an RMI call. This marshalling already gives:

org.hibernate.LazyInitializationException: failed to lazily initialize a collection - no session or session was closed

So, does anybody know if there is a way to use Hibernate lazy loading along with JTA or is this impossible?

The lazyloading has many advantages. but it is also a pain in the ass whenever you want to use with flex, or when yoiu want to unmarshall object then a lazy loading exception occurs.

The solution is to use a cleaner whenever you are done. if there was a way to upload the class i have implemented, i would have posted it here. if you need the class please contact me.

Hi! Great article, first it made me realize I had to build (or find) something to properly serialize Hibernate POJOs...and after finding Granite externalizers to do that, it helped me explain other why externalizers (POJO extractors?) are so important to send Hibernate Objects over the Flex side. bye

I implemented a similar method to what Maher describes in #16, but That only takes care of part of the problem.

There is still the issue of what happens when you try to save updates. Hibernate thinks there is a change to the child collection, so it tries to disassociate the children from the parent, which causes issues if its own.

I'm still trying to find a way to detach the object from Hibernate, remove the uninitialized collections, then properly re-hydrate the object when it comes back from the client.

I would think someone else has run across this before

What is the framework you are using? spring? struts?

I am solving the same problem but manually copying from the hibernate pojo to a new pojo that was not loaded by hibernate.

Has anyone written a class that cleans a pojo from any hibernate injections?

The other option for me would be to write a "deep clone" method on every pojo then use that "deep clone" method to retrieve a clean pojo for serialization. Of course this is ugly since I have over 100 pojo's. I guess I could also write a generic object cloner that uses reflection but I am betting performance-wise it isn't a great idea.

I can say from experience, since I implemented a "deep clone" solution on a project, that it is very painful and naive. Avoid that solution because it is a rabbit hole. Do check out the beanlib library as it tries to accomplish a similar goal.

To clarify, Hibernate does not *inject* anything. Rather, it instantiates its own collection implementations and then assigns those collection objects to the appropriate properties on your POJO. Hibernate's collection implementations are "session aware".

Hi Dan,

I'm actually running into a similar situation as your original post (trying to get rid of the Hibernate proxy). Your last comment suggested of using beanlib. I took a look at it but am not sure how this gets rid of the proxy. I guess from your experience, what is the best way of cleansing hibernate from my POJOs-- XStream or beanlib. Thanks.

I wound up diving down the rabbit hole.

All of my POJOs extend a "DataObject" class. In that class, I wrote a dehydrate and rehydrate method.

The dehydrate method does 3 things, using reflection: 1) If a given property is a collection, determine if it is a hibernate PersistentSet. If it is, and it is uninitialized, replace it with a null. 2) If a given property is a collection, try to see if it is the child end of a bidirectional association. If it is, set the parent attribute to null 3) dehydrate DataObject attributes, and dehydrate each DataObject child in a collection.

The dehydrate method returns a copy of the original object, because my unit tests were having issues with object identity otherwise.

The rehydrate method reverses this process, reassociating children to their parents, and replacing null collections with new Hibernate Persistent Sets if the object has an ID and the collection is not Transient.

I can see that this is a very messy solution, it probably has performance issues, and I'd bet I have some pretty nasty bugs in the code, but it appears to work in my app so far. I could eliminate the performance problem by requiring each POJO to have its own dehydrate/rehydrate pair, which would eliminate the need for reflection, but I don't know if I want to do this for large numbers of POJOs. Having it use reflection in the base class has the added benefit that other POJO developers don't need to get into the details of the Hibernate-SOAP handoff. Only the Service endpoints need to worry about it, and they only need to call the dehydrate/rehydrate methods.

I did notice that there were JAXB annotations to mark the collection attributes as XMLTransient, and map Hibernate Transient to XML properties. that may help with serialization, but I'm not sure how to reassociate parents and children using that approach.

For reference, I'm using Spring/Hibernate on the server, and trying to return object via SOAP without having to maintain 2 object hierarchies (one for Hibernate, one for SOAP)

As for beanlib and XStrem, I haven't had the time to look into those, but remember that removing the Hibernate proxies is only half of the problem - you still have to deal with the circular reference problem.

The research continues...

Can you please provide details on the actual implementation, i urgently need it

I have yet to get this to work. Anybody got a working example I can use for a reference?

Hi Dan, Well, i have went thru your pseudocode but what did I understand is that you are copying the Collection and that will be same as using EAGER mechanism? Is it possible that we use Lazy Loading without the LazyInitializationException??

@Adnan, you are correct in saying that what I am proposing here has the effect of eager loading the relationships. The difference, though, is that I am also doing an export of the records, unbinding them from the Hibernate session entirely.

For anyone listening, the way to enabling lazy loading without bumping into a LazyInitializationException is extremely straightforward. The Hibernate session that loaded the entity must remain open.

In most cases, Spring gets this all wrong and closes the session when the transaction ends. You can extend it for the duration of the request using the Open Session in View pattern, but that only gets you so far and has other complications. Alas, this is exactly the problem JBoss Seam was designed to solve. Using a Seam conversation, you can extend the lifetime of a Hibernate session across any number of requests, saving you from having to fear lazy loading. You can read more about Seam's solution in my book, Seam in Action.

There is an even better solution, depending on your application: turn off lazy fetching by specifying in the association mapping.

There's a reference in Chapter 19 of the hibernate reference documents.

Another solution if that isn't practical (you don't want non-lazy loading for the entire application) would be to change the fetch strategy to JOIN.

Steve... (comm #27)

Can you please provide me details of your solution? "DataObject" class with dehydrate/rehydrate methods?

I'm using Flex, BlazeDS, Java, Hibernate (POJO) and i urgently need it. Thanks!

For those who are interested, I've updated my code as I've found bugs and changed the architecture.

I moved the dehydration code out of my base DataObject (now called BaseEntity), and into a standalone utility class. We've also changed from method level annotations to field level.

We now support accessing our services via RMI, SOAP, or AMF (via BlazeDS). We are using Hibernate in a GlassFish container.

The biggest bug I fixed had to do with the CGLib generated proxies that were present in my object graph - Flex clients couldn't convert them into Flex classes because the class returned was not the Java entity class we had mapped to the Flex entity, it was the CGLib enhanced class. I now strip out all the proxy classes and replace them with the actual entities they represent.

I'd be happy to send a copy of my code to anyone who needs it. I'm still surprised that with all the issues getting Hibernate managed entities across the client-server divide, that there are not more solutions out there to take care of these things.

I've refactored my code once again - hopefully for the better }:-)

I've never been too happy with terms dehydrate and rehydrate to describe what I was doing, so I changed them to prune and unprune, since I am "pruning" problem elements from the object graph. The new class is called EntityPruner. I also never liked the way the EntityUtil depended on a particular base class, so the EntityPruner now depends on a PrunableEntity interface, which allows developers to define any entity hierarchy that makes sense in their organization, as long as entities implement the PrunableEntity interface.

In our organization, We've added REST to the list of standards we support via Jersey, and entities pruned with EntityPruner work just fine with Jersey.

As always, drop me an email if you are interested in the code.

Steve

Hi Guys, please check the code on http://maherkilani.blogspot.com/2008/12/hibernate-proxy-cleaner.html

This is great work. We will definitely be reviewing these two projects if/when we work on integrating a entity translation mechanism for remoting into Seam.

Hi all,

I am working since 18 month on a library that clone persistent entity, removing lazy properties and replacing persistent collections, and is able to merge this proxy information when back on server.

It was first named hibernate4gwt (it first goal was to allow to send Hibernate entities through GWT RPC serialization) and is now known as Gilead (http://gilead.sourceforge.net).

It can be used to serialize entities to XML, GWT, BlazeDS and so on.

Hope this helps Bruno

Hey @Bruno, I figured you would show up eventually. Thanks for the pointer.

For all of those following this blog working on similar projects, I encourage you to exchange ideas so that each project can improve, or to consolidate where it seems appropriate.

Bruno's serializer actually works really well for me :)

I finally got a BlazeDS + Hibernate/Spring + Flex working with lazy-loading using Gilead.

I tried to make a ant script for it, but the mxmlc task in ant is still kinda bugged but whatever great work bruno :)

for anyone finding this over google: take a look at dozer: http://dozer.sourceforge.net/. no need to reinvent the wheel...

Rex,

Are you using the dynamic proxy mode for the BlazeDS + hibernate/spring + flex solution? Are you able to post step by step instructions?

I have learned this the hard way: Don't ever serialize your hibernate business objects.

Have anyone tried the Castor Framework yet?

http://www.castor.org/how-to-map-hibernate-proxies.html

Simply configure proxies with

marshaller.setProperty("org.exolab.castor.xml.proxyInterfaces", "net.sf.cglib.proxy.Factory,org.hibernate.proxy.HibernateProxy");

and castor will do the proxy stuff for you.

On behalf of Steve Saliman:

The Google project for entity-pruner now up at http://code.google.com/p/entity-pruner. This library provides a class that prunes JPA entities so they can be serialized for web service calls. The code is all there, and the unit tests pass (if the database is setup).

Thanks for this thread. Currently I am running into these exact problems while processing hibernate entities using JBoss Rules (or Drools): MVEL compilies rules using reflection and cannot handle proxies well.

For those who are interested, I have updated my EntityPruner code to fix a bug with lazy loading of parent proxy objects.

This fix made some fundamental changes to the way the EntityPruner works, since I now need a session to unprune entities.

The code can be found at the link above (Thanks Dan), and the classes are pretty well documented.

I made another enhancement to the EntityPruner today. It now handles child collections that are Lists and SortedSets. I think this should take care of the overwhelming majority of Entity relationship use cases.

The latest code is always available at http://code.google.com/p/entity-pruner/

I wonder if hibernate team had thought about this matter when they designed hibernate.

I got a question for you. I get an object back from hibernate, call the getters that populate Sets that are mapped by hibernate, then close the session and I DO NOT DETACH anything from the session but its closed anyway. So when I go back to call these getters that were populated by hibernate they should still be there because they were loaded already right? Well I store this cached version in a web session and over time it seems to forget or loose the data that was once gotten by the session because it starts throwing lazy loading exceptions like it was trying to get the data again? Is there a caching mechanism or a recheck after so many minutes timer that hibernate has? Or can the close session on garbage collection collect all the objects it once loaded for m?

I developed a solution last year that solves the above problem along with the n+1 problem. I'm using CGLib to create proxy versions of the entity classes that are used for communication with the client. simple mask objects are used to describe the objects graph the client expects to receive. Proxies are then created just before the hibernate session closes and sent back to the client.

This approach works really well and it does have very little overhead in development.

If anyone is interested in it, just drop me an email.

N¡k

Maybe it will not resolve all your problems with serializing Hibernate to XStream but you could do something like that:

myXStream.omitField (AbstractPersistentCollection.class, "initialized"); myXStream.omitField (AbstractPersistentCollection.class, "storedSnapshot");

I released a new version of the EntityPruner today. The current version allows users to specify a maximum depth, or a specific list of attributes to exclude from the pruned graph. This is useful when business logic causes children to be loaded that clients are not interested in.

The latest code is always available at http://code.google.com/p/entity-pruner/

Hi Steve,

I am using entity pruner 2.3.0 in my application. I am seeing problem with collections inside an entity. They do not show up after pruning. Do you see what could be wrong? You can find the source here: http://code.google.com/p/loandb

But, here is the relevant code:

Application application = applicationService.getApp(Long.valueOf(appid)); if (application == null) { throw new NotFoundException("Application not found for id : " + appid); } EntityPrunerHibernateJpa pruner = new EntityPrunerHibernateJpa(); pruner.prune(application, 2); return application;

I see the application XML show up correctly, but the applicants collection do not show up.

Am I missing anything?

Thanks for keeping this awesome utility up-to-date.

-Arul

Just to say that entity-pruner was the tool I was looking for!!! Tks Steve!. This is a better approach than using xstream modifiers since you got your pojo "cleaned" before serializing it. This provides better flexlbility for your application once you can reuse it for everthing else.

I faced a similar problem although Im not using hibernate. I looked at the enitity-pruner but its not quite what i was looking for as I was looking for a more simpler solution.

I came up with a very simple solution using reflection that deproxies CGLIB enhaned objects like a charm.

please visit http://www.anzaan.com/2010/06/serializing-cglib-enhanced-proxy-into-json-using-xstream/ for example and code.

I found this wonderful blog post because I have the same issue but for different reasons that have nothing to do with XML serialization. It's stuff like this that convinces me that Hibernate just has an attitude problem. One shouldn't have to go to such measures to remove Hibernate's tainted claws from a normal POJO object. Unfortunately I also care about performance and am not sure if I want to pay the cost of shallow copy solution, reflection-based or otherwise. I will have to benchmark and see how much of a difference it makes. I can't imagine that refectlively allocating and copying a dozens of fields on thousands of objects can happen instantaneously.

My use case may be in the extreme. Has anyone else attempted something like this?

I want to keep a history table. As JPA entities are changed or deleted, I want to serialize their former content into a single column of my history table. The content would then be handy to be restored if needed. I think the concept discussed above, handling the hibernate proxy, comes to play, right?

From the conversation, I detect a two step process: 1) clean the hibernate proxy, 2) safely serialize. If so, then some of the solutions above are only doing the first part (the entity pruner, I think falls into this class). It looks like castor will do both at the same time.

Does jaxb do both steps at once? Seems to be an advantage over other libraries because jaxb is part of the java runtime.

@John, you may want to take a look at Envers for that use case.

The Envers project aims to enable easy auditing/versioning of persistent classes. All that you have to do is annotate your persistent class or some of its properties, that you want to audit, with @Audited. For each audited entity, a table will be created, which will hold the history of changes made to the entity. You can then retrieve and query historical data without much effort.

I'm not sure if I am supposed to be having this problem or not - reading this thread is giving me NHTPSIBAD ("Not Having This Problem, Should I Be? Anxiety Disorder") :)

We're not seeing a problem but then again we're not using GWT but using Wicket for the UI so, so far at least, there has never been a need to do this apparently, kludgey and awkward 'entity pruning' exercise.

Is this problem only restricted to GWT (or similar UI frameworks) because of the need to serialize objects up to the client and back over RPC? If that's the cause of the problem then hopefully I can safely ignore this post and continue in my 'ignorance is bliss' flight pattern.

Seems this post has been around for a while but still the same problem is encountered by many of us, there are a number of ways around the issue, all are a bit of a faff but I'd be interested in giving the entity pruner a bash so thanks for the effort on that..

I've recently returned to using Hibernate under JPA after a few years away and although it is more code have found that having a service layer above my daos which manually replace any hibernate collections etc. with java.util collections before returning them to any client code has worked ok so far. This is in the context of a struts2 web app and whilst maintaining the session could be an option I really don't want any database reads occurring whilst rendering the view and as such want all objects to be free from being Hibernate session aware before they are sent there. In many ways I wish Hibernate did proxy the pojos instead of providing it's own session aware implementations of collections so that once a session is closed it should be up to Hibernate (or any other ORM tool) to remove the proxies and any other session aware collections. A feature like this would benefit everyone greatly and IMO is the 'right' way for this to happen. Does anyone know of any other JPA implementations that actually do this or similar as I'd prefer to use those instead of coding aropund Hibernate specific issues?

I believe the fundamental problem that the original author ran into, was shipping Hibernate's POJO beyond where they should go. I do not believing binding external services nor UI code to Hibernate POJO's is a good idea.

In one of the responses above, someone laments that this is a pain in the arse and asks how are others doing this.

I've used two approaches, both with great success. One approach is to have a "mating" set of 'Data Transfer Objects' (DTOs) that mirror the POJOs. These objects can be snapped together in many different subgraphs from the original Hibernate POJOs. The DTOs are built while the Hibernate session is alive from "slices" of the POJOs. For the given server API, the appropriate subgraph DTOs are returned. The DTOs will serialize just fine and not have any 'proxy' issues.

Instead of DTOs, you can also use a "Map of Maps" representation of your return object sub-graphs. The client code (and down downstream services) would work on these Map of Maps. Each map corresponds to a Hibernate POJO, with the key being a field name and the value being the value from the Hibernate POJO.

Which is better, DTOs or Map of Maps? It depends. If there is a need for "localized" searching need within the return subgraphs, I would recommend using DTOs, so that you can hang "find" methods and such on those.

Check out the json-io project for a fast method to traverse object graphs: http://code.google.com/p/json-io/

Anybody tried this option: Remote Lazy Loading in Hibernate . . . What if these lazy loaded objects can still maintain their references even at the remote client layer (where there is no Hibernate Session) and still be able to lazy load data? This is quite possible and this concept of lazy loading data even from a remote client is called remote lazy loading.

In this article we'll discuss the solution by extendinag Hibernate's lazy loading framework. We'll use 3.2.x version of Hibernate library.

For those who still follow this thread (it's hard to believe it still has some legs), I've released a new version of the EntityPruner. It can be found at http://code.google.com/p/entity-pruner/

There are two main things this release addresses:

1) Performance. Reflection in Java is notoriously slow, so I now cache the metadata I get via reflection. In one test, I had an operation go from 500ms to just 50ms.

2) There are now more options when populating an entity with the EntityUtil class, or pruning it with the EntityPruner class. These classes now take an option Map, which has options like "select", "include", and "depth", which behave in a manner similar to Rails.

As always, the EntityPruner allows you to make entities safe for marshalling or serializing to a client without the need for DTOs.

Enjoy,

Steve Saliman

I needed Serialization support and go to it by subclassing ObjectOutputStream. Here is the code handling just PersistentSet at the moment:

package org.hibernate.collection;

import java.io.IOException;
import java.io.ObjectOutputStream;
import java.io.OutputStream;

public class PersistentObjectOutputStream
	extends ObjectOutputStream
{

    public PersistentObjectOutputStream( OutputStream out )
    	throws IOException
    {
    	super( out );
    	enableReplaceObject( true );
    }

    public Object replaceObject( Object obj )
    	throws IOException
    {
    	if( obj instanceof PersistentSet )
    	{
    		return ((PersistentSet)obj).set;
    	}

    	return obj;
    }

}

Works great and has the side benefit that the lazy load is not triggered here - that is, it's up to the application to do something to trigger the lazy load.

Maybe some of you still interested on this topic. You'll find several working solutions for clean Entity cloning below http://alexander.holbreich.org/2012/02/release-your-entities/

@Steve: I have my project configured in Maven, and i am planning to use the entity pruner in my project, appreciate if you can include pom entries for entity pruner project.

Thank you.

It's hard to believe this thread still has legs!

For those who are still following it, I've moved my Entity Pruner from Google to GitHub, and I've released the build artifacts to Maven Central. The entity pruner can now be used by adding net.saliman:entity-pruner:3.1.0 to your Maven and Gradle builds, and the source code can be found at https://github.com/stevesaliman/entity-pruner

Steve

Hi,

We are testing a migration of our application from H2 to H3 and believe we are running into this problem or something very similar. Multiple servers, old servers on H2 serialize objects to send to newer server on H3 and we can't de-serialize the objects on H3 server because the persistent collections jars (net.sf.hibernate*) have been removed. I am guessing that putting the H2 jars on the H3 server may not be a good idea? Sounds like we should be serializing WITHOUT the hibernate specific classes, right? Other suggestions?

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.