Stephen Forte's Blog

 RSS/Feedburner
      Home     Steve & The Tank       

  Friday, June 27, 2008

     Impedance Mismatch

Unless you have been living under a rock, you should already know about the controversy over the nHibernate mafia's Microsoft Entity Framework's Vote of No Confidence.  The manifesto says that the upcoming Entity Framework (to be released soon via .NET 3.5 SP1) is just not usable. As far as ORMs go, there are some valid points working against EF- in order of (my) importance:

  • Lack of Lazy Loading (The EF team said that they had "explicit" lazy loading, but that is just playing with words and the team has since retracted from that.)
  • Lack of persistence ignorance (at best this is inelegant design, at worst it will cripple the performance of your application and make it a burden to maintain)
  • Too many merge conflicts when using a model in a source control environment
  • Bad design for TDD development (and in my opinion, CI as well)

The manifesto also brings up some Domain Driven Design issues that I feel are less important than the above four bullets. To their credit the normally reserved EF team has responded quickly and nicely to the manifesto. On the heels of last week's transparent-design process announcement, Tim Mallalieu gave a detailed reply which has started a good discussion and wiki. They are listening and that is good.  Since the Entity Framework is starting work this week on v.next, they have also put together an advisory council including Eric Evans, Martin Fowler, Paul Hruby, Jimmy Nilsson, and myself.

To some degree the vote of no confidence worked, they got Tim's attention. I think that the manifesto has some very valid points, but I also think it lacks professionalism. Anyone can say something sucks, it is more important to give valid feedback, suggest solutions, and engage in a dialog. (The vote of no confidence was just too in your face-and a dialog was stated only because of the professionalism of the EF team.) In addition there are some mafia style attacks on anyone who does not agree with them, most recently against the always honest and open Julie Lerman.

So this blog post sounds like an endorsement of the Entity Framework and ORMs in general, right?

Wrong.

My first problem with ORMs in general is that they force you into a "objects first" box. Design your application and then click a button and magically all the data modeling and data access code will work itself out. This is wrong because it makes you very application centric and a lot of times a database model is going to support far more than your application. In addition an SOA environment will also conflict with ORM.

I prefer to build the application's object model and the data model at about the same time, with a "whiteboarding" approach that outlines the flows of data and functionality across the business process and problem set. Maybe it is the MBA talking but I tend to be "business and customers first" when I design a system. (Those of you that know me know that I have designed some very large and scalable systems in my day.) I usually like to "follow the money" as Ron Rose of Priceline taught me 10 years ago.  In addition, I prefer a very agile design process where you are constantly making changes to your design in the early sprints, the database and object model are both part of this process.

My second major problem with ORMs is that ORMs are a solution to a problem that should not be solved. Developers who write object oriented and procedural code like C# and Java have trouble learning the set-based mathematics theory that govern the SQL language. Developers are just plain old lazy and don't want to code SQL since it is too "hard." That is why you see bad T-SQL: developers try to solve it their way, not in a set-based way.

The premise of EF, LINQ, nHibernate, and Luca Bolongese’s original premise with ObjectSpaces, is that set-based theory causes an “impedance mismatch” between data access and all the other (more procedural) coding we do.  And it’s ORMs to the rescue to resolve the impedance mismatch.

So ORMs are trying to solve the issue of data access in a way that C# and VB developers can understand: objects, procedural, etc.  That is why they are doomed to fail. The further you abstract the developer from thinking in a set-based way and have them write in a procedural way and have the computer (ORM) convert it to a set-based way, the worse we will be off over time.

What I am saying (and have been saying for a long time) is that we should accept, no, embrace the impedance mismatch!  While others are saying we should eradicate it, I say embrace it.

ORM tools should evolve to get closer to the database, not further away.

One of the biggest hassles I see with LINQ to SQL is the typical many-to-many problem. If I have a table of Ocean Liners, vessels,  and ports, I’ll typically have a relational linking table to connect the vessels and ports via a sailing. (Can you tell I am working with Ocean Freight at the moment?) The last thing I want at the object layer is three tables! (And then another table to look up the Ocean Liner that operates the vessel.) Unfortunately, this is what most tools give me. Actually I don't even want one table, I want to hook object functionality to underlying stored procedures. I really want a port object with a vessel collection that also contains the ocean liner information. At least the Entity Framework does this however I have major concerns about the performance of the code it produces.

The ironic thing I’m now seeing is developers who are lazy and don't want to learn SQL using tools that will produce SQL for them. The SQL is bad, and now those same anti-SQL lazy developers are struggling to read pages of generated and near-unreadable SQL trying to solve performance problems. They’re dealing with SQL that’s more verbose and orders of magnitude harder to understand than what was needed in the first place!

So where do we go from here? We can't just keep pretending that this mismatch problem can be solved and keep throwing good money after bad. As Ted Neward said that is the Vietnam of Computer Science.

I say that developers should start embracing the impedance mismatch. I also say that Microsoft and other ORM vendors need to realize that embracing the mismatch is a good thing and design ORM tools that allow this. (This is the advice I will give at the EF council.) I am not anti-ORM (even tho I did trash LINQ on stage at TechEd last year), but I am pro using the right tool/technology for the right job. ORM is real good for CRUD and real bad at other things. Let's start from there. To quote Ted, "Developers [should] simply accept that there is no way to efficiently and easily close the loop on the O/R mismatch, and use an O/R-M to solve 80% (or 50% or 95%, or whatever percentage seems appropriate) of the problem and make use of SQL and relational-based access (such as "raw" JDBC or ADO.NET) to carry them past those areas where an O/R-M would create problems."

Hopefully "cooler heads will prevail."

 

Update 6/29/08

My fellow RD Greg Low brought to my attention that my discussion on Linq to SQL above is very similar to an email he sent not to long ago. At Triton Works (the ocean containers) we used his example in our own entity model debate and I realized after he pointed it out to me that the words above are just too close to his, so I am giving him the credit above. (Though he did not call programmers lazy and anti-SQL, I did.)

FYI, we went with POAN or Plain Old ADO.NET, ocean containers are old school.

The inspiration for this post came from not only the public debate on the EF vote of no confidence but also a private email discussion on nHybernate, EF and Linq to SQL between:  Greg, Andrew Brust, Rocky Lhotka, Sten Sundbland, Joel Semeniuk, Barry Gervin, and myself. I won't tell you who was on what side. If you want to know, you will have to ask them. Hint, hint, Canadians are lazy and anti-SQL. :)



SQL Server

Friday, June 27, 2008 8:04:49 AM (Eastern Standard Time, UTC-05:00)
Comments [4]  |  Trackback Related posts:
SQL Server 2008 RC0-Lost a Few Hours
SQL Server 2008 XML: XML DML Enhancements
SQL Server 2008 XML: XQuery Enhancements
SQL Server 2008 XML: XSD Enhancements-Union and List Types
SQL Server 2008 February CTP Installed: New XSD Features for XML Data
Programming SQL Server 2005 in French, Italian and Polish
Tracked by:
"ORM – We’re still stuck in Vietnam" (http://jonas.follesoe.no) [Trackback]
"http://jonas.follesoe.no/PermaLink,guid,ae6eeabf-2fc4-4402-a082-1848b7ab1a65.as... [Pingback]
"Ward & I talk over the EF Vote of No Confidence Document" (Jeremy D. Miller -- ... [Trackback]
"Ward & I talk over the EF Vote of No Confidence Document" (Community Blogs) [Trackback]
"Impedance Mismatch Reframing" (Greg Young [MVP]) [Trackback]
"Impedance Mismatch Reframing" (Community Blogs) [Trackback]
"http://codebetter.com/blogs/gregyoung/archive/2008/07/24/impedance-mismatch-ref... [Pingback]
"Impedance Mismatch and System Evolution" (Ayende @ Rahien) [Trackback]
"New and Notable 258" (Sam Gentile The World According to MSCOREE) [Trackback]
"The Data Disconnect" (CodeThinked) [Trackback]
"http://feeds.feedburner.com/~r/CodeBetter/~3/345206235/impedance-mismatch-refra... [Pingback]
"http://feeds.feedburner.com/~r/CodeBetter/~3/342711513/ward-amp-i-talk-over-the... [Pingback]



Tuesday, July 01, 2008 9:27:00 AM (Eastern Standard Time, UTC-05:00)
Have you looked at iBatis (iBatis.NET). It's well respected and if you want to keep the database as a first class citizen, it's the best solution I've seen in ORM land. It's definitely worth a look and is a great solution where it fits.
Thursday, July 24, 2008 6:29:31 PM (Eastern Standard Time, UTC-05:00)
I think it's fantastic of you to perpetuate the nHibernate Mafia moniker. I really appreciate that dialog in the mainstream Microsoft community is preserving this kind of dialectic trickery for future generations. Now lets say "weapons of mass destruction" and "Iraq" together ten times fast and see if we can win that argument through cheap tricks as well.
Friday, July 25, 2008 10:43:24 AM (Eastern Standard Time, UTC-05:00)
"Developers who write object oriented and procedural code like C# and Java have trouble learning the set-based mathematics theory that govern the SQL language. Developers are just plain old lazy and don't want to code SQL since it is too 'hard.'"

How arrogant are you? I started my development career as a database guy writing SQL and stored procedures until getting into .NET at 1.0. I HATED all the plumbing code that I had to write AND MAINTAIN just to do simple CRUD based operations. Using an ORM like NHibernate, even if you just do a simple object wrapper over your schema, saves MAJOR development time and increases productivity by handling the vast majority of that for you. So I guess it is a matter of being lazy... I want something else to do the redundant/mind-numbing tasks for me, not because I ignorant of SQL but because I DO understand it and would rather focus on solving real problems.

Besides, handle this requirement using your hand written SQL: "We are changing database vendors, how much application re-work is it to port this over to XXXdb?". NHibernate… change the mappings to the new database vendor… done. You… not so easy.
Monday, August 04, 2008 7:04:39 AM (Eastern Standard Time, UTC-05:00)
This section jumped out at me so I felt like commenting on it...

"One of the biggest hassles I see with LINQ to SQL is the typical many-to-many problem. If I have a table of Ocean Liners, vessels, and ports, I’ll typically have a relational linking table to connect the vessels and ports via a sailing. (Can you tell I am working with Ocean Freight at the moment?) The last thing I want at the object layer is three tables!"

With some ORMs you can definitely avoid the need to have the linking table in the model, if thats what you want. If the association itself is important (and I find it often is) then you can add it to the model, for example where the association is temporal or carries its own meaning/information.


"Unfortunately, this is what most tools give me."

I'm not sure this is true, I've definitely used ORMs that can handle situations like the ones you describe.
Comments are closed.









newtelligence dasBlog 2.0.7226.0

Copyright © 2008 Stephen Forte. Available under the Creative Commons Attribution 3.0 License.

 The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way