Why ORM Should Be Used, At Least Most Of TheTime

Every so often a new project starts up near me, and this project requires interacting with a database.  Someone who has used an ORM (Object Relation Mapping) tool in the past suggests using it again.  Whether it is nHibernate, LLBLGen Pro or Entity Framework, the person must have had a somewhat positive experience, or they wouldn’t suggest using again. 

Almost as certainly someone who has traditionally put CRUD and some business logic in stored procedures and not used an ORM tool in the past fights the suggestion to use the ORM tool.  They most likely have gotten good at writing and tweaking stored procedures and had a positive experience with this approach or they would not fight against the ORM tool.  As usual there is no right or wrong way, just different approaches that may be more appropriate for the task at hand.

Why Use An ORM Tool:

The reason I suggest that ORM tools are the right choice most of the time is because most projects I see have aggressive time lines and limited development resources.  To that end the ORM tool lets me write a lot less code to do the same amount of work.  Take the below two methods, one gets all customers using Entity Framework 4 and the other gets the list manually using a SqlConnection, SqlCommand and SqlDataReader:

/// <summary>
        /// Method using ORM To Get Customers
        /// </summary>
        /// <returns></returns>
        public List<Customer> GetAllCustomersEF()
        {
            using (NorthwindEntities nwDB = new NorthwindEntities())
            {
                return (from cus in nwDB.Customers
                        select cus).ToList();

            }
        }

        /// <summary>
        /// Method using straight SQL to get all customers
        /// </summary>
        /// <returns></returns>
        public List<Customer> GetAllCustomersSQL()
        {
            var retList = new List<Customer>();

            string cmdText = "SELECT * FROM CUSTOMERS";
            
            using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings[0].ConnectionString))
            using (SqlCommand cmd = new SqlCommand(cmdText, conn))
            {
                conn.Open();
                using (SqlDataReader dataRD = cmd.ExecuteReader())
                {
                    while(dataRD.Read())
                    {
                        retList.Add(CreateCustomerFromDataReader(dataRD));
                    }
                }

            }

            return retList;

        }

        /// <summary>
        /// Method To Create Customer From Datareader record
        /// </summary>
        /// <param name="dataRD"></param>
        /// <returns></returns>
        private static Customer CreateCustomerFromDataReader(SqlDataReader dataRD)
        {
            Customer customer = new Customer();
            customer.Address = dataRD["Address"].ToString();
            customer.City = dataRD["City"].ToString();
            customer.CustomerID = dataRD["CustomerID"].ToString();
            customer.CompanyName = dataRD["CompanyName"].ToString();
            customer.ContactName = dataRD["ContactName"].ToString();
            //.. repeat until all fields full

            return customer;
        }

As you can see above there is a lot less code required using Entity Framework 4.  The data class, Customer, is actually code genned for me.  And the projection into the object from the database is also done for me.  All I have to do is write a query using LINQ syntax.

To get the data using a SQLCommand I have write the query in SQL either inline (as is done above) or in a stored procedure.  I have to then iterate through the results and fill a Customer class of my own making, so I have to create and maintain the code for projecting the data from the database into my objects.

The above is a very simple case,  add filtering or prefetching data in the object graph and the amount of extra code I have to write in the straight sql solution grows exponentially compared to the Entity 4 solution.  Below is possible code for grabbing all customers but with their orders also.

/// <summary>
        /// Method using ORM To Get Customers
        /// </summary>
        /// <returns></returns>
        public List<Customer> GetAllCustomersWithOrdersEF()
        {
            using (NorthwindEntities nwDB = new NorthwindEntities())
            { 
                return (from cus in nwDB.Customers.Include("Orders")
                        select cus).ToList();

            }
        }

        /// <summary>
        /// Method using straight SQL to get all customers
        /// </summary>
        /// <returns></returns>
        public List<Customer> GetAllCustomersWithOrdersAllDataSQL()
        {
            var retList = new List<Customer>();

            string cmdText = "SELECT * FROM CUSTOMERS"
                             + " LEFT JOIN ORDERS "
                             + " ON CUSTOMERS.CustomerID = ORDERS.CustomerID Order By Customers.CustomerID";
            
            using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings[0].ConnectionString))
            using (SqlCommand cmd = new SqlCommand(cmdText, conn))
            {
                conn.Open();
                using (SqlDataReader dataRD = cmd.ExecuteReader())
                { 
                    Customer customer = null;
                    string customerID = "";
                    while(dataRD.Read())
                    {
                        if (customerID != dataRD["CustomerID"].ToString())
                        {
                            if (customer != null)
                            {
                                retList.Add(customer);
                            }
                            customer = CreateCustomerFromDataReader(dataRD);
                        }
                        
                        customer.Orders.Add(CreateOrderFromDataReader(dataRD));
                    }
                    if (customer != null)
                    {
                        retList.Add(customer);
                    }
                }

            }

            return retList;

        }

        private Order CreateOrderFromDataReader(SqlDataReader dataRD)
        {
            Order retOrder = new Order();
            ///...fill order
            ///
            return retOrder;
        }

        /// <summary>
        /// Method To Create Customer From Datareader record
        /// </summary>
        /// <param name="dataRD"></param>
        /// <returns></returns>
        private static Customer CreateCustomerFromDataReader(SqlDataReader dataRD)
        {
            Customer customer = new Customer();
            customer.Address = dataRD["Address"].ToString();
            customer.City = dataRD["City"].ToString();
            customer.CustomerID = dataRD["CustomerID"].ToString();
            customer.CompanyName = dataRD["CompanyName"].ToString();
            customer.ContactName = dataRD["ContactName"].ToString();
            //.. repeat until all fields full
           
            return customer;
        }

To add the prefetch of Orders in the straight SQL solution I have to write code to project into my customer than check each row to see if the customer changes, and then add orders to the customer. This supposes I do not mind that the customer data is duplicated for each order returned for the same company. If I want to optimize data transfer things get even more complex.

Even in a fairly simple example I have a lot less code to write and maintain using the ORM solution.  When the database schema changes this means I have less code I’ll have to update to accommodate a schema change.  If the ORM is statically typed, as is Entity Framework 4 and LLBL Gen Pro, then I’ll also have the benefit of seeing where my code breaks when the schema is changed at compile time.   The less code I have to write by hand and alter when I make changes to my data model the better, and the easier these changes are to find the better.

ORM tools have not always been so friendly to data model changes.  nHibernate uses an XML mapping file, or used to.  Up until Entity Framework4 synchronizing changes to your schema into code was not that straightforward.    LLBL Gen Pro code generates its data layer so handles changes to the schema pretty well, but was hard to learn to use.  Even given these earlier drawbacks, I would still argue they allow more rapid development with limited resources available.

Also, ORM tools allow you to rely on the expertise of those who wrote them when it comes to creating the SQL that is actually run on the database.  If you have a DBA and are confident that you can tweak your own SQL effectively this isn’t an issue.  More and more, however, I run into projects where no one has a very deep SQL background.

Why Use Straight SQL:

None of this is to say that straight SQL does not have its place.  I always use stored procedures in reporting oriented applications, as their needs are heavily data processing intensive.   In circumstances where performance is the highest priority and time line and available resources are not an issue then the straight SQL solution is better suited.  If you know what you are doing you can achieve higher performance with your own code.  However, you have to know what you are doing.  Writing your own data layer does not guarantee better performance.

Conclusion:

As the title suggests I believe an ORM tool is the best choice most of the time.  Most projects have short time frames and limited resources.  Where performance is the absolute top priority or you have heavily data processing tasks in the application then a straight SQL approach may make sense.  Even in these cases I might suggest a hybrid approach, where straight SQL is used only where required, so areas of the application that don’t require heavy data processing or top performance are easier to build and maintain.

Restful Services, Who Needs A WSDL Anyway?

I remember back when I first started working with web services how happy I was that .net created it’s own client proxies. Hitting any web service that exposed a WSDL was a snap in .net, I didn’t need to know anything about SOAP or even XML for that matter. All I need was to created my trusted web reference and off I went consuming web services both near and far.

Now later I came to realize that, in .net 2.0 anyway, all the services and their arguments were created in proxies with their own name spaces, and the proxy arguments had only public fields, not properties so couldn’t bind to them. This prompted me to create wrappers around all .net 2.0 services to transform arguments and returns types to common objects and such. Windows Communication Foundation (WCF) came along and took care of all these issues for me. Still, even with the introduction of WCF I was pointing my trusted IDE at a WSDL for the most part, and it was doing it’s client proxy magic. I still needed no jedi like WSDL knowledge, although to configure WCF I know actually did need to know something about XML (especially setting up certificates and error handling).

Now we come the world of Restful services. No more WSDL, but also, no more proxy magic! I can’t point my trusted IDE at a URL and have things done for me. Now I have to actually do things for myself, sort of. It seems most publishers of Restful services (Amazon, Netflix) give you documentation about their API, so you can consume their services directly yourself, but they also provide client implementations in popular platforms, so you don’t have to write code against the restful API yourself if you don’t want to. It’s probably better since I think I have more faith in client proxy code created by the publisher of the Restful API then I do with the code generated by my trusty IDE. Plus I can look at and makes sense of the communication in an interaction with a Restful API. SOAP was always a mystery, especially when it came to authentication.

Now as we start publishing Restful services ourselves on my current project, I’m eager to publish client proxies for those that will consume them. I Feel like I’m returning the favor of the Restful services we’re consuming by using client proxies provided by the service itself. We are using WCF to host or Restful services and that in itself has some interesting nuances. Microsoft publishes a Restful WCF starter kit, which is useful to get up something that requires very little configuration. We quickly ran into problems when needing to configure our service to use HTTPS and authentication and ended up abandoning the starter kits base service object WebServiceHost, for the traditional WCF ServiceHost configured with a WebHTTPBinding and WebHTTPBehavior.  Using these allowed us to configure our Restful Services with the same XML constructs we used in our SOAP based WCF services.

All IQueryables Are Not The Same

On my current project we use an Object Relational Mapping (ORM) tool named LLBL Gen Pro.  LLBL has a really nice LINQ implementation over it which allows for writing queries much more naturally and in a strongly typed environment.  nHibernate has long had a query language that looked similar to SQL named HQL (Hibernate Query Language).  Unlike LINQ, HQL was weakly typed so changes to types wouldn’t result in compile time errors.  Since LINQ is strongly typed this is not an issue.  When we decided on LLBL Gen Pro it was before nHibernate Linq existed.  We liked being able to use LINQ for querying since could leverage the learning from using LINQ to Objects from a syntactic perspective.

The above is why we choose LLBL and we have been please with the LINQ implementation.  When using LINQ, however, it is important to keep in mind who is the daddy of an IQueryable or IEnumerable you are working with. 

I recently came across a scenario like the below where I retrieved a result set  and then further refine the result set.


//--query against IQueryable that is LLBL Entity
var codes = from c in companyQ
                 where c.CompanyID == companyID
                 select c.PromoCodes;


if (codes.Count() > 0)
            {
               //--get the freetrial from result set
                var freeTrial = codes.Where(c => c.CodeType == (int)CodeTypeEnum.FreeTrial).SingleOrDefault();

              //--get list prepaid from result set
                var prePaids = codes.Where(c => c.CodeType == (int)CodeTypeEnum.PrePaid);


      }

What I didn’t expect was that the second refining, which I meant to happen to the result set in memory, actually spawned another query in the database.  Once I realized this it made sense, since the DataProvider for the IQueryable I was working with was the LLBL Gen Pro Implementation, and it is supposed to query against the database.   To do what I intended I converted the result set to a List before doing my refining which allowed the DataProvider to be the DataProvider underpinning LinqToObjects.  This then did what I wanted and queried against the data in memory.


//--query against IQueryable that is LLBL Entity
var codesFromDB = from c in companyQ
                 where c.CompanyID == companyID
                 select c.PromoCodes;

//--convert to list based Queryable so 
//   further refining done in memory

var codes = codesFromDB.ToList().AsQueryable();

if (codes.Count() > 0)
            {
               //--get the freetrial from result set
                var freeTrial = codes.Where(c => c.CodeType == (int)CodeTypeEnum.FreeTrial).SingleOrDefault();

              //--get list prepaid from result set
                var prePaids = codes.Where(c => c.CodeType == (int)CodeTypeEnum.PrePaid);


      }

The lesson I learned here was keep in mind who owns an IQueryable or IEnumerable when using it, or you might end up with some unexpected side effects.

Handling AJAX Page Method Server Errors

On my project we make use of jQuery to make AJAX calls to our web server. On the server we use aspnet page methods to serve data to our jQuery AJAX calls and process updates sent from jQuery. One interesting aspect of this is error handling.

Since we want to avoid any leakage of application information in the case of an error, we’ve taken the approach of putting a try/catch block around all functionality in a page method. In the case of an error we log the original error and then throw a friendly exception so the jQuery AJAX caller knows something has gone wrong, but get no exception detail. Here’s an example:

[WebMethod]
        public static string SaveCustomerInformation(string customerID, 
                                                string customerCode, 
                                                string reasonForCall, 
                                                string description)
        {
            string returnValue = string.Empty;
            try
            {
                bool isValidcustomerCode = true;
                customerCode = customerCode.Trim();
                
                //--validate input data
                if (!string.IsNullOrEmpty(customerCode) && (customerCode.Length < 3 || customerCode.Length > 15))
                {
                   throw new WebMethodException(Content.ReusableContentHelper.customer_CODE_LENGTH_ERROR);
                }
                else if (!string.IsNullOrEmpty(customerCode) && customerCode.ToLower().Equals(customerCodeLoginHelper.SERVICE_REQUEST_customer_CODE))
                {
                    throw new WebMethodException(Content.ReusableContentHelper.customer_CODE_CANNOT_USE_REQUESTED_ERROR);
                }
                //--End Validation
                
                //--Do Work
                CustomerImpl customerImpl = new customerImpl()
                {
                    customerID = int.Parse(JavaScriptHelper.DecryptValueAndHtmlDecode(customerID)),
                    customerCode = customerCode,
                    customerName = reasonForCall,
                    customerDescription = description
                };

                Factory.GetCustomerServices().SavecustomerInformation(customerImpl, currentUser);
                //--End Work                
                
            }
           catch (Exception e)
          {
               //--Log Exception
                Logger().LogException(e, ErrorLogImpl.ApplicationTypeEnum.Website, RidgeTool.BusinessInterfaces.Enums.SeverityLevelEnum.Error, HttpContext.Current.Request.Url.PathAndQuery, "customer.SavecustomerInformation()", currentUser);
               //--Throw Friendly To Caller
                throw new WebMethodException("An error occured in SaveCustomerInformation");
          }
          return returnValue;
        }

In our javascript we wrap all AJAX calls in a common method allowing callers to provide a method to handle processing if an error occurs, and if no error handler is passed then an common error handling routine is run showing an alert box telling the user a problem has occurred. Here’s what our jQuery AJAX call wrapper looks like.

[javascript]
function AJAXCall(url, data, successCallBackFunction, errorCallBackFunction, completeCallBackFunction)
{
data = JSON.stringify(data, UndefinedValueCheck);

$.ajax({
type: "POST",
url: url,
data: data,
contentType: "application/json; charset=utf-8",
dataType: "json",
error: function(xhr, status, exception)
{
var err = eval("(" + xhr.responseText + ")");
if (errorCallBackFunction != null)
{
errorCallBackFunction(err.Message);
}
else
{
alert(err.Message);
}
},
success: function(result)
{
if(successCallBackFunction != null)
{
//–eliminates .d from return values
if (result.d != null)
{
successCallBackFunction(result.d);
}
else
{
successCallBackFunction(result);
}
}
},
complete: function()
{
if(completeCallBackFunction != null)
{
completeCallBackFunction();
}
}
});
}
[/javascript]

As you can see the user can choose to provide or not provide an error handling function and if they don’t user still is alerted a problem has occurred.

Here’s an example of our jQuery AJAX wrapper in use:

[javascript]
var text = $("#testingJavascriptTextBox").val();
var passedData = "{‘testString’: ‘" + replaceForJavascript(text) + "’}";
var postData = new Object();
postData.testString = text;

AJAXCall("/WebMethods/Tags.aspx/TestingReplace", postData, function(textInput)
{
alert(textInput.d);
$("#testingJavascriptSpan").append(replaceForHTML(textInput.d));
$("#resultsJavascriptTextBox").attr("value", textInput.d);
});
[/javascript]

What all this means is that we protect ourselves form leaking application details in errors by never allowing a naked error to pass through an AJAX call to our web clients. We still alert our AJAX caller to the existence of an error, so we can still know on the rare occasion (hopefully) when one happens!

Dependency Injection And Javascript

As we move towards a more unit tested and designed approach to our javascript code we’ve been playing with the notion of using dependency injection to allow us to abstract and allow mocking of javascript AJAX server calls. What we’ve done is create a javascript object that handles making AJAX server calls. The AJAX server call object has one method that takes a url, data to post, a success function handler, and an error function handler.

[javascript]
//definition of our real AJAX Server Caller
function AJAXServerCaller()
{
//this created public method that can be called by our code
this.MakeServerCall = function(url, data, successHandler, errorHandler)
{
var dataCleaned = JSON.stringify(data);
$.ajax({
type: "POST",
url: url,
data: dataCleaned,
contentType: "application/json; charset=utf-8",
dataType: "json",
success: successHandler(result.d),
error:function (xhr, ajaxOptions, thrownError){
if(errorHandler != null)
errorHandler(thrownError);
else
alert(‘An Error Has Occurred’);
}
});

}

}
[/javascript]

We house our server caller object in a AJAXServerCaller.js file, and this file is included when any calls are made that use the object to make an AJAX server call in their javascript. In order to make an AJAX Server call an instance of the AJAX server calling object is passed in to a method as a parameter.
[javascript]
//example usage of ajax server caller

//function to handle successful return of AJAX call
function UpdateCustomerSaved(serverData)
{
///…do something to UI after ajax call completes successfully

}

//function to handle error result of AJAX call
function DisplayErrorCustomerSave(errorData)
{
///…do something to UI to notify of error
}

function SaveCustomerData(customerData, ajaxServerCaller)
{
//acutual AJAX processing handed off to our server caller object
ajaxServerCaller.MakeServerCall(‘http://portal/customerSave.aspx?SaveCustomer’, customerData, UpdateCustomerSaved, DisplayErrorCustomerSave);
}

///Click Handler calls save method, creates and passes in server caller instance
function CustomerSaveClickHandler(event)
{
var data = GetCustomerData();
SaveCustomerData(data, new AJAXServerCaller());
}
[/javascript]

When we’re testing any javascript that uses our AJAX server call object we simply include a file called ‘AJAXServerCallerMock.js’ that has an object that has the exact same signature as our real AJAX server calling object, but has an extra property for fake data to be returned by a service call. When testing the instance that is created is now of our mock AJAX server caller and javascript logic that relies on server calls can now be tested without actually hitting a server.

[javascript]
//definition of our mock AJAX Server Caller, must have same object name and
//also the MakeServerCall method defined to act in place of real object
function AJAXServerCaller()
{
//properties to hold fake data to return
//and indexes so can do multiple calls in one test
this.SuccessResultDataArray = new Array();
var usedMockSuccessResultIndex = 0;

//we’ll allow an error to be programmed on a specific call
this.ErrorOnCall = 0;
this.ErrorResultData = null;
var totalCallCount = 0;

//this created public method will fake real server call, pass back preprogrammed data.
this.MakeServerCall = function(url, data, successHandler, errorHandler)
{
totalCallCount++;
// act as if error returned if programmed to do so
if(this.ErrorOnCall == totalCallCount)
{
if(this.SuccessResultDataArray.length<=usedMockSuccessResultIndex)
{
errorHandler(this.ErrorResultData);
}
}
//call success method with data matched to index of call
//and increment method call index.
else
{
if(this.SuccessResultDataArray.length<=usedMockSuccessResultIndex)
{
successHandler(this.SuccessResultDataArray[usedMockSuccessResultIndex]);
usedMockSuccessResultIndex++;
}
else
{
throw ‘no success data available at index:’ + usedMockSuccessResultIndex;
}

}
});

}

}
[/javascript]

One issue with our approach is that our mock AJAX server call in testing is not exactly like the real AJAX call would be since the mock object is not asynchronous. Our mock caller simply fires the functions passed to handle the ajax server result with the programmed mocked return data. This hasn’t been a large issue, but might be problematic down the line. Another issue could also arise if we need to support complex return scenarios with our mock, like multiple return values, or checking the number of calls made. We will have to build a more complicated mock AJAX caller if we run into situations that require more complex mock object programming.

[javascript]
//example testing using QUnit taking advantage of ajax server caller

var glbSuccess = false;

//function to handle successful return of AJAX call
function UpdateCustomerSaved(serverData)
{
glbSuccess = true;

}

test("Test Save Customer Data calls server", function() {

// — if AJAXServerCallerMock.js is included for test this should be our mock
var ajaxCaller = new AJAXServerCaller();

//– program mock return value for one expected call
ajaxCaller.SuccessResultDataArray[0] = "{result:0}";

//–Act
SaveCustomerData(data, ajaxCaller);

//–Assert – check everything ok with ui.
ok(methodToCheckCustomerInGoodState());
});

[/javascript]

Between unit testing and adding dependency injection into our javascript it’s already beginning to look and feel a lot more organized. My hope is this will allow us to produce higher quality javascript code, in less overall time. We shall see!