September 2009 – BinzBlog

Using LINQ Expressions For Dynamic Predicates

I ‘ve recently been building a set of reports that all output data in a common format, and also group their data in a common format. Each report needs to be able to have a date predicate based on day, week or month. Using LINQ to LLBL Gen Pro (the ORM tool I’m currently using) I set up a base report class that has abstract methods defining the date predicates and query bodies, but share the grouping and output mechanisms. What allows this is being able to return a LINQ Expression in the methods that define the date predicates.

The abstract date predicates end up looking like the below:

protected abstract Expression<Func<T, bool>> GetDayDatePredicate(int reportBuckets);

Once implemented the date predicate ends up being something like:

protected overrides Expression<Func<dataEntity>,bool>> GetDayDatePredicate(int reportBuckets)

{

return d => d.CreatedDate.Date > DateTime.Now.AddDays(-1 * reportBuckets).Date;

}

Having the predicates in method form serves two purposes: first the method name clearly states what the predicate is doing, (fairly straightforward above but bucketing by weeks is not nearly as clear), and second it allows the base class to create the query by doing something like the below:

public void RunQuery(int numberOfBuckets, IQueryable<T> q)
{

Expression<Func<T, bool>> datePred = GetDayDatePredicate(numberOfBuckets);
q = q.where(datePred);
RunQuery(q);

}

This allows a base class to manage running several reports if they all have similar structure. When adding a report I just add a new implementation of the base class and fill in the abstract bits and then I know I have a report that our UI can use, since it relies on the common output the base class enforces on its children. Don’t know if it’s all that good a solution, we’ll see.

LINQ To LLBL Group By – Two Queries To Get Count

For a persistence layer I’ve been using an ORM tool called LLBL GenPro. I’ve chosen this tool up until now because LINQ to SQL or DLINQ or LINQ to Entities, as it’s called now, seemed like it was very prone to change. I decided to wait until it settled and may begin to use it soon.

In the meanwhile LLBLGen Pro put out a LINQ implementation of their own, so I’ve been happily using LINQ with my ORM tool. I’ve been having some interesting Group By issues, however.

The main problem I’ve had is when trying to use a function like count, or sum. When I write the below I get a sql error :

var customerOrderByDate = from c in Customer
join 0 in Order on c.CustomerID equals o.CustomerID into fullRow
from x in fullRow.DefaultIfEmpty() //forces left join
group x by new { x.CustomerID, x.CustomerName} into ordersGroup
select new { ordersGroup.Key.CustomerID, ordersGroup.Key.CustomerName, ordersGroup.Count(x.OrderDate != null) };

Oddly enough I can make the above create valid sql by separating the LINQ query that creates the data to group and the grouping itself into two separate LINQ queries. Even though the below is two LINQ queries it produces one sql query which does what I want.

var customerOrderByDateData = from c in Customer
join 0 in Order on c.CustomerID equals o.CustomerID into fullRow
from x in fullRow.DefaultIfEmpty() //forces left join

var customerOrderByDateGroup = from data in customerOrderByDateData
group data by new {data.CustomerID, data.OrderDate.Date } in ordersGroup
select new { ordersGroup.Key.CustomerID, ordersGroup.Key.CustomerName, ordersGroup.Count(og.OrderDate != null) };

I would expect the first query to function the same as the the two split out but they don’t. Probably an issue with the LINQ to LLBL implementation. I’m sure that was very difficult to do and I’m happy it works as well as it does.

One other problem I’ve noticed is that the boolean lambda expression in the Count method is required, but it doesn’t seem to do anything. The SQL always produces Count(*) regardless of what is in the lambda expression and Count() doesn’t compile.

Changing To A More Focused Approach

I remember when I first really understood mock objects. I was working as a developer at Progressive Insurance and all of a sudden it hit me that I could test a method without having to test the objects it called in its work. This seems like an obvious statement, but it takes a little time to sink in to a brain such as mine. Once I got my head wrapped around the mocked object concept I was the proverbial hammer lover looking for nails to pound. What this led to were long, serpentine, and very brittle tests. These tests looked like a negative of the method they were testing, every nook and cranny of the method behavior was brought out, in each test. I was happy since I was able to mock like there was no tomorrow, but something was not right.

Take the example method below.

What I have traditionally done is written a few tests that have names like ‘ProcessOrdersTest’ and run down one full logical path through the entire method. It might have looked something like the below:

The problem with the above is that if anything changes in the method the test breaks. This is good, since I want the test to break, but I don’t know what caused the test to break, since the test is testing so many things. Also, if I have two or three methods like the above to test all code paths then one change can break all the tests. I’ll have perhaps three broken tests without any idea what is breaking them.

What I’ve started to do to mitigate this is to write tests that test one particular thing in a method and ignore everything else. Instead of testing a whole code path what I’ve started doing is making sure my test name states one condition to test, and then I try and test that one condition and relying on as little else in the method staying the same. What that leads to is something like the below, and it seems to be working much better. When a test breaks I have more of an idea as to what has gone wrong, and when I purposefully change things I have fewer tests to update.

I Think I Applied SRP Today?

I came across batch job code similar to the below ( the below is just for an example and far simpler than reality!) that was written to update orders in a database from an xml response from a payment system. The code should do the following:

Load the response into an object we can read the data from.
Get a list of ‘batches’ returned in the data.
Update the batches as received in our database.
Get a list of orders for each batch and update them accordingly in our database.

The original code does what it’s supposed to do but I wished to put unit tests around it. What I found was it was difficult to do because the entry point for my mocks was at the highest level. The class was also doing three things, getting and parsing the payment response, updating a batch in our system and updating orders in our system. If I wasn’t careful tests I wrote to tests that orders are updated correctly could break because of changes to how batches are updated, not good. Continue reading “I Think I Applied SRP Today?”

Functional Requirements?

I’ve struggled quite a bit with trying to define what are the appropriate level of requirements necessary to begin coding an application. Back in my days at Andersen Consulting their methodology dictated a very formal process, the deep V. As you went down into the V requirements started off with a problem statement, then a scope statement, then functional requirements that were translated into a technical design and at the bottom of the V, implementation occurred based on the technical design. Going back up the other side of the V were testing steps meant to test the documents at their level on the other side of the V.

The Andersen waterfall looked great on paper and off I went on my projects comforted that the V would lead to straightforward project design, implementation and testing. What I found was this did not occur. Maybe if requirements had not changed the waterfall would have worked, and maybe it did on many projects were this was the case. On the projects I worked on, however, this wasn’t the case.

The situation I encountered often back then, and still today, is that requirements fluctuate. Not only do requirements flucatuate but it’s only after implementation begins that they really begin to fluctuate. I would find myself frustrated, pointing a the V diagram, and protesting that these issues should have already been decided. Slowly I have started to realize that many decisions are not made until implementation begins because it is only then that stake holders start to see their system taking shape. They rethink decisions they made before they had an actual UI to play with or data on reports. Based on actual things to look at stake holders refine their thinking and even out right change their minds. What I have come to accept is that this is how the process really works with any sort of application that is heavily UI driven. Agile development processes have taken this to heart, and I’m sure if Andersen Consulting still existed today (actually it does it’s called Accenture now) they’re process would reflect a far more iterative process that more readily allows for requirement changes.

The one tool that I have embraced that helps to reign in fluctuating requirements for applications that are heavily UI driven is storyboarding. Continue reading “Functional Requirements?”