Category: Coding

Decoupling For Unit Testing: Why And How

 

Abstracting Dependencies:

In order to allow code that has dependencies to be tested at the method level, the code must be architected in a manner that allows those dependencies to be replaced and mocked during method level testing.    One example of this is the below class that has one method that needs to print something as part of its processing.  In order to test the method we do not want to actually have to print something as this means to test the method we would need to have a printer setup and operating somewhere every time we tested our method.

The below versions of our example class does not allow for dependency free testing.  In the example the two dependencies it relies on, the repository and printer, are created in the method itself and we cannot exercise the code in this method without accessing a real repository and printer.

public class PersonDetailManager
{    
   public void PrintPersonDetails(int personId)   
      {
          PersonRepository repo = new PersonRepository();        
          Person person = repo.GetPerson(personId);       
             
          StringBuilder personStringToPrint = new StringBuilder();  
  
          personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);       
          personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);      
          personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);       
          personStringToPrint.Append("Address: " + person.Address + Environment.NewLine);          
 
         Printer printer = new Printer();         

         printer.Print(personStringToPrint);    
       }
}

 

The above code cannot be tested unless the entire system can run from end to end.  We should always create functional or integration tests to ensure our entire system functions correctly end to end.  The above method will not be under test until both the person repository and printer objects exist and are configured to run end to end.  By definition this means that if the above code is written before either the printer or person repository is finished, it will not be able to be tested at the time of its completion.  It could be a significant amount of time between when the above code is completed and when it is even possible to test it using this approach.

In order to make the method code itself testable without relying on its concrete dependencies we need to allow the dependencies to be mocked.  To do this two things must happen, first the dependencies have to be delivered as an abstraction.

Delivering  dependencies as an abstraction means they must be provided to our method as either an interface or a base class that marks the methods we use as virtual so that a fake implementation can be created and fake the behavior we wish to test.  Using interfaces makes sure that any fake dependencies we provide during testing will be replaceable, base classes can have implementations that can cause problems, so we prefer using interfaces.

public class PersonDetailManager
{    
   public void PrintPersonDetails(int personId)    
   {        
      IPersonRepository repo = new PersonRepository();        
      Person person = repo.GetPerson(personId);                    
     
      StringBuilder personStringToPrint = new StringBuilder();       
     
      personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);     
      personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);      
      personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);       
      personStringToPrint.Append("Address: " + person.Address + Environment.NewLine);       
     
      IPrinter printer = new Printer();   
      
      printer.Print(personStringToPrint);     
   }
}

 

The second thing that must happen for our method code to be tested on its own is the dependencies must be provided in such a way that fake dependencies can be used when testing.  This means that we cannot ‘new up’ a dependency in our code as we are doing in the first example as this makes the dependency impossible to replace.

There are several means of delivering dependencies in a manner that allows them to be replaced.  The first is simply to pass them in to our method as arguments:

public class PersonDetailManager
{    
   public void PrintPersonDetails(int personId,
                                  IPersonRepository repo,
                                  IPrinter printer)    
   {       
      Person person = repo.GetPerson(personId);                    

      StringBuilder personStringToPrint = new StringBuilder();

      personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);
      personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);
      personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);
      personStringToPrint.Append("Address: " + person.Address + Environment.NewLine);
      
      printer.Print(personStringToPrint);     
   }
}

 

The above strategy allows us to provide fake implementations of our dependencies at run time so we can now test our method code regardless of the state of our dependencies.  We may not want our callers to have to know about the dependencies so we could always create an overload that provides the real dependencies if we want our callers to be unaware:

public class PersonDetailManager
{    
   public void PrintPersonDetails(int personId)   
   {        
      PrintePersonDetails(personId, new  PersonRepository(), new  Printer());   
   }       

   public void PrintPersonDetails(int personId,
                                  IPersonRepository repo,
                                  IPrinter printer)    
   {       
      Person person = repo.GetPerson(personId);                    

      StringBuilder personStringToPrint = new StringBuilder();

      personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);
      personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);
      personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);
      personStringToPrint.Append("Address: " + person.Address + Environment.NewLine);
      
      printer.Print(personStringToPrint);     
   }

 

The downside to the above is that we need to know what the actual concrete dependencies are when we are building our code in order to create the overload method that provides them.  The IPrinter interface may be defined, a matter of fact it needs to be, for us to build our class, but there is no guarantee when we build our code that the implementations of IPrinter will exist or be known.  In order to allow for this we can deliver our dependencies through a factory.  In this way for us to build and test our method code only the factory needs to exist at the time we are writing our code:

public class PersonDetailManager
{     
   public void PrintPersonDetails(int personId)    
   {       
      IPersonRepository repo = DependencyFactory().Get<IPersonRepositor>();
      Person person = repo.GetPerson(personId);

      StringBuilder personStringToPrint = new StringBuilder(); 

      personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);
      personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);
      personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);
      personStringToPrint.Append("Address: " + person.Address + Environment.NewLine);
 
      IPrinter printer = DependencyFactory().Get<IPrinter>();

      printer.Print(personStringToPrint);    
   }
}

 

As long as our factory exists when we are building our method code, and can be changed in testing to return fake implementations of our dependencies, the factory pattern will allow us to deliver the implementations of our dependencies in a decoupled fashion.  This will allow for testing just our method code discreetly.

Delivering Dependencies:

Above we abstracted our dependencies and used a factory pattern to deliver instances of our dependencies in a decoupled manner.  Our class will never know what the actual implementation of IPrinter is, it just knows the factory will provide one when we need it.  In order to use a factory pattern we have to build and maintain our factory.  We need to have mappings from abstractions to implementations at run time that allow the correct implementation to be delivered.

We can build our factory by hand, but it turns out there are in existence a number of third party tools called Inversion of Control containers, ‘Ioc Containers’ for short, that do essentially this already.   Ioc Containers allow you to register implementations of abstractions and then have the registered implementations delivered when an abstraction is requested.  All that has to happen is for implementations to be registered as the implementer of an abstraction.

Microsoft’s Ioc Container is called ‘Unity’ and to register implementations for our example we would  run something similar to the following code somewhere in the startup of our application:

var container = new UnityContainer();
container.RegisterType<IPrinter, Printer>();
container.RegisterType<IPersonRepository, PersonRepository>();

 

The above code tells our inversion of control container, Unity in this case, that if an IPrinter is requested, return a Printer instance to satisfy the request.  In our code instead of our factory we can reference the container to resolve our dependency.  Using the container directly in your code to resolve dependencies is called the service locator pattern.  It is called this because you are locating the service you need at the time you need it.  One downside to this is that the dependency is not easily seen unless you look at the method’s implementation.  Looking at the class definition itself you would never know the class requires an IPrinter and an IPersonRepository to operate.

 

public class PersonDetailManager
{     
   public void PrintPersonDetails(int personId)
   {
      var container = new UnityContainer();

      IPersonRepository repo = container.Resolve<IPersonRepository>();
      Person person = repo.GetPerson(personId);
      StringBuilder personStringToPrint = new StringBuilder();

      personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);
      personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);
      personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);
      personStringToPrint.Append("Address: " + person.Address + Environment.NewLine);

     IPrinter printer = container.Resolve<IPrinter>();
     printer.Print(personStringToPrint);
  }
}

 

We can write test code that registers fake implementations so our method code can be exercised without any real implementations yet existing.  We can create our own test classes that implement IPrinter and IPersonRepository and register them in our container for testing.  Our fakes might hard code return values and just save values sent to them for us to inspect in the asserts of our tests.  A sample test could look like the below:

[TestFixture]
    public class PersonalDetailManagerTests
    {
        class FakePersonRepository : IPersonRepository
        {
            internal static Person FakePersonReturned;
 
            public Person GetPerson(int personId)
            {
                return FakePersonReturned;
 
            }
        }
 
        class FakePrinter : IPrinter
        {
            internal static StringBuilder BuilderSentToPrint;
 
            public void Print(StringBuilder personStringToPrint)
            {
                BuilderSentToPrint = personStringToPrint;
            }
        }
 
        [TestFixtureSetUp]
        public void SetUp()
        {
            var container = new UnityContainer();
            container.RegisterType<IPrinter, FakePrinter>();
            container.RegisterType<IPersonRepository, FakePersonRepository>();
        }
 
        [Test]
        public void First_Name_First_Line_Of_Print_Test()
        {
            //--Arrange
            int personId = 33;
            FakePersonRepository.FakePersonReturned = new Person
                {
                    FirstName = "firsty",
                };
 
            var itemToTest = new PersonDetailManager();
 
            //--Act
            itemToTest.PrintPersonDetails(personId);
 
            //--Assert
            Assert.IsTrue(FakePrinter.BuilderSentToPrint.ToString().StartsWith("First Name: " + FakePersonRepository.FakePersonReturned.FirstName));
        }

 

 

The service locator relies on our code using the container as a dependency to acquire the dependencies it requires.  This means that our method needs to have an actual container to work, so we are coupled to a container, but at least only to this.  This is why in the above test code as a setup we have to register our fakes with the container, because our code actually uses the container to get the dependencies.  Also, as we stated before, there is no way of knowing what this class is dependent on without looking through the methods themselves.

In order to resolve the issues of being coupled to a container and not having visibility into what dependencies a class has most Ioc Containers implement a feature called constructor injection.  Constructor injection puts the dependencies for  a class discreetly in the constructor, removing the dependency to the container itself, and making it clear to any user of the class what dependencies the class has.

 

Constructor Injection:

Instead of asking the container for our dependencies we can change our class so that any dependencies are taken in our class’s constructor.  The dependencies are then stored as local private variables.  Our class would be changed to the below:

 

public class PersonDetailManager
{
  IPersonRepository _repository;
  IPrinter _printer;

  public PersonDetailManager(IPersonRepository repository,
                             IPrinter printer)
    {
         _repository = repository;
         _printer = printer;
    }

    public void PrintPersonDetails(int personId)
    {

       Person person = _repository.GetPerson(personId);           
 
       StringBuilder personStringToPrint = new StringBuilder();
 
       personStringToPrint.Append("First Name: " + person.FirstName + Environment.NewLine);
       personStringToPrint.Append("Last Name: " + person.LastName + Environment.NewLine);
       personStringToPrint.Append("Phone: " + person.Phone + Environment.NewLine);
       personStringToPrint.Append("Address: " + person.Address + Environment.NewLine); 

       _printer.Print(personStringToPrint);
 
    }
}

 

Now we can give our dependencies to our class directly so our test code can be changed to eliminate the need to interact with the container at all in our tests:

[TestFixture]
    public class PersonalDetailManagerTests
    {
        class FakePersonRepository : IPersonRepository
        {
            internal static Person FakePersonReturned;
 
            public Person GetPerson(int personId)
            {
                return FakePersonReturned;
 
            }
        }
 
        class FakePrinter : IPrinter
        {
            internal static StringBuilder BuilderSentToPrint;
 
            public void Print(StringBuilder personStringToPrint)
            {
                BuilderSentToPrint = personStringToPrint;
            }
        }
 
        [Test]
        public void First_Name_First_Line_Of_Print_Test()
        {
            //--Arrange
            int personId = 33;
            FakePersonRepository.FakePersonReturned = new Person
                {
                    FirstName = "firsty",
                };
 
            var itemToTest = new PersonDetailManager(new FakePersonRepository(),
					             new FakePrinter());
 
            //--Act
            itemToTest.PrintPersonDetails(personId);
 
            //--Assert
            Assert.IsTrue(FakePrinter.BuilderSentToPrint.ToString().StartsWith("First Name: " + FakePersonRepository.FakePersonReturned.FirstName));
        }

The above tests and code work, but it would appear anyone who wants to use our code would have to pass in the actual implementations of our dependencies, as we have in our test. Most Ioc containers, however, have a feature where they will provide dependencies in the constructors of instances they produce if the dependencies are registered with the container.  What this means is that if I request a PersonDetailManager from the container and I have registered implementations for the dependencies PersonDetailManager needs the container will automatically provide them and pass them as parameters to the PersonDetailManager when it builds it.

Practically this means that in an application I only need to request an instance of an object at the top of a dependency tree and the Ioc Container will handle fulfilling all dependencies listed in the constructors of instances the container provides.  If at the start of my application I have the following registration code:

 

var container = new UnityContainer();
container.RegisterType<IPrinter, Printer>();
container.RegisterType<IPersonRepository, PersonRepository>();
container.RegisterType<PersonDetailManager, PersonDetailManager >();

 

Then later in code I request a PersonDetailManger from the container, the container will automatically deliver the registered instance for IPersonRepository and IPrinter that it will need to construct a PersonDetailManager.

 

var personDetailMangaer = container.Resolve<PersonDetailManager>();

 

The above means that we only need to use the service locator pattern at the top of dependency trees.  Examples of the top of a dependency tree are a ServiceHostFactory in WCF, ControllerFactory in ASP.net MVC or the ObservableFactory in our MVVM framework.   Anything that is consumed by the top level or further down in the tree just needs to be created with its dependencies listed as abstractions in its constructor.

What constructor injection allows you to do is make a clear pattern for how to access dependencies in development, put the dependency as an abstraction in your constructor, and allow the container to manage delivering the dependency at runtime.  This allows the developer to focus on building code as they have a known pattern for how to build code, and also to test as they have a known pattern for how to replace dependencies in test code.   Constructor Injection also allows developers to build and test code without all dependency implementations being fully operational or accessible at the time they are building.  Class dependencies are also explicitly stated as they are all listed in the class constructor.

In our test examples above, we create our own fakes or mock dependencies, but similarly to Ioc Containers there are many mock libraries that exist that already provide rich mocking functionality so we don’t have to roll our own.

 

Mock Objects:

There are many mock libraries for .net, NMock, Moq, RhinoMocks just to name a few.  What these libraries do is allow you to quickly create a mock instance of an interface and program that instance to act as you would like in your test, and also record calls against it so you can ensure the dependency was called as you expect in your code.  Each mock library has slightly different syntax, but each performs the same basic set of behaviors, allowing the programming of a mock instance, and interrogating how the mock instance was called.

Using nMock the below shows how we program a mock instance of our IPersonRepository to return a predefined person, and how we would check to see how our IPrinter mock instance was called as we expect it to be:

[Test]
public void First_Name_First_Line_Of_Print_Test()
{
   //--Arrange
   int personId = 33;
   var fakePerson = new Person
    {
        FirstName = "firsty",
    };

   var mockFactory = new MockFactory();

   Mock<IPersonRepository> mockPersonRepository = mockFactory.CreateMock<IPersonRepository>();
   Mock<IPrinter> mockPrinter = mockFactory.CreateMock<IPrinter>();
 
   //--program mock to return fake person if personId passed
   mockPersonRepository.Stub.Out.MethodWith(x => x.GetPerson(personId)).WillReturn(fakePerson);
            
   //--program mock to expect first name first line of stringbuilder passed in
   mockPrinter.Expects.AtMostOnce.Method(x => x.Print(new StringBuilder()))
	.With(NMock.Is.Match<StringBuilder>(sb => sb.ToString().StartsWith("First Name: " + fakePerson.FirstName)));
 
   var itemToTest = 
	new PersonDetailManager(mockPersonRepository.MockObject, mockPrinter.MockObject);
 
   //--Act
   itemToTest.PrintPersonDetails(personId);
 
   //--Assert
   //--make sure expectations met, will enforce expectations, but not stub calls.
   mockFactory.VerifyAllExpectationsHaveBeenMet();
}

 

The syntax above would be similar for Moq or RhinoMocks and would achieve the same purpose.  The below is the syntax for RhinoMocks.  What I like better about Rhinomocks and Moq is I can make my verifications explicit and place them at the end after I run my test instead of setting them before in expectations.  This is called the Arrange, Act, Assert or AAA testing patter:

[Test]
public void First_Name_First_Line_Of_Print_Test()
{
//--Arrange
int personId = 33;
var fakePerson = new Person
    {
        FirstName = "firsty",
    };
 
IPersonRepository mockPersonRepository = MockRepository.GenerateMock<IPersonRepository>();
IPrinter mockPrinter = MockRepository.GenerateMock<IPrinter>();
 
//--program mock to return fake person if personId passed
mockPersonRepository.Expect(x => x.GetPerson(personId)).Return(fakePerson);
            
 
var itemToTest = new PersonDetailManager(mockPersonRepository, mockPrinter);
 
//--Act
itemToTest.PrintPersonDetails(personId);
 
//--Assert
//--program mock to expect first name first line of stringbuilder passed in.
mockPrinter.AssertWasCalled(x => x.Print(Arg<StringBuilder>.Matches(sb => sb.ToString().StartsWith("First Name: " + fakePerson.FirstName))));
}
}

You can also use AAA syntax in Moq as below:

[Test]
public void First_Name_First_Line_Of_Print_Test()
{
//--Arrange
int personId = 33;
var fakePerson = new Person
    {
        FirstName = "firsty",
    };
 
    Mock<IPersonRepository> mockPersonRepository = new Mock<IPersonRepository>();
    Mock<IPrinter> mockPrinter = new Mock<IPrinter>();
 
//--program mock to return fake person if personId passed
mockPersonRepository.Setup(x => x.GetPerson(personId)).Returns(fakePerson);
            
 
var itemToTest = new PersonaDetailManager(mockPersonRepository.Object, mockPrinter.Object);
 
//--Act
itemToTest.PrintPersonDetails(personId);
 
//--Assert
//--program mock to expect first name first line of stringbuilder passed in.
mockPrinter.Verify(x => x.Print(It.Is<StringBuilder>(sb => sb.ToString().StartsWith("First Name: " + fakePerson.FirstName))));
}

 

AutoMocking:

 

What you may have noticed in both my nMock and RhinoMocks tests was that I had to explicitly declare my mock objects, this can get tedious if you multiple dependencies and many classes you are testing.  Automocking is a feature that some Ioc Containers provide that automatically creates mock instances of all dependencies on a class when you ask for the class under test.  Doing this removes the need for the developer to add code to create each mock dependency the class under test will require.

Below is how our test would look using an Ioc named StructureMap with RhinoMocks and  automocking.  Notice the mocked dependencies exist on the class under test by default now:

[Test]
public void First_Name_First_Line_Of_Print_Test()
{
//--Arrange
int personId = 33;
var fakePerson = new Person
    {
        FirstName = "firsty",
    };
 
var autoMockedItem = new RhinoAutoMocker<PersonDetailManager>();
 
//--program mock to return fack person if personId passed
autoMockedItem.Get<IPersonRepository>().Expect(x => x.GetPerson(personId)).Return(fakePerson);
 
    
//--Act
autoMockedItem.ClassUnderTest.PrintPersonDetails(personId);
 
//--Assert
//--program mock to expect first name first line of stringbuilder passed in.
autoMockedItem.Get<IPrinter>().AssertWasCalled(x => x.Print(Arg<StringBuilder>.Matches(sb => sb.ToString().StartsWith("First Name: " + fakePerson.FirstName))));
}

 

This is the same automocking feature using StructureMap and Moq:

[Test]
public void First_Name_First_Line_Of_Print_Test()
{
    //--Arrange
    int personId = 33;
    var fakePerson = new Person
    {
        FirstName = "firsty",
    };
 
    var autoMockedItem = new MoqAutoMocker<PersonaDetailManager>();
 
    //--program mock to return fack person if personId passed
    autoMockedItem.Get<Mock<IPersonRepository>>().Setup(x => x.GetPerson(personId)).Returns(fakePerson);
 
 
    //--Act
    autoMockedItem.ClassUnderTest.PrintPersonDetails(personId);
 
    //--Assert
    //--program mock to expect first name first line of stringbuilder passed in.
    autoMockedItem.Get<Mock<IPrinter>>().Verify(x => x.Print(It.Is<StringBuilder>(sb => sb.ToString().StartsWith("First Name: " + fakePerson.FirstName))));
}

 

Using automocking above lowers the setup overhead of using multiple dependencies in a class.  Automocking combines a container and a mock object library (above StructureMap and RhinoMocks or Moq), to allow dependencies to be automatically fulfilled by mock objects without having to define each mock object and pass to its constructor.  This is helpful where new dependencies are added so older tests do not have to be updated to add new dependencies in their constructor, automocking will automatically add a non-programmed mock to any test that creates its test class using the automocker.

 

Creating Tests When Creating Code

 

Test Driven Development is the process of writing tests that fail first, then writing code that makes the tests pass.  This ensures that tests exist, that they can fail if the condition they are testing is not met, and that the code written makes the test pass.  It has been documented that this process significantly decreases defect rate without adding significant development time. (http://www.infoq.com/news/2009/03/TDD-Improves-Quality).  Writing tests when code is written ensures that tests exist and that code is architected to be testable.  Whether or not the tests are truly written test first or written as you go along is not that important.  What is important is that the tests are not added later as this increases time as the test writer has to become familiar with the code they are writing tests for, and if any bugs are found any code that has come to rely on the class with the bug may need to also be changed to accommodate any fixes.  Finding bugs long after the code has been writing becomes more costly to fix since there may now be dependencies on the flawed code.

Writing unit tests that exercise method code at the time the method code is written is to ensure that the code does what the developer intends it to do when the developer writes the code.   As a process developers should write method testing unit tests when they write their code and should use constructor injection, mock libraries and automocking to establish efficient patterns to make this testing relatively quick and easy to implement.  Using this architecture also allows developers to write tests regardless of whether there are working or configured instances of the concrete implementations of their dependencies.  This allows development of code and its dependencies to be concurrent, code can be written as long as the signature of a dependency is known, not its full implementation.  Testing at the time of development allows for the possibility to catch bugs at the earliest possible time, and perhaps even stop some from ever existing.

 

Automating Method Tests In Continuous Integration Builds

 

Once method level tests exist they show that method code does what the developer intended it to do.  They also mock dependencies so should run relatively quickly and require no other resources to run.  Method level unit tests can be run without any dependencies actually existing or being reachable.  For this reason it is these tests that should be run on Continuous Integration builds.  By running just these tests builds run quickly, and these test ensure code has not been broken on the method level.

Functional and integration tests should be automated but should run separately and not block Continuous Integration builds as they can be long running and brittle and remain broken for long periods of time, rendering the Continuous Integration build less effective as when it is broken it is more likely due to brittle functional tests than actual code issues and is frequently ignored.

Loving ToDictionary

I love ToDictionary()! It allows me to easily transform disparate data into a common format I can use to send to methods making those methods less coupled and more readily reusable. Let me provide an example.

I have a method that needs to process header values of web request and do certain things based on their values, the signature looks something like this:

public ComService  GetServiceBasedOnSource(IServiceFactory factory, 
                                           HttpRequestHeaders headers, 
                                           string tokenValue)

I was given a requirement to have the same processing occur based of off query string values if headers are not available. I was passing headers directly to my method, however, so wasn’t sure what to do. I figured I could pull out common code and have two methods, one with signature for headers and one with signature for query string parameter values, which is turns out are a collection of KeyValuePair if you use the Request.GetQueryNameValuePairs() method. When thinking about this I realized if I pulled the common code out I’d have to transform the different input into something I could process, so if I was going to alter data why not let caller do it and have one method with a signature that wasn’t dependent on headers or keyvaluepairs?

This is where ToDictionary makes life easy. I changed my signature to take a dictionary instead of headers or keyvaluepairs and toDictionary makes this transformation a snap. My method now looks like:

public ComService  GetServiceBasedOnSource(IServiceFactory factory, 
                                           Dictionary<string, string> dictionary, 
                                           string tokenValue)

And to use it I just ToDictionary the inputs:

public ComService  GetServiceBasedOnSource(factory, 
                                           Request.Headers.ToDictionary(x => x.Key, x => x.Value.FirstOrDefault()),
                                           token);

or 

public ComService  GetServiceBasedOnSource(factory, 
                                           Request.GetQueryNameValuePairs().ToDictionary(x => x.Key, x => x.Value), 
                                           token);

RegEx For Parsing

Recently had a situation where we needed to parse city, state and zip code from an input string that could be in three or four different formats. I was provided with some legacy code that, supposedly, handled most of the cases. The code was in vb.net, already not making me happy, and several hundred lines long.   The code was not accessible to me as a component, so on top of it being suspect I had to copy and paste it to work with it.

The code walked through it’s assumptions and if they weren’t meant it would give back wrong results. It turned out it only handled one input case as well.
Here is just a taste of what I had to work with:

[vb]
Private Sub ParsePlace(ByVal sPlace As String, ByVal data As Location)
Dim sCity As String = String.Empty
Dim sState As String = String.Empty
Dim sCounty As String = String.Empty
Dim iIndex As Integer = 0

iIndex = sPlace.IndexOf(“,”)

If (iIndex > 0) Then ‘ “1 NW Clifton, NJ, Pasaic”
sCity = Left(sPlace, iIndex) ‘ City is left part up to first comma (1 NW Clifton)
sPlace = sPlace.Substring(iIndex + 1, sPlace.Length – iIndex – 1).Trim ‘ Place is from the 1st comma on (NJ, Pasaic)

If (sPlace.Length > 0) Then
iIndex = sPlace.IndexOf(“,”)

If (iIndex > 0) Then
sState = Left(sPlace, iIndex)
ElseIf sPlace.Length = 2 Then
sState = sPlace
End If

‘ We’re not using county yet, but here’s the code if we need it.
‘sPlace = sPlace.Substring(iIndex + 1, sPlace.Length – iIndex).Trim
‘If (sPlace.Length > 0) Then
‘ iIndex = sPlace.IndexOf(“,”, iIndex)
‘ sCounty = Left(sPlace, iIndex)
‘End If
End If
Else
[/vb]

There were many problems with this. First the code is painful to follow, this was just the start of many more lines of finding spaces and substringing. The second major problem is that it doesn’t handle multiple inputs, and making it do so would be painful.

To handle this we rewrote this using regular expressions. The benefit of the expressions, is that we can see if the input discreetly matches the pattern we think it should, and if it does pull off the pieces we want from the regex itself. This allows us to not have to write messy parsing code, and also to throw an error if we find a input format we don’t support as opposed to returning bad results.

The change looks like the below:

            string LocationExactMatch = @"^(0.0)\s+([\d]{5}|[\w]{6})\s+([\w|\s]+),\s*(\w{2})";
            string LocationDistanceTo = @"^(\d+.\d+)\s+\b(N|S|E|W|NE|NW|SE|SW|NNE|NNW|SSE|SSW)\b\s+([\d]{5}|[\w]{6})\s+([\w|\s]+),\s*(\w{2})";
            string LocationNoDistanceTo = @"^([\d]{5}|[\w]{6})\s+([\w|\s]+),\s*(\w{2})";
            string LocationNoZipCode = @"^(\d+.\d+)\s+\b(N|S|E|W|NE|NW|SE|SW|NNE|NNW|SSE|SSW)\b\s+([\w|\s]+),\s*(\w{2})";
            string LocationNoZipCodeExactMatch = @"^(0.0)\s+([\w|\s]+),\s*(\w{2})";

            var cityData = new CityData() { Comments = result };
            var match = Regex.Match(result, LocationDistanceTo);

            if ((match.Success))
            {
                cityData.MilesFromCity = Convert.ToDouble(match.Groups[1].Value);
                cityData.DirectionFromCity = match.Groups[2].Value;
                cityData.ZipCode = match.Groups[3].Value;
                cityData.CityName = match.Groups[4].Value;
                cityData.State = match.Groups[5].Value;
            }

            if(!match.Success &&  (match = Regex.Match(result, LocationExactMatch)).Success)
            {
                cityData.MilesFromCity = Convert.ToDouble(match.Groups[1].Value);
                cityData.ZipCode = match.Groups[2].Value;
                cityData.CityName = match.Groups[3].Value;
                cityData.State = match.Groups[4].Value;
            }

            if (!match.Success && (match = Regex.Match(result, LocationNoDistanceTo)).Success)
            {
                cityData.ZipCode = match.Groups[1].Value;
                cityData.CityName = match.Groups[2].Value;
                cityData.State = match.Groups[3].Value;
            }

            if (!match.Success && (match = Regex.Match(result, LocationNoZipCode)).Success)
            {
                cityData.MilesFromCity = Convert.ToDouble(match.Groups[1].Value);
                cityData.DirectionFromCity = match.Groups[2].Value;
                cityData.CityName = match.Groups[3].Value;
                cityData.State = match.Groups[4].Value;
            }

            if (!match.Success && (match = Regex.Match(result, LocationNoZipCodeExactMatch)).Success)
            {
                cityData.MilesFromCity = Convert.ToDouble(match.Groups[1].Value);
                cityData.CityName = match.Groups[2].Value;
                cityData.State = match.Groups[3].Value;
            }

            if (!match.Success)
            {
                throw new ArgumentException("Input Not Valid Format.");
            }

The above uses the ability of the regex to pull out values by group to do parsing, and errors if input does not match any of our expectations, much better in my book.

Asserting Rest Functional Tests

I’ve been doing a lot of work with Rest services lately. Since I’ve been able to use .Net 4.5 this has meant using WebAPI and, luckily enough, I have been able to put together a nice decoupled, unit tested stack (@500 unit tests 65% code coverage), leveraging dependency injection and WebApi’s more decoupled nature than WCF.

As I’ve been adding functional tests around completed functionality I’ve had to find a quick way in C# to parse our returned json into something that can be understood in C#. At first I started parsing the returned json as a string and looking for particular structures in the string. Looked something like this:

requestUri = new Uri("http://localhost/api/students/34");

//--base test class method to prepare web request
PrepareGetService(requestUri);

//--base test class method to execute web request
var result = GetResponse(requestUri);

Assert.IsTrue(result.Contains("firstName:""John,"), "Bad First Name.");

The above has some obvious problems.

  • FirstName could exist elsewhere in my model
  • The spacing could be different than what I’m expecting in the returned json
  • Firstname could be the last field and have no comma
  • If I take the comma off FirstName could match “Johnny” as well as “John”, etc
  • There is no guarantee this is a model at all, could be just that string that is returned

I did a little googling and found that I can parse a json object into a C# dictionary fairly quickly:

var resultObj = new JavaScriptSerializer().DeserializeObject(result) as Dictionary<string, object>;

Assert.AreEqual("John", resultObj["firstName"], "Bad First Name.");

This is a little better as now I know the exact field and value and can compare them discreetly. It won’t matter where in the json FirstName is, and if the response is not deserializable into a dictionary I probably have some badly formatted json, which is also good to know.

When I realized I could deserialize into a c# object it dawned on me I could just serialize into the c# model, if I have one. Then this would work:

var resultObj = new JavaScriptSerializer().DeserializeObject(result) as Student;

Assert.AreEqual("John", resultObj.FirstName, "Bad First Name.");

The above is great as long as the json was created from serializing a type in my c# code, in this case student. This is not always the case, as sometimes the return objects are being dynamically created as anonymous types:

 public HttpResponseMessage Get(int id)
        {
            IRestContext context = _restContextProvider.GetContext(Request);
           
            Student stuudent = _studentRepository.GetStudent(id);

            return context.CreateResponse(HttpStatusCode.Ok, new { firstName = student.FirstName, lastName = student.LastName, pullDate = DateTime.UtcNow });           
        }

In the above I’ll have no type to deserialize into, but the dictionary should work just fine for this.

In short parsing strings for checking json results in my functional tests had some issues, so went to using explicit types where possible, and a dictionary where getting back anonymous types.

Controller Folder Location Route Constraint

Recently I came across a scenario on my current project where we needed a webApiRoute to only be applicable to controllers in a specific folder. At first I was thinking I have to build a constraint to only allow a match on a route if the controller was in a specific virtual directory. Then it dawned on me the controllers aren’t in any virtual directory at all, they are not deployed, they are compiled into the application!

Lucky for me our folders match our namespace structure. Based on this I was able to create a custom constraint making sure the requested controller had a namespace that ended with the desired folder path. Ends up looking like below:

  public class NameSpaceConstraint : IHttpRouteConstraint
    {
        private string _nameSpace;

        public NameSpaceConstraint(string nameSpace)
        {
            this._nameSpace= nameSpace;
        }

        public bool Match(System.Net.Http.HttpRequestMessage request, IHttpRoute route, string parameterName, IDictionary<string, object> values, HttpRouteDirection routeDirection)
        {
            if (!values.ContainsKey("Controller"))
            {
                return false;
            }
          
            string controllerName = values["Controller"].ToString();

            Type typeOfController = Assembly.GetExecutingAssembly().GetTypes().FirstOrDefault(x => x.Name == controllerName + "Controller");

            return typeOfController != null && typeOfController.Namespace.EndsWith(this._nameSpace, StringComparison.CurrentCultureIgnoreCase);
        }
    }

Works well, just need to define route with constraint in startup and all is well:

  public static void Register(HttpConfiguration config)
        {
            config.Routes.MapHttpRoute(
               name: "FolderName",
               routeTemplate: "api/FolderName/{controller}/{id}",
               defaults: new { id = RouteParameter.Optional },
               constraints: new { NameSpace= new NameSpaceConstraint(".FolderName") }
               );
}

Of course this is assuming that my controller is in the same assembly that is executing my route constraint. If this wasn’t the case I’d have to get a little more crafty looking for types. Not an issue at this point, however.