Programmerdave.com

A couple of weeks ago I sent an email to my team asking them some questions around the yield keyword. Normally when I send these weekly emails I make sure they are challenging but for whatever reason my brain wasn’t working a couple of weeks ago so thought I would give them an easy week.

I was a little surprised to learn that although a couple of the team knew what yield was and how it should be used a number of other team members didn’t, we had members who had not heard of yield before.

This was a little bit of a surprise to me, but given that this was the case I thought it might make a good blog post, This is nothing more than a brief introduction.

For more detail on iterators look at Jon Skeet’s C# in depth. You can get the chapter on iterator for free from manning here

What is yield and why you should care.

Basically yield is just a little syntax sugar, it implements an iterator and returns a single item, on the next call it gets the next item with movenext and returns it.

This is advantageous when you calculate the items in the list or if the list returns an infinite set, example might be a set of primes or random numbers. You can never return the whole set so you return it incrementally.

Random numbers are a good example.

Say we have a method called GetRandomNumbers(int maxCount)

public static IEnumerable GetRandomNumbers(int maxCount)
{
    var rand = new Random();
    for (int i = 0; i < maxCount; i++)
    {
        Thread.Sleep(500);
        yield return rand.Next();
    }
}

I have set the method to sleep for 500ms on each loop to add a little bit of calculation time to the items in the loop.

Next I added another method that returns an IList like this:

public static IList GetRandomNumberList(int maxCount)
{
    var numberList = new List();
    var rand = new Random();
    for (int i = 0; i < maxCount; i++)
    {
        Thread.Sleep(500);
        numberList.Add(rand.Next());
    }
    return numberList;
}

Essentially both methods do the same thing, get N random numbers. The way they do it and the affect that has on your UI and both the perceived and actual performance of your application is striking and something that you should be aware of.

To really see this for yourself run the following console app: https://gist.github.com/david-mclean/5228317

Although the yield example was actually 167ms slower, It looked and felt like it was faster.

This perception of performance and the feedback an application gives the end user is sometimes more important than the application being 167ms faster.

Another example, this time where yield is more performant than list.

This time we will have methods GetRandoms and GetRandomsList : https://gist.github.com/david-mclean/5228693

These methods will each return 5,000,000 random numbers.

In this example I have a need for only 10 random number, so I use the Take extension method.

So my calls look like this:

GetRandoms().Take(10);
GetRandomsList().Take(10);

Now when I run the code I get the following result:

GetRandoms: 1ms
GetRandomsList: 123ms

In this instance GetRandomsList had to get all 5,000,000 results before Take could get the first 10 items. GetRandoms on the other hand only returned 10 items, linq uses yield for a lot of what it does, that is why the return type for a lot of the extensions are IEnumerable.

Should you use yield.

Well that depends on what you are looking to achieve. In cases where you are updating a UI and the background process has to do some intensive work for each item then its worth considering.

This has come up for discussion recently, my usual off the cuff answer of “don’t test privates, use your public api to test what you need” has not really been good enough.

This is my attempt at giving a detailed answer to this often contentious topic, my view of this has changed a couple of times of the last few years but as of today this is how I see it.

The basic rules for me are:
- Don’t test private methods directly
- If you think you need to, chances are you actually need to refactor.
- Don’t just make the private method public.

Something I have been asked a few times recently is “OK so I shouldn’t test private methods directly, but why is that?”.

Good question, the reasons that instantly come to mind for me are:
- Often implementation detail (We are interested that we get the result we are looking for, not interested in how it gets there).
- Makes unit tests fragile (with regard to refactoring).
- Generally agreed as a bad practice (A number of people who’s opinions I respect site it as bad practice, this has a strong impact on my point of view – in short I agree).

To try and give a better explanation I set up a contrived example, in doing so some points became more obvious.

If you need to run a lot of tests for a private method then doing so via a public method can have two negative side effects.

1. Your tests get muddied with lots of test that are actually for another method, this makes navigating and maintaining the tests more work.
2. Setting up your tests for the private method by testing the public method can become tricky.

What I mean is:

1. You will end up with a lot of tests for classx_methodx_{expected result}. The problem with this is when you change classx_methodx with a breaking change, you then need to change all the tests that relate to classx_methodx but you also need to change all the tests that relate to your private method, even though that code has not changed.

2. Chances are your test will Assert on a result. You know that your private method should return 22 with the given inputs, problem is the public method does some other stuff to the value returned from your private method before returning it. Meaning you need to make sure you do the same stuff that your public method does to the result so you can Assert that the value returned is what you expected. I’m sure you can see what a nightmare this could become.

So we are running into the problems described above, we could just make the private method public. This would mean we can test its logic directly and sidestep the two issue outlined. The problem I have with this is the impact it has on the public api, the method was private so it should be safe to assume that a consumer of your library / service / whatever doesn’t need to know about the method. I’m of the opinion that if they shouldn’t need to know about the method, then they shouldn’t see it, making it public will only make your public api harder for others to use.

The way I would get around this would be to refactor the method out as a class.

If the class is going to be of use outside of the application make it public if not then make it internal.

You can test internal classes by adding [assembly:InternalsVisibleTo("Test.Assembly.Namespace")] to the projects AssemblyInfo.

This gives you the separation you really needed, making all your tests more focused, maintainable and readable.

In conclusion, for the most part you will probably not come up against this issue as most private methods should be testable via your public api. If you start writing a lot of tests to test a private method via a public method, it’s probably time to start looking at refactoring the private method out to a class of its own.

I was having one of them dumb days a couple of days back and I couldnt for the life of me workout how to get global filters property injects and simple injector all to play nice together.

Thankfully there is an awesome community over at stack overflow and they came to the rescue (See here)

In the end I have got a solution that actually seems rather elegant.

If you haven’t used single injector then I will quickly explain what the MVC NuGet package adds to your solution.

You get references to the dlls and a folder called App_Start which contains a class SimpleInjectorInitializer.

This class is where you register all your dependancies.

As it comes it has two methods, Initialize and InitializeContainer.

I added a third one RegisterGlobalFilters, yes you have got a method of the same name in your global.asax.cs this one has a different signature though.

public static void RegisterGlobalFilters(GlobalFilterCollection filters, Container container)
{
	//Add simple injector resolved types.
	filters.Add(container.GetInstance<UserAuthorisation>());
}

Now this gets called from inside the Initialize method.

public static void Initialize()
{
	var container = new Container();
	InitializeContainer(container);
	container.RegisterMvcControllers(Assembly.GetExecutingAssembly());
	container.RegisterMvcAttributeFilterProvider();
	container.Verify();
	DependencyResolver.SetResolver(new SimpleInjectorDependencyResolver(container));

	RegisterGlobalFilters(GlobalFilters.Filters, container);
}

A quick note on GlobalFilters.Filters.

This is a static property on a static class with its initial value initialised by the class when it is loaded.

So we can treat it as a singlton because calling GlobalFilters.Filters will return the same GlobalFilterCollection no matter where we call it from.

OK so The full code looks something like this…

public static class SimpleInjectorInitializer
{
	/// <summary>Initialize the container and register it as MVC3 Dependency Resolver.</summary>
	public static void Initialize()
	{
		var container = new Container();
		InitializeContainer(container);
		container.RegisterMvcControllers(Assembly.GetExecutingAssembly());
		container.RegisterMvcAttributeFilterProvider();
		container.Verify();
		DependencyResolver.SetResolver(new SimpleInjectorDependencyResolver(container));

		RegisterGlobalFilters(GlobalFilters.Filters, container);
	}

	private static void InitializeContainer(Container container)
	{
		//Rgistrations here.

	}

	public static void RegisterGlobalFilters(GlobalFilterCollection filters, Container container)
	{
		//Add simple injector resolved types.
		filters.Add(container.GetInstance<Type>());
	}
}

And there we have it.

Now we can setup our types with simple injector and have it inject the properties for us too.

See here for a quick introduction to property injection with simple injector (Coming soon).

Its worth noting that we have added this capability without changing the global.asax.cs.

…Your process does not have access rights to this namespace (see http://go.microsoft.com/fwlink/?LinkId=70353 for details).

So I had a little issue when trying to host a WCF service from a Windows service.

Turns out the fix is pretty simple, I am adding it here as the provided help link wasn’t working for me.

Open a command prompt (make sure you are running as administrator).

Cd c:\windows\system32

netsh http add urlacl url=http://+:80/{yourService}/ user=\Everyone

After running the above command you should get a success message and everything should be working.

The Mighty Singleton.

I have been watching the awesome Jon Skeet over on TekPub in his new series.

The first episode is all about the singleton, let the debate begin?

Not this time, I am going to put my own interpretation of what Mr Skeet has taught me

For me the best way of learning is by doing, so I have been having a play.

What seems to be the “classic” singleton:

public class Singleton
{
	private static Singleton instance;
	//Private constructor to stop instance initialisation.
	Singleton(){}

	public static Instance
	{
		get
		{
			//Check to see if instance is instantiated.
			if(instance == null)
			{
				instance = new Singleton();
			}
			return instance;
		}
	}

	public void SingletonMethod(){}
}

The above is not thread safe though, Its possible for more than one thread to pass the if(instance == null) check. Then we have more than one instance of the singleton #fail!

So lets make it thread safe:

public class Singleton
{
	private static readonly object lock = new object();
	private static Singleton;

	Singleton(){}

	public static Instance
	{
		get
		{
			lock(lock)
			{
				if(instance == null)
				{
					instance = new Singleton();
				}
			}
			return instance;
		}
	}

	public void SingletonMethod(){}
}

Thread safe. There isn’t really anything wrong with the above – people seem to think that locks are expensive so they try to avoid them.

I have not done any bench marking of my own so I cant say for sure how expensive locks are.

An obvious change to try and avoid the lock would be to add another instance check around the lock, so you have:- check, lock, check.

This seems fine, but Mr Skeet saves us from ourselves here. There is an obscure issue around memory barriers that can cause more than one instance to be created.

I will be honest I have no prior knowledge of memory barriers. A little reading tells me that on some environments writes can be re-ordered and other similar things can happen, these strange behaviours can be fixed by specifically telling .net not to do any reordering.

Some links about memory barriers:
Memory Model Memory Barrier and Singleton Pattern
http://en.wikipedia.org/wiki/Memory_barrier

For simplicity I think I would stick with the first Singleton as we know that is thread safe and does what we expect.

Possibly the simplest way to write a singleton would be:

Public class Singleton
{
	private static readonly Singleton instance = new Singleton();

	public static Singleton Instance { get { return instance; } }

	public void SingletonMethod(){}
}

No locks or instance checks, well not by us anyway. This is possibly the safest of all. Even safer than the lock, check implementation as we let the CLR take care of everything for us.

This works because of the way static initialises work in .net. Basically the CLR guarantees that only one thread can be in a static initialiser at any time.

If you need to know that the Singleton will not be initialised until directly before it is used then you can add an empty static constructor.

This touches on the concept of lazy initialisation, which is the act of creating an instance only when you need to use it. There are a couple of ways to write lazy singletons and a general pattern that is useful beyond the scope of singletons.

I will talk about these in my next post.

This is my personal list of every day shortcuts I think .net developers should be aware of.

That is the list of my most commonly used shortcuts, it is hard remembering them all to write a post like this as I know most of them only by the pattern they form on the keyboard.

I have kept almost all of my shortcuts binded to the standard VS Key combinations, this makes sense for me as I work on a few machines. I would consider changing a few of the shortcuts as they can be a bit of stretch.

My favourite shortcuts in each of the sections above are actually pretty easy to pick out, In Unit testing I would choose Ctrl+R, T and Ctrl+R, Ctrl+C. In Search and finding I would choose Ctrl+k, Ctrl+R and F12. In Editing and moving I would go for Ctrl+Space first and Ctrl+. second.

There are a lot more shortcuts that I use like tabbing back and forth between open tabs, there are so many more shortcuts, most of which I have never used and a few I should probably learn. I think the key is finding the shortcuts that allow you to keep your hands on the keyboard and as productive as possible.

Now I have streamlined my use of my IDE I think next is to streamline my use of my brain!

If you think I am missing a must use shortcut please share so we can all learn.

This has grown out of the need to extend a legacy system in a way that will amend the way some existing code works.

This code is a little intertwined in places, this makes it very hard to test but also pretty tricky to work out what the tests should be.

The first problem I had was trying to work out where to start.

To get around this I started mapping out the flow of the code, I found three well worn paths.

Now I had these I was able to start working through them and finding all the relevant dependencies that also needed to be tested.

I could have just done a few overview integration tests but I wasn’t confident that level of testing would catch possible breaking changes from the new work and inevitable refactoring.

Breaking down the flow into a few well worn paths gave me visibility of the code that was doing the meat of the work, and the code that it depended on, creating unit tests for this code gave me coverage I felt confident in.

This approach has worked reasonably well.

The next issue was how to actually unit test this code, remember its legacy code that was not written with testing (other than high level integration tests) in mind.

I got around the problems of testing protected methods by sub classing.

Sub-classing is just inheriting from the class you want to test and adding public methods that expose the Protected methods of the base class.

Next I had to deal with testing some private methods, I got told that “if you need to test it, then it probably shouldn’t be private”. That’s fine but I needed to test these private methods as private methods as I didn’t want to re factor the code at all until I had good coverage of the code as it originally was.

So I had to find a way.. it turns out that its actually pretty simple, You can create something called a “Private Accessor”, this gives you access to all the private and public methods of a class.

To create a “Private Accessor” right click on a private method, select Create Private Accessor and choose the Test project to create it in.

What MS have to say: Private accessors, How to test a private method

Next I needed to remove the dependency on the file system. All I want is to test the code, not that the code can talk to the file system. This wasn’t the simplest thing I have ever done, but I have managed to achieve a working solution with the help of MS moles.

I have also created a test helper for StreamReader and StreamWritter and a dummy file that I use in-place of the file system when reading and writing files.

The DummyFile class just inherits a List. The DummyStreamReader and DummyStreamWritter just read and write from a DummyFile. Having this in place means I can very quickly detour the code that needs to read or write to a file in the file system and then assert that it has done what is was supposed to.

This might seem like a lot of effort to do some tests that avoid the file system, and it was a little bit. The result of it is that I now have a unit test suit that is quick to run and can run with no problems anywhere I can run the test harness.

The exact code needed to detour these requested is not as simple as I initially thought it would be but once you get a view of what it is you need to detour its actually pretty simple stuff (that is MS Moles).

I will go into more detail about moles in another post as it is an interesting subject.

So that was my first real adventure into legacy code testing ahead of refactoring and new code additions. I think the code is now in a much better state and even better is that we now have some proof that the code is doing what it is supposed to!

A general update on progress.

Since the first post I have got married and had my honeymoon, so for a while I didnt get a chance to make any headway.

With that in mind I have opted to use an opensource cart.

I have gone for http://code.google.com/p/sutekishop/ as it is light weight (only has features I actually want) simple to administer and well written, meaning its dead simple to extend.

It was originally developed by Mike Hadlow – Mike on twitter.

I am initially going to use paypal express checkout so I have made some changes to the flow of the cart as I dont need to store credit card information and paying over the phone is not an option.

I have made changes to the database scheme around orders so I can store information releated to the paypal payment against the order.

I have also made a change to the category pages urls, they were not very seo friendly to start with. But I now have them in the following format {siteurl}/category/{categoryslug}, that should be plenty friendly enough.

The product Urls are not too bad, its a format of {siteurl}/product/{productslug}

Next is to do my product import.

So far performance looks pretty good although I would like to get the initial page hit time down a bit.

This is especially true when you consider that most shared hosting providers will recycle the app pool if there are no http requestes for a 20 minuet period.

As I’m not expecting high traffic, maybe 20 visits a day to start the last thing I want is for these visitors to have to wait for the app to set itself up before serving a page.

Will post another update soon.

In this short article I am going to talk about where we (the team I work with) are and where I think we need to get to.

My aim is to make our lives easier, I want to empower the people around me to get on with “real” work and remove the hassles that are often faced when dealing with hard dependencies. Be that difficulties while writing tests, hard to maintain and read code or where easy to make mistakes could lead to big performance issues.

We have made progress:

One of the biggest initial problems was around testing and WCF, because every DB call we do goes through WCF we need to be able to moq the calls so we can have some control of the data being returned so we can test.

We are now in a place where by we have managed to put an abstraction in place to remove our hard dependencies around WCF, this means we can test code reliant on WCF with comparative ease.

To enable this testing we need to enable a level of constructor injection, we are currently doing this by hand, which is working well.

In small applications Constructor injection by hand can be very effective and is without doubt faster than using a DI container as there is effort involved in setting up said container.

Once an application grows maintaining this injection by hand can become a little cumbersome and error prone.

You also have to start thinking about the lifetime of objects in your application, how they are going to be passed around and where the objects get initialised.

Using an IoC container will remove this problem as it will be handled as part of the IoC container.

We have a second hard dependency on nHibernate, mainly its Session object. We have got to a place where by we have one session per request. This has made life much simpler and has allowed us to stop worrying about the session object and let nHibernate deal with it. This means we can get on with doing real work.

The problem I still have is in the code that sets this up, at this point in time we have a nasty hard dependency that we could get around but it would mean a bit of hackery which I am not really happy with.

Where would we benefit?

So seeing that we may benefit from an IoC container at some point for the bit mentioned above I thought it prudent to see if it could also help with the issue we are having here.

A quick look and I believe that an IoC container would fix the issue I have in a rather elegant way.

So that’s two immediate benefits, let the IoC container look after object lifetime and instantiation and fix a horrible hard dependency to improve the reuse of some code.

I think that would equate to some real benefit and Ive not mentioned the word testing once!

Another benefit I hope would come from this would be developers thinking a little harder about separation of concerns.

Considerations:

Something else to consider is the fact we are developing with Webforms and not MVC, this does make life a little more tricky, especially when we want to do usually simple things like constructor injection.

The problem is in the System.Web.UI.Page class, although it is a problem in System.Web.UI.UserControl class too.

The problem is that ASP.Net expects these types to have a default constructor.

As such we have to wire up the constructor injection in the page _Default method, this is not ideal and stops the usage being as fluid as it might over wise be.

There are ways around this, .net junkie (steven) gives a good example.

This wouldn’t be an issue if we developed with MVC instead, I wont hold my breath though.

How to sell this to the team?

If we are going to move to using an IoC there has to be very visible positives from doing so and they need to be painfully apparent from the outset. Without that we will not get buy in from the team and without that its dead in the water.

The only way to get the buy in needed is to create a project that shows off all the benefits, or create something else that relies on an IoC (maybe the nHibernate stuff) that is so awesome all on its own it gets the buy in needed to push this forward.

I will have to give a little more thought about how to get this up and running within the team.

If you are running a query and get the following linq error “Sequence contains no elements”.

It will be because your query does not return any results.

This is because PetaPoco makes use of the .single and .first linq commands.

If you have this issue it may be worth you using the SingleOrDefault and FirstOrDefault methods rather than the Single or First methods.

As you might guess the *OrDefault methods make use of the linq *OrDefault methods meaning in a case when there are no results you have the default (that would be a null) returned.