Tag Archives: NUnit

Introduction to Unit Tests (with examples in .Net) – Part 2 – Refactoring and Mocking

This forms the second in a short series of posts on unit tests. You can find the first post in this series here.

In this post, I’ll be expanding the points raised in the first post to cover a slightly more realistic scenario, and offering some tips of how re-factoring might help with the creation of unit tests. We’ll also cover the basic principles behind mocking – I’m intending to cover this in more detail in a future post of this series.

A Quick Recap

You’re welcome (and encouraged) to go back to the first post in the series; however, to summarise, we discussed the Arrange/Act/Assert pattern, and how it can help us structure a unit test; we spoke about the FIRST principles of testing, and thereby the attributes that we should look for in a good unit test.

What we specifically didn’t cover was any testing frameworks, the concept of mocking or any mocking frameworks, or how to write a unit test in a scenario where you’re not simply adding two numbers together.

A More Realistic Unit Test

If we take an example of any level of complexity, we might question some of the points that were made in the first post. After all, very few methods would simply take two numbers and add them together – or, if they do, perhaps we need to reconsider the language that we’re using.

Let’s look at a simple console application:

int myNumber = Random.Shared.Next(100) + 1;

Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
var guess = Console.ReadLine();
if (string.IsNullOrEmpty(guess))
{
    Console.WriteLine("Invalid guess");
    return;
}

if (int.Parse(guess) == myNumber)
{
    Console.WriteLine("Well done, you guessed!");
}
else
{
    Console.WriteLine($"Sorry, that was the wrong number, I was thinking of {myNumber}");
}

If you had to test this code, how would you do it?

In fact, it’s really difficult, because every time you run it, the number is different. This is a simple piece of code, there’s only 3 code paths; arguably, your strategy could be: run it once and enter a blank value, run it once and enter a non-numeric value, run it 100 more times and hope that you’ll get the number right once.

Writing a Test

As before, let’s start with the manual test that we’ve just described; arguably, we could simply automate this. We’d probably do something like this:

        public static void RunTest()
        {
            // Run method here - check that a blank entry works

            // Run method here - check that a numeric entry works

            for (int i = 0 ; i < 100; i++)
            {
                // Run method here - exit this loop once we've determined that we have a correct and incorrect guess

            }
        }

In fact, running that exact test would be possible – we could simply redirect the console input and output; however, for the purposes of this post, we’ll bypass that method (as it is quite specific to writing a .Net Console app), and we’ll re-factor our code a little.

Refactoring

We can refactor it by splitting the method into two; one method that accepts the input, and one that runs the business logic:

void RunMethod()
{
    int myNumber = Random.Shared.Next(100) + 1;

    Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
    var guess = Console.ReadLine();
    BusinessLogic(myNumber, guess);
}

void BusinessLogic(int myNumber, string guessedNumber)
{
    if (string.IsNullOrEmpty(guessedNumber))
    {
        Console.WriteLine("Invalid guess");
        return;
    }

    if (int.Parse(guessedNumber) == myNumber)
    {
        Console.WriteLine("Well done, you guessed!");
    }
    else
    {
        Console.WriteLine($"Sorry, that was the wrong number, I was thinking of {myNumber}");
    }
}

All we’ve done here is split the method into two methods – the code is exactly the same as it was before. However, now we can run the code in our test without worrying about the input:

        public static void RunTest()
        {
            // Check that a blank entry works
            BusinessLogic(3, "");

            // Check that a non-numeric entry works
            BusinessLogic(3, "aardvark");

            for (int i = 1; i <= 100; i++)
            {
                // Check that false and true numbers work
                BusinessLogic(i, "2");
            }
        }

It still feels a lot like we’re only testing half of the code. We aren’t testing the calling method.

Mocking

Let’s refactor a little further. Instead of using the console, we’ll simply create our own method that performs the same task:

private static string? GetInput() => Console.ReadLine();    

Again, no real change here, we’re just wrapping the code that accepts input in a method that we control.

Now that we’ve done this, we can use our own method to accept input, instead of the Console methods. There’s a number of ways we could do this but, perhaps the easiest, is to pass the GetInput method into the RunMethod method as a parameter:

public static void RunMethod(Func<string> readData)
    {

        int myNumber = Random.Shared.Next(100) + 1;

        Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
        string? guess = readData();
. . .

Here, we’re simply changing two things: we’re accepting a delegate into our main method, and then we’re calling that, instead of the Console.ReadLine().

What’s the point of doing that? Well, now that we control that function as a parameter, we can mock the function, and replace it with our own functionality.

In fact, we are not technically discussing a mock here, but a stub. For the purpose of this post, we’ll simply group them together with the working definition that a mock is: “anything that replaces functionality for the purpose of testing”. I intend to re-visit this in a future post and go into more detail on the difference between the two.

Let’s jump to our test.

Arrange

In the test, we can now replace this functionality with a specific value:

        public static void RunTest()
        {
             // Arrange
            Func<string> mockInput = () => "5"; . . .

We’ve now established the input of the method, the next step is to be able to assert that the test worked. In fact, that’s very difficult with the code as it currently is, because we just display output to the user. To finish this post off, we’ll refactor this as little as we can; imagine we take the business logic function and change it to be like this:

string BusinessLogic(int myNumber, string guessedNumber)
{
    if (string.IsNullOrEmpty(guessedNumber))
    {        
        return "Invalid guess";
    }

    if (int.Parse(guessedNumber) == myNumber)
    {
        return "Well done, you guessed!";
    }
    else
    {
        return $"Sorry, that was the wrong number, I was thinking of {myNumber}";
    }
}

All we’ve changed here is that we’re returning the string, instead of outputting it. We can then change the calling method to do the output:

string RunMethod(Func<string> readData)
{
    int myNumber = Random.Shared.Next(100) + 1;

    Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
    string? guess = readData();
    string result = BusinessLogic(myNumber, guess);
    Console.WriteLine(result);
    return result;
}

Same idea again, a very small change of writing the output, and returning the result again. The functionality hasn’t changed, but now we have something to test against.

Assert

We can now write out test method to look something like this:

for (int i = 1; i <= 100; i++)
{
    // Arrange
    Func<string> mockInput = () => "5";

    // Act
    string result = RunMethod(mockInput);

    // Assert
    if (result == "Well done, you guessed!")
    {
        Console.WriteLine("Test Passed");
        break;
    }
}

for (int i = 1; i <= 100; i++)
{
    // Arrange
    Func<string> mockInput = () => "5";

    // Act
    string result = RunMethod(mockInput);

    // Assert
    if (result.StartsWith("Sorry, that was the wrong number"))
    {
        Console.WriteLine("Test Passed");
        break;
    }
}

{
    // Arrange
    Func<string> mockInput = () => "";

    // Act
    string result = RunMethod(mockInput);

    // Assert
    if (result == "Invalid guess")
    {
        Console.WriteLine("Test Passed");
    }
}

There’s three distinct tests here and, unless unlucky, they’ll all pass. There’s definitely some work left to do here, as we still have the following problems:

1. Although the tests can pass, we have to visually ascertain that.
2. We’re outputting to the console needlessly.
3. Our tests are not resilient – if I change a single character in the user output, the tests will break.
4. The tests are not deterministic – they are dependent on the result of a pseudo random number.

In the next post, we’ll address these issues: we’ll introduce a test framework, and further refactor this code such that we can be confident that cosmetic changes will not break the tests.

Live Testing in Visual Studio 2017

While working on this project, I decided that I would see how useful the new feature of “Live Testing” is for TDD.

Once you’ve created your suite of tests, turning live testing on gives you some interesting information:

As you can see, there are some slight contradictions here in what it’s telling me. The first test is failing, the test for Count has a line (which indicates not covered). However, code lens is telling me there are two covering tests.

My suspicion here is that live testing is being too clever for its own good; for example, looking at the tests themselves, shows that only the first Enqueue statement is covered:

My thoughts are that the engine works out where the test fails, and so, in the case above, it’s telling me that the tests are not running the line of code that tests the Count functionality. What I’m guessing is that this is because the tests are failing before that.

I can prove this by creating a further unit test:

And, indeed, this now shows coverage:

As I start to add the functionality, I can quickly see that the tests are now passing:

Changing the way code is written

The interesting thing here is that is promotes two things:
1. Defined tests – if a test covers more than one thing, it will show as uncovered initially
2. Tests first – obviously, there need to be tests to show test coverage

Caveats

1. Annoyingly, by default, you continually have to opt in for live unit testing. It only starts when you select to “Start” live unit testing, and this must be done each time you launch VS. This is configurable, and can be switched on by default in options:

2. If the build fails, it shows nothing.

3. Live testing is only available in Visual Studio Enterprise.

NUnit

One thing that only occurred to me as notable while I was writing this is that the tests that I’ve written are in NUnit. So the Live Testing clearly works with none MSTests, and probably works with anything that’ll show up in Test Explorer / Code Lens.

References

https://blogs.msdn.microsoft.com/visualstudio/2017/03/09/live-unit-testing-in-visual-studio-2017-enterprise/

NUnit TestCaseSource

While working on this project, I found a need to abstract away a base type that the unit tests use (in this instance, it was a queue type). I was only testing a single type (PriorityQueue); however, I wanted to create a new type, but all the basic tests for the new type are the same as the existing ones. This led me to investigate the TestCaseSource attribute in NUnit.

As a result, I needed a way to re-use the tests. There are definitely multiple ways to do this; the simplest one is probably to create a factory class, and pass in a string parameter. The only thing that put me off this is that you end up with the following test case:

        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string first, string last, params string[] queueItems)
        {

Becoming:

        [TestCase("PriorityQueue", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue2", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue", "a1", "a", "a1", "b", "c", "d", "a"]
        [TestCase("PriorityQueue2", "a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string queueType, string first, string last, params string[] queueItems)
        {

This isn’t very scaleable when adding a third or fourth type.

TestCaseSource

It turns out that the (or an least an) answer to this is to use NUnit’s TestCaseSource attribute. The NUnit code base dog foods quite extensively, so that is not a bad place to look for examples of how this works; however, what I couldn’t find was a way to mix and match. To better illustrate the point; here’s the first test that I changed to use TestCaseSource:

        [Test]
        public void Queue_NoEntries_CheckCount()
        {
            // Arrange
            PQueue.PriorityQueue<string> queue = new PQueue.PriorityQueue<string>();

            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

Which became:

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        public void Queue_NoEntries_CheckCount(IQueue<string> queue)
        {
            // Arrange


            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

(For completeness, the TestableQueueItemFactory is here):

    public static class TestableQueueItemFactory
    {
        public static IEnumerable<IQueue<string>> ReturnQueueTypes()
        {
            yield return new PQueue.PriorityQueue<string>();
        }
    }

However, when you have a TestCase like the one above, there’s a need for the equivalent of this (which doesn’t work):

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9")]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a")]
        public void Queue_Dequeue_CheckResultOrdering(string first, string last, params string[] queueItems)
        {

A quick look at the NUnit code base reveals these attributes to be mutually exclusive.

Compromise

By no means is this a perfect solution, but the one that I settled on was to create a second TestCaseSource helper method, which looks like this (along with the test):

        private static IEnumerable Queue_Dequeue_CheckResultOrdering_TestCase()
        {
            foreach(var queueType in TestableQueueItemFactory.ReturnQueueTypes())
            {
                yield return new object[] { queueType, "test", "test9", new string[] { "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9" } };
                yield return new object[] { queueType, "a1", "a", new string[] { "a1", "b", "c", "d", "a" } };
            }
        }

        [Test, TestCaseSource("Queue_Dequeue_CheckResultOrdering_TestCase")]
        public void Queue_Dequeue_CheckResultOrdering(
            IQueue <string> queue, string first, string last, params string[] queueItems)
        {

As you can see, the second helper method doesn’t really help readability, so it’s certainly not a perfect solution; in fact, with a single queue type, this makes the code more complex and less readable. However, When a second and third queue type are introduced, the test suddenly becomes resilient.

YAGNI

At first glance, this may appear to be an example of YAGNI. However, in this article, Martin Fowler does state:

Yagni only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.

Which, I believe, is what we are doing here.

References

http://www.smaclellan.com/posts/parameterized-tests-made-simple/

http://stackoverflow.com/questions/16346903/how-to-use-multiple-testcasesource-attributes-for-an-n-unit-test

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

http://dotnetgeek.tumblr.com/post/2851360238/exploiting-nunit-attributes-valuesourceattribute

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

Testing for Exceptions using the Arrange Act Assert Pattern in C# 7

Unit testing massively benefits from following the Arrange / Act / Assert pattern. I’ve seen tests that are not written in this way, and they can be sprawling and indecipherable, either testing many different things in series, or testing nothing at all except the .Net Framework.

I recently found an issue while trying to test for an exception being thrown, which is that Nunit (and probably other frameworks) test for an exception by accepting a delegate to test. Here’s an example:

        [Test]
        public void Test_ThrowException_ExceptionThrown()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act / Assert
            Assert.Throws(typeof(Exception), tc.ThrowException);
        }

We’re just testing a dummy class:

    public class TestClass
    {
        public void ThrowException()
        {
            throw new Exception("MyException");
        }
    }

C# 7 – Inline functions

If you look in the references at the bottom, you’ll see something more akin to this approach:

        public void Test_ThrowException_ExceptionThrown2()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act
            TestDelegate throwException = () => tc.ThrowException();            

            // Assert
            Assert.Throws(typeof(Exception), throwException);
        }

However, since C# 7, the option on a local function has arrived. The following has the same effect:

        [Test]
        public void Test_ThrowException_ExceptionThrown3()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act
            void CallThrowException()
            {
                tc.ThrowException();
            }

            // Assert
            Assert.Throws(typeof(Exception), CallThrowException);
        }

I think that I, personally, still prefer the anonymous function for this; however, the local function does present some options; for example:

        [Test]
        public void Test_ThrowException_ExceptionThrown4()
        {
            void CallThrowException()
            {
                // Arrange
                TestClass tc = new TestClass();

                // Act
                tc.ThrowException();
            }

            // Assert
            Assert.Throws(typeof(Exception), CallThrowException);
        }

Now I’m not so sure that I still prefer the anonymous function.

References

http://stackoverflow.com/questions/33897323/nunit-3-0-and-assert-throws

https://pmbanugo.wordpress.com/2014/06/16/exception-testing-pattern/

http://stackoverflow.com/questions/24070115/best-approach-towards-applying-the-arrange-act-assert-pattern-when-expecting-exc

Unit Tests Are Not Discoverable

I recently had a situation where I loaded a solution containing a suite of NUnit tests, but the test explorer would not recognise them. The following is a series of checks to make that may cause unit tests to be not visible. Most of these are applicable to all tests:

1. Tests must be declared as public. For example:

        public void MyTestMethod()
        {

2. Tests must be decorated with a test attribute.

For NUnit this is is [Test]:

	        [Test]
	        public void MyNUnitTest()
	        {
	

Or [TestCase]:

	        [TestCase(1)]
	        [TestCase(2)]
	        public void MyParameterisedTest(int testNum)
	        {
	
	

For MSTest this is [TestMethod]:

	        [TestMethod]
	        public void MyTestMethod()
	        {
	

3. If using NUnit – check that the correct version is installed (remember that v3 is not an official release yet – that is, at the time of writing).

The current release test adapter is here

The test adaptor for NUnit3 is here

As usual, this is more for my own reference, but if it helps anyone else then all to the good. Also, if you think of or encounter another then please let me know in the comments and I’ll add it on.