Tag Archives: Assert

Introduction to Unit Tests (with examples in .Net) – Part 1 – Structuring Tests

I’m intending this to be the first of a series on Unit Testing. In the series, I’ll discuss the basics of unit tests, the principles behind them, what makes a good unit test, what makes a bad unit test, and the technologies that you may choose to use to help you with them. I will not be covering test-driven development – this is simply about the mechanics and the reasons, not the methodology.

In this article, we’ll talk about what a unit test is, and how you might structure one. We will not be using any external tools for this, and what we do here should be possible in just about any language.

What is a Unit Test

A unit test is any way in which a single unit of functionality can be verified: it doesn’t have to be written before the code to be a test, it doesn’t have to be written in a test framework to be a test; it just has to run the code, and have some way of telling that the code has worked (the term “worked” is filled with ambiguity, but we’ll ignore that for the minute).

There are typically three parts to a unit test: they vary based on methodology, but essentially they are that you set-up the test, run the test, and check that the test worked; this is sometimes referred to as arrange, act, and assert.

For this section, we’ll be testing the following simple method:

int AddNumbers(int a, int b) => a + b;

Arrange

This is the part of the test where you configure the system under test (SUT). Given that you’re only testing a single piece of functionality, this can sometimes be quite involved, in order to get the system to place where it is ready to be tested, and actually running in a realistic manner. For our example above, this may look similar to the following:

// Arrange
int a = 4;
int b = 2;

Remember, we’re not using any external tools just yet – the above code could simply be in the Program.cs of a console application, or whatever the equivalent is in your language of choice; that is, just a simple program.

Act

The next part of the test actually exercises the code. The key thing here is that this is a unit test, so you would expect this to test a single unit of functionality; i.e., this should be a single line. In our case, it might look like this:

// Act
int result = AddNumbers(a, b);

We’ll come back to concepts such as mocking later in the series, but for now, let’s just agree with the comment that this part should exercise actual code; for example, there would be no advantage to the following code:

// Act
int result = a + b;

Your test may pass, but all you’re really testing is you compiler / interpreter. Writing tests that don’t actually test anything that you’re interested in is one of the biggest mistakes that I’ve seen with people new to unit testing. I would argue that having no tests at all is more valuable than a test that appears to provide coverage, but does not. After all, if there is no test, then you know that you need to create a test.

Assert

The final part of the test is to validate that the test passes – arguably this is around 50% of what you get from a testing tool like XUnit or JUnit – however, the following will work:

// Assert
System.Diagnostics.Debug.Assert(result == 6);

As in fact, will the more universal:

// Assert
if (result != 6) throw new Exception("Fail");

Unlike with the Act section, you can check several things are true; however, the test should be geared towards a single assertion. It’s worth bearing in mind that your assertion is that the functionality works correctly, not that a specific result is produced. This means that the test that we’ve discussed in this post is too specific.

Broadening a Test

Thinking about other possible scenarios, it’s tempting to introduce a randomised element into the test; that is, given two random numbers, the function will return the same result as that which the system independently calculates. I’m not saying this is a bad approach, but it isn’t a consistent one. This kind of test often leads to tests failing on some runs, and passing on others.

First Principles

I have no doubt that this has fallen out of favour somewhere, but the FIRST acronym provides some useful principles for testing:

Fast. Independent. Repeatable. Self-validating. Timely.

I won’t cover each one of these, but the essence of this principle is that when you run a test, you should have confidence that you can re-run the test with the same result (given the same inputs), and that your tests should be relevant to what you’re testing.

How to Broaden Our Test

Given our constraints, one easy way to broaden the test scope is to simply introduce multiple defined input parameters. In our case, perhaps instead of having two integers to feed in, we have an array and iterate through the array.

Naming a Test

The final thing that I want to cover in this first section is naming. There are many opinions on this, so there’s no right way; however, there probably are wrong ways. In general terms, the test should be named in a way that any person reading it could ascertain what is being tested; one popular version of this is to use the Given/When/Then form:

Given_TwoValidNumbers_When_AddNumbers_Then_CorrectResultIsReturned

Another one, that I personally use, is the format: Method Name/State Under Test/Expected Result; for example:

AddNumbers_TwoValidNumbers_CorrectResultIsReturned

The key here is consistency (i.e., don’t mix and match), and clarity; the following is an example of a bad test name:

AddNumbers_Works

In the next post in this series, we’ll talk about more complex tests, and mocking.

NUnit TestCaseSource

While working on this project, I found a need to abstract away a base type that the unit tests use (in this instance, it was a queue type). I was only testing a single type (PriorityQueue); however, I wanted to create a new type, but all the basic tests for the new type are the same as the existing ones. This led me to investigate the TestCaseSource attribute in NUnit.

As a result, I needed a way to re-use the tests. There are definitely multiple ways to do this; the simplest one is probably to create a factory class, and pass in a string parameter. The only thing that put me off this is that you end up with the following test case:

        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string first, string last, params string[] queueItems)
        {

Becoming:

        [TestCase("PriorityQueue", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue2", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue", "a1", "a", "a1", "b", "c", "d", "a"]
        [TestCase("PriorityQueue2", "a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string queueType, string first, string last, params string[] queueItems)
        {

This isn’t very scaleable when adding a third or fourth type.

TestCaseSource

It turns out that the (or an least an) answer to this is to use NUnit’s TestCaseSource attribute. The NUnit code base dog foods quite extensively, so that is not a bad place to look for examples of how this works; however, what I couldn’t find was a way to mix and match. To better illustrate the point; here’s the first test that I changed to use TestCaseSource:

        [Test]
        public void Queue_NoEntries_CheckCount()
        {
            // Arrange
            PQueue.PriorityQueue<string> queue = new PQueue.PriorityQueue<string>();

            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

Which became:

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        public void Queue_NoEntries_CheckCount(IQueue<string> queue)
        {
            // Arrange


            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

(For completeness, the TestableQueueItemFactory is here):

    public static class TestableQueueItemFactory
    {
        public static IEnumerable<IQueue<string>> ReturnQueueTypes()
        {
            yield return new PQueue.PriorityQueue<string>();
        }
    }

However, when you have a TestCase like the one above, there’s a need for the equivalent of this (which doesn’t work):

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9")]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a")]
        public void Queue_Dequeue_CheckResultOrdering(string first, string last, params string[] queueItems)
        {

A quick look at the NUnit code base reveals these attributes to be mutually exclusive.

Compromise

By no means is this a perfect solution, but the one that I settled on was to create a second TestCaseSource helper method, which looks like this (along with the test):

        private static IEnumerable Queue_Dequeue_CheckResultOrdering_TestCase()
        {
            foreach(var queueType in TestableQueueItemFactory.ReturnQueueTypes())
            {
                yield return new object[] { queueType, "test", "test9", new string[] { "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9" } };
                yield return new object[] { queueType, "a1", "a", new string[] { "a1", "b", "c", "d", "a" } };
            }
        }

        [Test, TestCaseSource("Queue_Dequeue_CheckResultOrdering_TestCase")]
        public void Queue_Dequeue_CheckResultOrdering(
            IQueue <string> queue, string first, string last, params string[] queueItems)
        {

As you can see, the second helper method doesn’t really help readability, so it’s certainly not a perfect solution; in fact, with a single queue type, this makes the code more complex and less readable. However, When a second and third queue type are introduced, the test suddenly becomes resilient.

YAGNI

At first glance, this may appear to be an example of YAGNI. However, in this article, Martin Fowler does state:

Yagni only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.

Which, I believe, is what we are doing here.

References

http://www.smaclellan.com/posts/parameterized-tests-made-simple/

http://stackoverflow.com/questions/16346903/how-to-use-multiple-testcasesource-attributes-for-an-n-unit-test

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

http://dotnetgeek.tumblr.com/post/2851360238/exploiting-nunit-attributes-valuesourceattribute

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

Testing for Exceptions using the Arrange Act Assert Pattern in C# 7

Unit testing massively benefits from following the Arrange / Act / Assert pattern. I’ve seen tests that are not written in this way, and they can be sprawling and indecipherable, either testing many different things in series, or testing nothing at all except the .Net Framework.

I recently found an issue while trying to test for an exception being thrown, which is that Nunit (and probably other frameworks) test for an exception by accepting a delegate to test. Here’s an example:

        [Test]
        public void Test_ThrowException_ExceptionThrown()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act / Assert
            Assert.Throws(typeof(Exception), tc.ThrowException);
        }

We’re just testing a dummy class:

    public class TestClass
    {
        public void ThrowException()
        {
            throw new Exception("MyException");
        }
    }

C# 7 – Inline functions

If you look in the references at the bottom, you’ll see something more akin to this approach:

        public void Test_ThrowException_ExceptionThrown2()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act
            TestDelegate throwException = () => tc.ThrowException();            

            // Assert
            Assert.Throws(typeof(Exception), throwException);
        }

However, since C# 7, the option on a local function has arrived. The following has the same effect:

        [Test]
        public void Test_ThrowException_ExceptionThrown3()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act
            void CallThrowException()
            {
                tc.ThrowException();
            }

            // Assert
            Assert.Throws(typeof(Exception), CallThrowException);
        }

I think that I, personally, still prefer the anonymous function for this; however, the local function does present some options; for example:

        [Test]
        public void Test_ThrowException_ExceptionThrown4()
        {
            void CallThrowException()
            {
                // Arrange
                TestClass tc = new TestClass();

                // Act
                tc.ThrowException();
            }

            // Assert
            Assert.Throws(typeof(Exception), CallThrowException);
        }

Now I’m not so sure that I still prefer the anonymous function.

References

http://stackoverflow.com/questions/33897323/nunit-3-0-and-assert-throws

https://pmbanugo.wordpress.com/2014/06/16/exception-testing-pattern/

http://stackoverflow.com/questions/24070115/best-approach-towards-applying-the-arrange-act-assert-pattern-when-expecting-exc