Tag Archives: Unit Testing

Introduction to Unit Tests (with examples in .Net) – Part 1 – Structuring Tests

I’m intending this to be the first of a series on Unit Testing. In the series, I’ll discuss the basics of unit tests, the principles behind them, what makes a good unit test, what makes a bad unit test, and the technologies that you may choose to use to help you with them. I will not be covering test-driven development – this is simply about the mechanics and the reasons, not the methodology.

In this article, we’ll talk about what a unit test is, and how you might structure one. We will not be using any external tools for this, and what we do here should be possible in just about any language.

What is a Unit Test

A unit test is any way in which a single unit of functionality can be verified: it doesn’t have to be written before the code to be a test, it doesn’t have to be written in a test framework to be a test; it just has to run the code, and have some way of telling that the code has worked (the term “worked” is filled with ambiguity, but we’ll ignore that for the minute).

There are typically three parts to a unit test: they vary based on methodology, but essentially they are that you set-up the test, run the test, and check that the test worked; this is sometimes referred to as arrange, act, and assert.

For this section, we’ll be testing the following simple method:

int AddNumbers(int a, int b) => a + b;

Arrange

This is the part of the test where you configure the system under test (SUT). Given that you’re only testing a single piece of functionality, this can sometimes be quite involved, in order to get the system to place where it is ready to be tested, and actually running in a realistic manner. For our example above, this may look similar to the following:

// Arrange
int a = 4;
int b = 2;

Remember, we’re not using any external tools just yet – the above code could simply be in the Program.cs of a console application, or whatever the equivalent is in your language of choice; that is, just a simple program.

Act

The next part of the test actually exercises the code. The key thing here is that this is a unit test, so you would expect this to test a single unit of functionality; i.e., this should be a single line. In our case, it might look like this:

// Act
int result = AddNumbers(a, b);

We’ll come back to concepts such as mocking later in the series, but for now, let’s just agree with the comment that this part should exercise actual code; for example, there would be no advantage to the following code:

// Act
int result = a + b;

Your test may pass, but all you’re really testing is you compiler / interpreter. Writing tests that don’t actually test anything that you’re interested in is one of the biggest mistakes that I’ve seen with people new to unit testing. I would argue that having no tests at all is more valuable than a test that appears to provide coverage, but does not. After all, if there is no test, then you know that you need to create a test.

Assert

The final part of the test is to validate that the test passes – arguably this is around 50% of what you get from a testing tool like XUnit or JUnit – however, the following will work:

// Assert
System.Diagnostics.Debug.Assert(result == 6);

As in fact, will the more universal:

// Assert
if (result != 6) throw new Exception("Fail");

Unlike with the Act section, you can check several things are true; however, the test should be geared towards a single assertion. It’s worth bearing in mind that your assertion is that the functionality works correctly, not that a specific result is produced. This means that the test that we’ve discussed in this post is too specific.

Broadening a Test

Thinking about other possible scenarios, it’s tempting to introduce a randomised element into the test; that is, given two random numbers, the function will return the same result as that which the system independently calculates. I’m not saying this is a bad approach, but it isn’t a consistent one. This kind of test often leads to tests failing on some runs, and passing on others.

First Principles

I have no doubt that this has fallen out of favour somewhere, but the FIRST acronym provides some useful principles for testing:

Fast. Independent. Repeatable. Self-validating. Timely.

I won’t cover each one of these, but the essence of this principle is that when you run a test, you should have confidence that you can re-run the test with the same result (given the same inputs), and that your tests should be relevant to what you’re testing.

How to Broaden Our Test

Given our constraints, one easy way to broaden the test scope is to simply introduce multiple defined input parameters. In our case, perhaps instead of having two integers to feed in, we have an array and iterate through the array.

Naming a Test

The final thing that I want to cover in this first section is naming. There are many opinions on this, so there’s no right way; however, there probably are wrong ways. In general terms, the test should be named in a way that any person reading it could ascertain what is being tested; one popular version of this is to use the Given/When/Then form:

Given_TwoValidNumbers_When_AddNumbers_Then_CorrectResultIsReturned

Another one, that I personally use, is the format: Method Name/State Under Test/Expected Result; for example:

AddNumbers_TwoValidNumbers_CorrectResultIsReturned

The key here is consistency (i.e., don’t mix and match), and clarity; the following is an example of a bad test name:

AddNumbers_Works

In the next post in this series, we’ll talk about more complex tests, and mocking.

Asp.Net Policy Authorization Based on a DB Field on the User Table

If you read this, you’ll learn how to create authorisation, based on a policy. Specifically, they use the example of the user’s age, and create a restriction to say that only users over the age of 21 can access a resource.

The age was something that they got from a claim, stored against the user. But what if your requirement is a little more complex? For example, what if you have a situation, such as a popular question and answer site that you may recognise, where you are trying to restrict access to a resource based on something that can change during the user session. In this post, I’m covering how you can follow the same policy structure, but force the program to go back to the DB to get details about the user, each time they try to access the restricted resource.

As a background, the particular use case here is that a user of my site can approve something, but only where they have a sufficient rating.

Basic Set-up

The first thing that you’ll need is a custom Authorization Handler. For my project, I’ve created a sub directory called Authorization (I’ve even spelled it wrong to be consistent):

The Authorization Handler inherits from an abstract class, which forces you to override a single method: HandleRequirementAsync. Essentially, you tell the handler about your requirement (we’ll come to the requirement next), and then it passes this back to you in the method; let’s have a look at some code:

    public class ApproverAuthHandler : AuthorizationHandler<ApproverAuthRequirement>
    {
        protected override Task HandleRequirementAsync(
            AuthorizationHandlerContext context, 
            ApproverAuthRequirement requirement)
        {

The Requirement is just a class to hold the relevant data that you need. In our example, the requirement would hold the user’s rating; here’s the requirement code:

    public class ApproverAuthRequirement : IAuthorizationRequirement
    {
        public int UserRating { get; set; }

        public ApproverAuthRequirement(int userRating)
        {
            UserRating = userRating;
        }
    }

IAuthorizationRequirment is what is known as a marker interface. To be honest, I hadn’t come across the term before, but I have used the pattern before. Essentially, this is an empty interface: its purpose it to allow you to pass the class type as a strongly typed interface; but in reality, the class can me anything.

Anyway, let’s get back to the Authorization Handler. Inside HandlerRequirementAsync, you can do anything you choose; should your check be successful, you call context.Succeed(requirement), otherwise, do nothing. In the above linked code, they check a claim against the requirement; we’re going to just get some information from the DB, and check that against the requirement:

        protected override Task HandleRequirementAsync(
            AuthorizationHandlerContext context, 
            ApproverAuthRequirement requirement)
        {
            int? userRating = _userService.GetSubjectRating(context.User);            
            if (!userRating.HasValue)
            {
                return Task.CompletedTask;
            }

            if (userRating >= requirement.UserRating)
            {
                context.Succeed(requirement);
            }

            return Task.CompletedTask;
        }

User service is simply a service that calls into a repository, and returns the user rating.

Unit Testing

All well and good, and this looks eminently testable. In fact, it is: you simply mock out the service, and the test looks like this:

            // Arrange
            var requirements = new[] { new ApproverAuthRequirement(100) };            
            var user = new ClaimsPrincipal(
                        new ClaimsIdentity(
                            new Claim[] { },
                            "Basic")
                        );

            var userService = Substitute.For<IUserService>();
            userService.GetSubjectRating(Arg.Any<IPrincipal>()).Returns(100);

            var context = new AuthorizationHandlerContext(requirements, user, null);
            var sut = new ApproverAuthHandler(userService);

            // Act
            await sut.HandleAsync(context);

            // Assert
            Assert.True(context.HasSucceeded);

No doubt, you could check for the exact service principal under test, although you’re testing the Authorization Handler, so I think this is sufficient.

Resource Based Handler

In this article, there is a similar process discussed; it warrants some further investigation on my part, but I don’t currently see any real difference between my implementation, and this abstraction (although, admittedly, this version looks easier to test).

References

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/policies?view=aspnetcore-3.1

https://stackoverflow.com/questions/51272610/unit-test-authorizationhandler

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/resourcebased?view=aspnetcore-3.1

NUnit TestCaseSource

While working on this project, I found a need to abstract away a base type that the unit tests use (in this instance, it was a queue type). I was only testing a single type (PriorityQueue); however, I wanted to create a new type, but all the basic tests for the new type are the same as the existing ones. This led me to investigate the TestCaseSource attribute in NUnit.

As a result, I needed a way to re-use the tests. There are definitely multiple ways to do this; the simplest one is probably to create a factory class, and pass in a string parameter. The only thing that put me off this is that you end up with the following test case:

        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string first, string last, params string[] queueItems)
        {

Becoming:

        [TestCase("PriorityQueue", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue2", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue", "a1", "a", "a1", "b", "c", "d", "a"]
        [TestCase("PriorityQueue2", "a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string queueType, string first, string last, params string[] queueItems)
        {

This isn’t very scaleable when adding a third or fourth type.

TestCaseSource

It turns out that the (or an least an) answer to this is to use NUnit’s TestCaseSource attribute. The NUnit code base dog foods quite extensively, so that is not a bad place to look for examples of how this works; however, what I couldn’t find was a way to mix and match. To better illustrate the point; here’s the first test that I changed to use TestCaseSource:

        [Test]
        public void Queue_NoEntries_CheckCount()
        {
            // Arrange
            PQueue.PriorityQueue<string> queue = new PQueue.PriorityQueue<string>();

            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

Which became:

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        public void Queue_NoEntries_CheckCount(IQueue<string> queue)
        {
            // Arrange


            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

(For completeness, the TestableQueueItemFactory is here):

    public static class TestableQueueItemFactory
    {
        public static IEnumerable<IQueue<string>> ReturnQueueTypes()
        {
            yield return new PQueue.PriorityQueue<string>();
        }
    }

However, when you have a TestCase like the one above, there’s a need for the equivalent of this (which doesn’t work):

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9")]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a")]
        public void Queue_Dequeue_CheckResultOrdering(string first, string last, params string[] queueItems)
        {

A quick look at the NUnit code base reveals these attributes to be mutually exclusive.

Compromise

By no means is this a perfect solution, but the one that I settled on was to create a second TestCaseSource helper method, which looks like this (along with the test):

        private static IEnumerable Queue_Dequeue_CheckResultOrdering_TestCase()
        {
            foreach(var queueType in TestableQueueItemFactory.ReturnQueueTypes())
            {
                yield return new object[] { queueType, "test", "test9", new string[] { "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9" } };
                yield return new object[] { queueType, "a1", "a", new string[] { "a1", "b", "c", "d", "a" } };
            }
        }

        [Test, TestCaseSource("Queue_Dequeue_CheckResultOrdering_TestCase")]
        public void Queue_Dequeue_CheckResultOrdering(
            IQueue <string> queue, string first, string last, params string[] queueItems)
        {

As you can see, the second helper method doesn’t really help readability, so it’s certainly not a perfect solution; in fact, with a single queue type, this makes the code more complex and less readable. However, When a second and third queue type are introduced, the test suddenly becomes resilient.

YAGNI

At first glance, this may appear to be an example of YAGNI. However, in this article, Martin Fowler does state:

Yagni only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.

Which, I believe, is what we are doing here.

References

http://www.smaclellan.com/posts/parameterized-tests-made-simple/

http://stackoverflow.com/questions/16346903/how-to-use-multiple-testcasesource-attributes-for-an-n-unit-test

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

http://dotnetgeek.tumblr.com/post/2851360238/exploiting-nunit-attributes-valuesourceattribute

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

Check table exists in SQLite (in a UWP app)

I’ve recently started investigating the use of SQLite for the purpose of storing data in my UWP application. Before I start, the following are the best tutorials that I could find on the net to get started with SQLite:

http://blogs.u2u.be/diederik/post/2015/09/08/Using-SQLite-on-the-Universal-Windows-Platform.aspx

https://blogs.windows.com/buildingapps/2016/05/03/data-access-in-universal-windows-platform-uwp-apps/

The SQLite home page is here.

Finally, the nuget package to integrate SQLIte with your UWP app is here.

I didn’t find too much information on checking if a table exists, and I think it’s quite a key thing to be able to do, as you can’t really have a deployment script that runs when you deploy your UWP app. As a result I create a base data access class. Let’s start with the interface:

public interface IDataAccess
{
    void Connect();
    bool DoesTableExist<T>() where T : class;
    bool RemoveTable<T>() where T : class;

    bool CreateTable<T>() where T : class;
}

Next, I created a unit test to make sure that the method will work:

    [TestClass]
    public class TestBaseMethods
    {       
        [DataTestMethod]
        [DataRow("MyApp.DataAccess.SQLLiteDataAccess, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
        public void TablesCreate(string dataAccess)
        {
            // Arrange            
            IDataAccess bda = Activator.CreateInstance(Type.GetType(dataAccess)) as IDataAccess;
            bda.Connect();

            Assert.AreEqual(false, bda.DoesTableExist<TestEntity>());

            // Act
            bda.CreateTable<TestEntity>();

            // Assert
            Assert.AreEqual(true, bda.DoesTableExist<TestEntity>());


            // Clean-up
            bda.RemoveTable<TestEntity>();
            Assert.AreEqual(false, bda.DoesTableExist<TestEntity>());
        }

If you want more information about DataRow, then see my recent post

Implementation

We’ll start with a basic class that provides a Connect method and cleans up after itself:

namespace MyApp.DataAccess
{
    public class SQLLiteDataAccess : IDisposable, IDataAccess
    {
        SQLiteConnection _connection;

        public void Connect()
        {
            _connection = new SQLiteConnection(new SQLitePlatformWinRT(),
                Path.Combine(ApplicationData.Current.LocalFolder.Path, "Storage.sqlite"));
        }

        public void Dispose()
        {
            _connection.Dispose();
        }

Then we need to add the functionality from the interface:

        /// <summary>
        /// Generic method to determine is an object type exists within the DB
        /// </summary>
        /// <param name="type"></param>
        /// <param name="name"></param>
        /// <returns></returns>
        private bool DoesTypeExist(string type, string name)
        {
            SQLiteCommand command = _connection.CreateCommand("SELECT COUNT(1) FROM SQLITE_MASTER WHERE TYPE = @TYPE AND NAME = @NAME");
            command.Bind("@TYPE", type);
            command.Bind("@NAME", name);
            
            int result = command.ExecuteScalar<int>();

            return (result > 0);
        }

        /// <summary>
        /// Check whether a specific table exists in the database
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <returns>true if the specified table exists</returns>
        public bool DoesTableExist<T>() where T : class
        {
            bool exists = DoesTypeExist("table", typeof(T).Name);
            return exists;
        }

        /// <summary>
        /// Create a new table in the DB
        /// </summary>
        /// <typeparam name="T">Object to base the table on</typeparam>
        /// <returns></returns>
        public bool CreateTable<T>() where T : class
        {
            bool exists = DoesTableExist<T>();

            if (!exists)
            {
                int error = _connection.CreateTable<T>();
                exists = (error == 0);
            }

            return exists;
        }

        /// <summary>
        /// Remove / drop the specified table from the DB
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <returns></returns>
        public bool RemoveTable<T>() where T : class
        {
            bool exists = DoesTableExist<T>();

            if (exists)
            {
                int error = _connection.DropTable<T>();
                exists = (error != 0);
            }

            return exists;
        }

That’s it; I’m still not sure how I feel about SQLite. For a small, local storage, it feels like it could be an XML file; but I suppose it comes into its own when we have a potentially large local storage.

Using MSTest DataRow as a Substitute for NUnit TestCase

I used to believe that Nunit’s TestCase test (that is, an ability to define a test and then simply pass it alternate parameters) was denied MSTest users. It appears that this is, at least now, fallacious.

The following article implies that this is a recent change:

Taking the MSTest Framework forward with “MSTest V2”

This particular example is in a UWP application:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 12)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Will result in a failing test, and:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 16)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Results in a passing one.

If you want additional information relating to the test, you can use this syntax:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6, DisplayName = "First test")]
        [DataRow(8, 2, 3, 13, DisplayName = "Second test")]
        [DataRow(8, 5, 3, 15, DisplayName = "This will fail")]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Given the constant problems that I have with finding the correct NUnit test adaptor, and trying to work out which are the right libraries, I think, despite coming late to this party, MS might actually drag people back to MSTest with this.

Testing by Interface

Given a standard interface, how to retrieve all implementing classes and run the interface methods.

Unit testing through test driven development is definitely a good idea; but what if you have a number of methods that all effectively do the same thing; that is, each method might do something completely different, but as far as it’s interface goes, it’s identical.

Imagine, for example, that you have a method that calls to the DB, and accepts a number of parameters in, and returns a given parameter. In a unit testing scenario, the DB would be mocked out, and the method called directly from the unit test. Okay, so in this case, you may want some test coverage that your methods call a mocked out DB function, don’t crash, accept a given object, accept null, etc…

Facing the same problem, it occurred to me that it should be possible to write a single test method that would test every existing and future implementation of this, without having to laboriously re-create the test each time I create a method; what’s more, as soon as I create my method name that implements the interface, I get a failing test.

Below is an interface and a test class; it is entirely for the purpose of illustration:

    public class ModelClass
    {
        public string TestProperty { get; set; }
    }


    public interface ITest
    {
        void method1();

        void method2(ModelClass model, int i);
    }

    public class Class1 : ITest
    {
        public void method1()
        {
        }

        public void method2(ModelClass model, int i)
        {
            if (model == null) throw new Exception("test");
            //if (string.IsNullOrWhiteSpace(model.TestProperty)) throw new Exception("Doh");
        }
    }

    public class Class2 : ITest
    {
        public void method1()
        {
            //throw new NotImplementedException();
        }

        public void method2(ModelClass model, int i )
        {
            
        }
    }

    public class Class3
    {
        public void  NonInterfaceMethod(ModelClass model)
        {
            throw new Exception("Doh!");
        }
    }

    public class Class4 : ITest
    {
        public void method1()
        {
            //throw new NotImplementedException();
        }

        public void method2(ModelClass model, int i)
        {
            
        }

        public void test()
        {

        }
    }

 

As you can see, there are a number of interface, and non-interface methods here. There’s nothing particularly interesting, although have a look at Class1.method2(), which should do nothing, but switching the statements should cause a runtime error, if my method works. Also, have a look at Class3.NonInterfaceMethod() – this should never be called, but will throw an exception if it is.

The following is the test code:

        [TestMethod]
        public void TestITestImplementations()
        {
            // Use reflection to get the available methods for the interface
            Type desiredType = typeof(ITest);
            Assembly assembly = desiredType.Assembly;
            var interfaceMethods = desiredType.GetMethods();
            
            // Iterate through each implementation of the interface
            foreach (Type type in assembly.GetTypes())
            {
                if (desiredType.IsAssignableFrom(type) && !type.IsInterface)
                {
                    // Where an implementation is found, instantiate it 
                    // and build a list of available methods to call
                    var classInstance = Activator.CreateInstance(type, null);
                    var methods = type.GetMethods()
                        .Where(m => interfaceMethods.Any(i => i.Name == m.Name)
                            && m.IsPublic
                            && !m.DeclaringType.Equals(typeof(object)));
                    foreach(var method in methods)
                    {
                        // Establish the available parameters and pass them to the call
                        var p = method.GetParameters();
                        object[] p2 = p.Select(a => Activator.CreateInstance(a.ParameterType)).ToArray();

                        try
                        {
                            // Call the method and, where a value should be returned, ensure one is
                            object result = method.Invoke(classInstance, p2);
                            Assert.IsFalse(method.ReturnType != typeof(void) && result == null);
                        }
                        catch(Exception ex)
                        {
                            // Where an error is thrown, print a sensible error
                            Assert.Fail("Call failed: {0}.{1}\nException: {2}", 
                                type.Name, method.Name, ex);
                        }
                    }
                }
            }            
        }

The code above is relatively straight-forward and, if I’m being honest, is only a cursory test. It tests that the methods can be called without throwing an error and, where a value should be returned, checks that it’s not null.

Obviously, it might be perfectly valid that it is null, or an exception might be the desired behaviour. This code is basically just a starting point, but it does provide some very basic test coverage where otherwise, there might be none.

It is also true to say that the code doesn’t deal with overloads, which are not necessary in my particular circumstance.

Unit Tests Are Not Discoverable

I recently had a situation where I loaded a solution containing a suite of NUnit tests, but the test explorer would not recognise them. The following is a series of checks to make that may cause unit tests to be not visible. Most of these are applicable to all tests:

1. Tests must be declared as public. For example:

        public void MyTestMethod()
        {

2. Tests must be decorated with a test attribute.

For NUnit this is is [Test]:

	        [Test]
	        public void MyNUnitTest()
	        {
	

Or [TestCase]:

	        [TestCase(1)]
	        [TestCase(2)]
	        public void MyParameterisedTest(int testNum)
	        {
	
	

For MSTest this is [TestMethod]:

	        [TestMethod]
	        public void MyTestMethod()
	        {
	

3. If using NUnit – check that the correct version is installed (remember that v3 is not an official release yet – that is, at the time of writing).

The current release test adapter is here

The test adaptor for NUnit3 is here

As usual, this is more for my own reference, but if it helps anyone else then all to the good. Also, if you think of or encounter another then please let me know in the comments and I’ll add it on.

MVVM Cross – Stubbing out IMvxMainThreadDispatcher with RhinoMocks

This article describes how to stub out the IMvxMainThreadDispatcher, used by MVVM Cross, using Rhino Mocks.

Here is an excellent article on unit testing in MVVM Cross.

In it, Stuart Lodge describes a manual mock to replace the `IMvxMainThreadDispatcher`. I’ve recently started using RhinoMocks again, and the following is basically the manual mock described in the above article, in RhinoMocks:

var mainThreadDispatcher = MockRepository.GenerateMock<IMvxMainThreadDispatcher>();
mainThreadDispatcher.Stub(x => x.RequestMainThreadAction(Arg<Action>.Is.Anything)).WhenCalled(a => ((Action)a.Arguments.First())()).Return(true);
Mvx.RegisterSingleton<IMvxMainThreadDispatcher>(mainThreadDispatcher);            

As with many of my posts, this is predominantly for my own use; and, as with those other posts, I hope you’ll find this useful.