Tag Archives: Unit Testing

NUnit TestCaseSource

While working on this project, I found a need to abstract away a base type that the unit tests use (in this instance, it was a queue type). I was only testing a single type (PriorityQueue); however, I wanted to create a new type, but all the basic tests for the new type are the same as the existing ones. This led me to investigate the TestCaseSource attribute in NUnit.

As a result, I needed a way to re-use the tests. There are definitely multiple ways to do this; the simplest one is probably to create a factory class, and pass in a string parameter. The only thing that put me off this is that you end up with the following test case:

        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string first, string last, params string[] queueItems)
        {

Becoming:

        [TestCase("PriorityQueue", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue2", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue", "a1", "a", "a1", "b", "c", "d", "a"]
        [TestCase("PriorityQueue2", "a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string queueType, string first, string last, params string[] queueItems)
        {

This isn’t very scaleable when adding a third or fourth type.

TestCaseSource

It turns out that the (or an least an) answer to this is to use NUnit’s TestCaseSource attribute. The NUnit code base dog foods quite extensively, so that is not a bad place to look for examples of how this works; however, what I couldn’t find was a way to mix and match. To better illustrate the point; here’s the first test that I changed to use TestCaseSource:

        [Test]
        public void Queue_NoEntries_CheckCount()
        {
            // Arrange
            PQueue.PriorityQueue<string> queue = new PQueue.PriorityQueue<string>();

            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

Which became:

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        public void Queue_NoEntries_CheckCount(IQueue<string> queue)
        {
            // Arrange


            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

(For completeness, the TestableQueueItemFactory is here):

    public static class TestableQueueItemFactory
    {
        public static IEnumerable<IQueue<string>> ReturnQueueTypes()
        {
            yield return new PQueue.PriorityQueue<string>();
        }
    }

However, when you have a TestCase like the one above, there’s a need for the equivalent of this (which doesn’t work):

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9")]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a")]
        public void Queue_Dequeue_CheckResultOrdering(string first, string last, params string[] queueItems)
        {

A quick look at the NUnit code base reveals these attributes to be mutually exclusive.

Compromise

By no means is this a perfect solution, but the one that I settled on was to create a second TestCaseSource helper method, which looks like this (along with the test):

        private static IEnumerable Queue_Dequeue_CheckResultOrdering_TestCase()
        {
            foreach(var queueType in TestableQueueItemFactory.ReturnQueueTypes())
            {
                yield return new object[] { queueType, "test", "test9", new string[] { "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9" } };
                yield return new object[] { queueType, "a1", "a", new string[] { "a1", "b", "c", "d", "a" } };
            }
        }

        [Test, TestCaseSource("Queue_Dequeue_CheckResultOrdering_TestCase")]
        public void Queue_Dequeue_CheckResultOrdering(
            IQueue <string> queue, string first, string last, params string[] queueItems)
        {

As you can see, the second helper method doesn’t really help readability, so it’s certainly not a perfect solution; in fact, with a single queue type, this makes the code more complex and less readable. However, When a second and third queue type are introduced, the test suddenly becomes resilient.

YAGNI

At first glance, this may appear to be an example of YAGNI. However, in this article, Martin Fowler does state:

Yagni only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.

Which, I believe, is what we are doing here.

References

http://www.smaclellan.com/posts/parameterized-tests-made-simple/

http://stackoverflow.com/questions/16346903/how-to-use-multiple-testcasesource-attributes-for-an-n-unit-test

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

http://dotnetgeek.tumblr.com/post/2851360238/exploiting-nunit-attributes-valuesourceattribute

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

Check table exists in SQLite (in a UWP app)

I’ve recently started investigating the use of SQLite for the purpose of storing data in my UWP application. Before I start, the following are the best tutorials that I could find on the net to get started with SQLite:

http://blogs.u2u.be/diederik/post/2015/09/08/Using-SQLite-on-the-Universal-Windows-Platform.aspx

https://blogs.windows.com/buildingapps/2016/05/03/data-access-in-universal-windows-platform-uwp-apps/

The SQLite home page is here.

Finally, the nuget package to integrate SQLIte with your UWP app is here.

I didn’t find too much information on checking if a table exists, and I think it’s quite a key thing to be able to do, as you can’t really have a deployment script that runs when you deploy your UWP app. As a result I create a base data access class. Let’s start with the interface:

public interface IDataAccess
{
    void Connect();
    bool DoesTableExist<T>() where T : class;
    bool RemoveTable<T>() where T : class;

    bool CreateTable<T>() where T : class;
}

Next, I created a unit test to make sure that the method will work:

    [TestClass]
    public class TestBaseMethods
    {       
        [DataTestMethod]
        [DataRow("MyApp.DataAccess.SQLLiteDataAccess, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
        public void TablesCreate(string dataAccess)
        {
            // Arrange            
            IDataAccess bda = Activator.CreateInstance(Type.GetType(dataAccess)) as IDataAccess;
            bda.Connect();

            Assert.AreEqual(false, bda.DoesTableExist<TestEntity>());

            // Act
            bda.CreateTable<TestEntity>();

            // Assert
            Assert.AreEqual(true, bda.DoesTableExist<TestEntity>());


            // Clean-up
            bda.RemoveTable<TestEntity>();
            Assert.AreEqual(false, bda.DoesTableExist<TestEntity>());
        }

If you want more information about DataRow, then see my recent post

Implementation

We’ll start with a basic class that provides a Connect method and cleans up after itself:

namespace MyApp.DataAccess
{
    public class SQLLiteDataAccess : IDisposable, IDataAccess
    {
        SQLiteConnection _connection;

        public void Connect()
        {
            _connection = new SQLiteConnection(new SQLitePlatformWinRT(),
                Path.Combine(ApplicationData.Current.LocalFolder.Path, "Storage.sqlite"));
        }

        public void Dispose()
        {
            _connection.Dispose();
        }

Then we need to add the functionality from the interface:

        /// <summary>
        /// Generic method to determine is an object type exists within the DB
        /// </summary>
        /// <param name="type"></param>
        /// <param name="name"></param>
        /// <returns></returns>
        private bool DoesTypeExist(string type, string name)
        {
            SQLiteCommand command = _connection.CreateCommand("SELECT COUNT(1) FROM SQLITE_MASTER WHERE TYPE = @TYPE AND NAME = @NAME");
            command.Bind("@TYPE", type);
            command.Bind("@NAME", name);
            
            int result = command.ExecuteScalar<int>();

            return (result > 0);
        }

        /// <summary>
        /// Check whether a specific table exists in the database
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <returns>true if the specified table exists</returns>
        public bool DoesTableExist<T>() where T : class
        {
            bool exists = DoesTypeExist("table", typeof(T).Name);
            return exists;
        }

        /// <summary>
        /// Create a new table in the DB
        /// </summary>
        /// <typeparam name="T">Object to base the table on</typeparam>
        /// <returns></returns>
        public bool CreateTable<T>() where T : class
        {
            bool exists = DoesTableExist<T>();

            if (!exists)
            {
                int error = _connection.CreateTable<T>();
                exists = (error == 0);
            }

            return exists;
        }

        /// <summary>
        /// Remove / drop the specified table from the DB
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <returns></returns>
        public bool RemoveTable<T>() where T : class
        {
            bool exists = DoesTableExist<T>();

            if (exists)
            {
                int error = _connection.DropTable<T>();
                exists = (error != 0);
            }

            return exists;
        }

That’s it; I’m still not sure how I feel about SQLite. For a small, local storage, it feels like it could be an XML file; but I suppose it comes into its own when we have a potentially large local storage.

Using MSTest DataRow as a Substitute for NUnit TestCase

I used to believe that Nunit’s TestCase test (that is, an ability to define a test and then simply pass it alternate parameters) was denied MSTest users. It appears that this is, at least now, fallacious.

The following article implies that this is a recent change:

Taking the MSTest Framework forward with “MSTest V2”

This particular example is in a UWP application:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 12)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Will result in a failing test, and:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 16)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Results in a passing one.

If you want additional information relating to the test, you can use this syntax:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6, DisplayName = "First test")]
        [DataRow(8, 2, 3, 13, DisplayName = "Second test")]
        [DataRow(8, 5, 3, 15, DisplayName = "This will fail")]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Given the constant problems that I have with finding the correct NUnit test adaptor, and trying to work out which are the right libraries, I think, despite coming late to this party, MS might actually drag people back to MSTest with this.

Testing by Interface

Given a standard interface, how to retrieve all implementing classes and run the interface methods.

Unit testing through test driven development is definitely a good idea; but what if you have a number of methods that all effectively do the same thing; that is, each method might do something completely different, but as far as it’s interface goes, it’s identical.

Imagine, for example, that you have a method that calls to the DB, and accepts a number of parameters in, and returns a given parameter. In a unit testing scenario, the DB would be mocked out, and the method called directly from the unit test. Okay, so in this case, you may want some test coverage that your methods call a mocked out DB function, don’t crash, accept a given object, accept null, etc…

Facing the same problem, it occurred to me that it should be possible to write a single test method that would test every existing and future implementation of this, without having to laboriously re-create the test each time I create a method; what’s more, as soon as I create my method name that implements the interface, I get a failing test.

Below is an interface and a test class; it is entirely for the purpose of illustration:

    public class ModelClass
    {
        public string TestProperty { get; set; }
    }


    public interface ITest
    {
        void method1();

        void method2(ModelClass model, int i);
    }

    public class Class1 : ITest
    {
        public void method1()
        {
        }

        public void method2(ModelClass model, int i)
        {
            if (model == null) throw new Exception("test");
            //if (string.IsNullOrWhiteSpace(model.TestProperty)) throw new Exception("Doh");
        }
    }

    public class Class2 : ITest
    {
        public void method1()
        {
            //throw new NotImplementedException();
        }

        public void method2(ModelClass model, int i )
        {
            
        }
    }

    public class Class3
    {
        public void  NonInterfaceMethod(ModelClass model)
        {
            throw new Exception("Doh!");
        }
    }

    public class Class4 : ITest
    {
        public void method1()
        {
            //throw new NotImplementedException();
        }

        public void method2(ModelClass model, int i)
        {
            
        }

        public void test()
        {

        }
    }

 

As you can see, there are a number of interface, and non-interface methods here. There’s nothing particularly interesting, although have a look at Class1.method2(), which should do nothing, but switching the statements should cause a runtime error, if my method works. Also, have a look at Class3.NonInterfaceMethod() – this should never be called, but will throw an exception if it is.

The following is the test code:

        [TestMethod]
        public void TestITestImplementations()
        {
            // Use reflection to get the available methods for the interface
            Type desiredType = typeof(ITest);
            Assembly assembly = desiredType.Assembly;
            var interfaceMethods = desiredType.GetMethods();
            
            // Iterate through each implementation of the interface
            foreach (Type type in assembly.GetTypes())
            {
                if (desiredType.IsAssignableFrom(type) && !type.IsInterface)
                {
                    // Where an implementation is found, instantiate it 
                    // and build a list of available methods to call
                    var classInstance = Activator.CreateInstance(type, null);
                    var methods = type.GetMethods()
                        .Where(m => interfaceMethods.Any(i => i.Name == m.Name)
                            && m.IsPublic
                            && !m.DeclaringType.Equals(typeof(object)));
                    foreach(var method in methods)
                    {
                        // Establish the available parameters and pass them to the call
                        var p = method.GetParameters();
                        object[] p2 = p.Select(a => Activator.CreateInstance(a.ParameterType)).ToArray();

                        try
                        {
                            // Call the method and, where a value should be returned, ensure one is
                            object result = method.Invoke(classInstance, p2);
                            Assert.IsFalse(method.ReturnType != typeof(void) && result == null);
                        }
                        catch(Exception ex)
                        {
                            // Where an error is thrown, print a sensible error
                            Assert.Fail("Call failed: {0}.{1}\nException: {2}", 
                                type.Name, method.Name, ex);
                        }
                    }
                }
            }            
        }

The code above is relatively straight-forward and, if I’m being honest, is only a cursory test. It tests that the methods can be called without throwing an error and, where a value should be returned, checks that it’s not null.

Obviously, it might be perfectly valid that it is null, or an exception might be the desired behaviour. This code is basically just a starting point, but it does provide some very basic test coverage where otherwise, there might be none.

It is also true to say that the code doesn’t deal with overloads, which are not necessary in my particular circumstance.

Unit Tests Are Not Discoverable

I recently had a situation where I loaded a solution containing a suite of NUnit tests, but the test explorer would not recognise them. The following is a series of checks to make that may cause unit tests to be not visible. Most of these are applicable to all tests:

1. Tests must be declared as public. For example:

        public void MyTestMethod()
        {

2. Tests must be decorated with a test attribute.

For NUnit this is is [Test]:

	        [Test]
	        public void MyNUnitTest()
	        {
	

Or [TestCase]:

	        [TestCase(1)]
	        [TestCase(2)]
	        public void MyParameterisedTest(int testNum)
	        {
	
	

For MSTest this is [TestMethod]:

	        [TestMethod]
	        public void MyTestMethod()
	        {
	

3. If using NUnit – check that the correct version is installed (remember that v3 is not an official release yet – that is, at the time of writing).

The current release test adapter is here

The test adaptor for NUnit3 is here

As usual, this is more for my own reference, but if it helps anyone else then all to the good. Also, if you think of or encounter another then please let me know in the comments and I’ll add it on.

MVVM Cross – Stubbing out IMvxMainThreadDispatcher with RhinoMocks

This article describes how to stub out the IMvxMainThreadDispatcher, used by MVVM Cross, using Rhino Mocks.

Here is an excellent article on unit testing in MVVM Cross.

In it, Stuart Lodge describes a manual mock to replace the `IMvxMainThreadDispatcher`. I’ve recently started using RhinoMocks again, and the following is basically the manual mock described in the above article, in RhinoMocks:

var mainThreadDispatcher = MockRepository.GenerateMock<IMvxMainThreadDispatcher>();
mainThreadDispatcher.Stub(x => x.RequestMainThreadAction(Arg<Action>.Is.Anything)).WhenCalled(a => ((Action)a.Arguments.First())()).Return(true);
Mvx.RegisterSingleton<IMvxMainThreadDispatcher>(mainThreadDispatcher);            

As with many of my posts, this is predominantly for my own use; and, as with those other posts, I hope you’ll find this useful.