Tag Archives: Unit Test

Short Walks – XUnit Warning

As with many of these posts – this is more of a “note to self”.

Say you have an assertion that looks something like this in your Xunit test:

Assert.True(myEnumerable.Any(a => a.MyValue == "1234"));

In later versions (not sure exactly which one this was introduced it), you’ll get the following warning:

warning xUnit2012: Do not use Enumerable.Any() to check if a value exists in a collection.

So, Xunit has a nice little feature where you can use the following syntax instead:

Assert.Contains(myEnumerable, a => a.MyValue == "1234");

Short Walks – NSubstitute – Subclassing and Partial Substitutions

I’ve had this issue a few times recently. Each time I have it, after I’ve worked out what it was, it makes sense, but I keep running into it. The resulting frustration is this post – that way, it’ll come up the next time I search for it on t’internet.

The Error

The error is as follows:

“NSubstitute.Exceptions.CouldNotSetReturnDueToNoLastCallException: ‘Could not find a call to return from.”

Typically, it seems to occur in one of two circumstances: substituting a concrete class and partially substituting a concrete class; that is:

var myMock = Substitute.ForPartsOf<MyClass>();

Or:

var myMock = Substitute.For<MyClass>();

Why?

If you were to manually mock out an interface, how would you do that? Well, say you had IMyClass, you’d just do something like this:

public class MyClassMock : IMyClass 
{
	// New and imaginative behaviour goes here
}

All’s good – you get a brand new implementation of the interface, and you can do anything with it. But what would you do if you were trying to unit test a method inside MyClass that called another method inside MyClass; for example:

public class MyClass : IMyClass
{
	public bool Method1()
{
		int rowCount = ReadNumberOfRowsInFileOnDisk();
		Return rowCount > 10;
	}
	
	public int ReadNumberOfRowsInFileOnDisk()
	{
		// Opens a file, reads it, and returns the number of rows
	}
}

(Let’s not get bogged down in how realistic this scenario is, or whether or not this is good practice – it illustrates a point)

If you want to unit test Method1(), but don’t want to actually read a file from the disk, you’ll want to replace ReadNumberOfRowsInFileOnDisk(). The only real way that you can do this is to subclass the class; for example:

public class MyClassMock : MyClass

You can now test the behaviour on MyClass, via MyClassMock; however, you still can’t* override the method ReadNumberOfRowsInFileOnDisk() because it isn’t virtual. If you make the method virtual, you can override it in the subclass.

The same is true with NSubstitute – if you want to partially mock a class in this way, it follows the same rules as you must if you would to roll your own.

Footnotes

* There may, or may not be one or two ways to get around this restriction, but let’s at least agree that they are, at best, unpleasant.

NUnit TestCaseSource

While working on this project, I found a need to abstract away a base type that the unit tests use (in this instance, it was a queue type). I was only testing a single type (PriorityQueue); however, I wanted to create a new type, but all the basic tests for the new type are the same as the existing ones. This led me to investigate the TestCaseSource attribute in NUnit.

As a result, I needed a way to re-use the tests. There are definitely multiple ways to do this; the simplest one is probably to create a factory class, and pass in a string parameter. The only thing that put me off this is that you end up with the following test case:

        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string first, string last, params string[] queueItems)
        {

Becoming:

        [TestCase("PriorityQueue", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue2", "test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9"]
        [TestCase("PriorityQueue", "a1", "a", "a1", "b", "c", "d", "a"]
        [TestCase("PriorityQueue2", "a1", "a", "a1", "b", "c", "d", "a"]
        public void Queue_Dequeue_CheckResultOrdering(
            string queueType, string first, string last, params string[] queueItems)
        {

This isn’t very scaleable when adding a third or fourth type.

TestCaseSource

It turns out that the (or an least an) answer to this is to use NUnit’s TestCaseSource attribute. The NUnit code base dog foods quite extensively, so that is not a bad place to look for examples of how this works; however, what I couldn’t find was a way to mix and match. To better illustrate the point; here’s the first test that I changed to use TestCaseSource:

        [Test]
        public void Queue_NoEntries_CheckCount()
        {
            // Arrange
            PQueue.PriorityQueue<string> queue = new PQueue.PriorityQueue<string>();

            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

Which became:

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        public void Queue_NoEntries_CheckCount(IQueue<string> queue)
        {
            // Arrange


            // Act
            int count = queue.Count();

            // Assert
            Assert.AreEqual(0, count);
        }

(For completeness, the TestableQueueItemFactory is here):

    public static class TestableQueueItemFactory
    {
        public static IEnumerable<IQueue<string>> ReturnQueueTypes()
        {
            yield return new PQueue.PriorityQueue<string>();
        }
    }

However, when you have a TestCase like the one above, there’s a need for the equivalent of this (which doesn’t work):

        [Test, TestCaseSource(typeof(TestableQueueItemFactory), "ReturnQueueTypes")]
        [TestCase("test", "test9", "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9")]
        [TestCase("a1", "a", "a1", "b", "c", "d", "a")]
        public void Queue_Dequeue_CheckResultOrdering(string first, string last, params string[] queueItems)
        {

A quick look at the NUnit code base reveals these attributes to be mutually exclusive.

Compromise

By no means is this a perfect solution, but the one that I settled on was to create a second TestCaseSource helper method, which looks like this (along with the test):

        private static IEnumerable Queue_Dequeue_CheckResultOrdering_TestCase()
        {
            foreach(var queueType in TestableQueueItemFactory.ReturnQueueTypes())
            {
                yield return new object[] { queueType, "test", "test9", new string[] { "test", "test2", "test3", "test4", "test5", "test6", "test7", "test8", "test9" } };
                yield return new object[] { queueType, "a1", "a", new string[] { "a1", "b", "c", "d", "a" } };
            }
        }

        [Test, TestCaseSource("Queue_Dequeue_CheckResultOrdering_TestCase")]
        public void Queue_Dequeue_CheckResultOrdering(
            IQueue <string> queue, string first, string last, params string[] queueItems)
        {

As you can see, the second helper method doesn’t really help readability, so it’s certainly not a perfect solution; in fact, with a single queue type, this makes the code more complex and less readable. However, When a second and third queue type are introduced, the test suddenly becomes resilient.

YAGNI

At first glance, this may appear to be an example of YAGNI. However, in this article, Martin Fowler does state:

Yagni only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.

Which, I believe, is what we are doing here.

References

http://www.smaclellan.com/posts/parameterized-tests-made-simple/

http://stackoverflow.com/questions/16346903/how-to-use-multiple-testcasesource-attributes-for-an-n-unit-test

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

http://dotnetgeek.tumblr.com/post/2851360238/exploiting-nunit-attributes-valuesourceattribute

https://github.com/nunit/docs/wiki/TestCaseSource-Attribute

Testing for Exceptions using the Arrange Act Assert Pattern in C# 7

Unit testing massively benefits from following the Arrange / Act / Assert pattern. I’ve seen tests that are not written in this way, and they can be sprawling and indecipherable, either testing many different things in series, or testing nothing at all except the .Net Framework.

I recently found an issue while trying to test for an exception being thrown, which is that Nunit (and probably other frameworks) test for an exception by accepting a delegate to test. Here’s an example:

        [Test]
        public void Test_ThrowException_ExceptionThrown()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act / Assert
            Assert.Throws(typeof(Exception), tc.ThrowException);
        }

We’re just testing a dummy class:

    public class TestClass
    {
        public void ThrowException()
        {
            throw new Exception("MyException");
        }
    }

C# 7 – Inline functions

If you look in the references at the bottom, you’ll see something more akin to this approach:

        public void Test_ThrowException_ExceptionThrown2()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act
            TestDelegate throwException = () => tc.ThrowException();            

            // Assert
            Assert.Throws(typeof(Exception), throwException);
        }

However, since C# 7, the option on a local function has arrived. The following has the same effect:

        [Test]
        public void Test_ThrowException_ExceptionThrown3()
        {
            // Arrange
            TestClass tc = new TestClass();

            // Act
            void CallThrowException()
            {
                tc.ThrowException();
            }

            // Assert
            Assert.Throws(typeof(Exception), CallThrowException);
        }

I think that I, personally, still prefer the anonymous function for this; however, the local function does present some options; for example:

        [Test]
        public void Test_ThrowException_ExceptionThrown4()
        {
            void CallThrowException()
            {
                // Arrange
                TestClass tc = new TestClass();

                // Act
                tc.ThrowException();
            }

            // Assert
            Assert.Throws(typeof(Exception), CallThrowException);
        }

Now I’m not so sure that I still prefer the anonymous function.

References

http://stackoverflow.com/questions/33897323/nunit-3-0-and-assert-throws

https://pmbanugo.wordpress.com/2014/06/16/exception-testing-pattern/

http://stackoverflow.com/questions/24070115/best-approach-towards-applying-the-arrange-act-assert-pattern-when-expecting-exc

Using MSTest DataRow as a Substitute for NUnit TestCase

I used to believe that Nunit’s TestCase test (that is, an ability to define a test and then simply pass it alternate parameters) was denied MSTest users. It appears that this is, at least now, fallacious.

The following article implies that this is a recent change:

Taking the MSTest Framework forward with “MSTest V2”

This particular example is in a UWP application:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 12)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Will result in a failing test, and:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 16)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Results in a passing one.

If you want additional information relating to the test, you can use this syntax:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6, DisplayName = "First test")]
        [DataRow(8, 2, 3, 13, DisplayName = "Second test")]
        [DataRow(8, 5, 3, 15, DisplayName = "This will fail")]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Given the constant problems that I have with finding the correct NUnit test adaptor, and trying to work out which are the right libraries, I think, despite coming late to this party, MS might actually drag people back to MSTest with this.

Generic Method to Clear a Class and Intelli-Test

Recently, I published this article on copying a class dynamically. I then found that I could use the same approach to clear a class. Here’s the method:

        private static void ClearClass<T>(T classToClear)
        {
            if (classToClear == null)
                throw new Exception("Must not specify null parameters");

            var properties = classToClear.GetType().GetProperties();

            foreach (var p in properties.Where(prop => prop.CanWrite))
            {
                p.SetValue(classToClear, GetDefault(p.PropertyType));
            }
        }

        /// <summary>
        /// Taken from http://stackoverflow.com/questions/474841/dynamically-getting-default-of-a-parameter-type
        /// </summary>
        /// <param name="type"></param>
        /// <returns></returns>
        public static object GetDefault(Type type)
        {
            return type.IsValueType ? Activator.CreateInstance(type) : null;
        }

As you can see, I had a little help from Jon Skeet with this one. Once I’d written it, I thought I’d have a play with the IntelliTest feature: if you right click the method and select “Create IntelliTest”, you’re presented with this:

IT1

IT2

It generated this:

    /// <summary>This class contains parameterized unit tests for Program</summary>
    [PexClass(typeof(Program))]
    [PexAllowedExceptionFromTypeUnderTest(typeof(InvalidOperationException))]
    [PexAllowedExceptionFromTypeUnderTest(typeof(ArgumentException), AcceptExceptionSubtypes = true)]
    [TestClass]
    public partial class ProgramTest
    {
        /// <summary>Test stub for ClearClass(!!0)</summary>
        [PexGenericArguments(typeof(int))]
        [PexMethod]
        internal void ClearClassTest<T>(T classToClear)
        {
            Program.ClearClass<T>(classToClear);
            // TODO: add assertions to method ProgramTest.ClearClassTest(!!0)
        }
    }

The interesting thing about this, is that it can’t be found as a test. What actually happens is this creates an intelli-test, which, as far as I can see, you have to right-click on the created test and select “Run Intelli-test”. This then creates you an actual unit test:

IT3

It looks something like this:

namespace ConsoleApplication13.Tests
{
    public partial class ProgramTest
    {

[TestMethod]
[PexGeneratedBy(typeof(ProgramTest))]
public void ClearClassTest861()
{
    this.ClearClassTest<int>(0);
}
    }
}

That then can be found and run:

IT4

Obviously, looking at the unit test, it’s not a particularly good one; it effectively tests that your code doesn’t crash, so it increases your code coverage, but doesn’t really test anything per-se.

Unit Tests Are Not Discoverable

I recently had a situation where I loaded a solution containing a suite of NUnit tests, but the test explorer would not recognise them. The following is a series of checks to make that may cause unit tests to be not visible. Most of these are applicable to all tests:

1. Tests must be declared as public. For example:

        public void MyTestMethod()
        {

2. Tests must be decorated with a test attribute.

For NUnit this is is [Test]:

	        [Test]
	        public void MyNUnitTest()
	        {
	

Or [TestCase]:

	        [TestCase(1)]
	        [TestCase(2)]
	        public void MyParameterisedTest(int testNum)
	        {
	
	

For MSTest this is [TestMethod]:

	        [TestMethod]
	        public void MyTestMethod()
	        {
	

3. If using NUnit – check that the correct version is installed (remember that v3 is not an official release yet – that is, at the time of writing).

The current release test adapter is here

The test adaptor for NUnit3 is here

As usual, this is more for my own reference, but if it helps anyone else then all to the good. Also, if you think of or encounter another then please let me know in the comments and I’ll add it on.

MVVM Cross – Stubbing out IMvxMainThreadDispatcher with RhinoMocks

This article describes how to stub out the IMvxMainThreadDispatcher, used by MVVM Cross, using Rhino Mocks.

Here is an excellent article on unit testing in MVVM Cross.

In it, Stuart Lodge describes a manual mock to replace the `IMvxMainThreadDispatcher`. I’ve recently started using RhinoMocks again, and the following is basically the manual mock described in the above article, in RhinoMocks:

var mainThreadDispatcher = MockRepository.GenerateMock<IMvxMainThreadDispatcher>();
mainThreadDispatcher.Stub(x => x.RequestMainThreadAction(Arg<Action>.Is.Anything)).WhenCalled(a => ((Action)a.Arguments.First())()).Return(true);
Mvx.RegisterSingleton<IMvxMainThreadDispatcher>(mainThreadDispatcher);            

As with many of my posts, this is predominantly for my own use; and, as with those other posts, I hope you’ll find this useful.

Mock Current Date and Time in SQL Server

Occasionally, if you’re especially lucky, you’ll get into a situation where you have SQL procedures and functions that are so complicated that they require unit tests all of their own. They’ll have business logic embedded in them, and not testing them will leave a massive hole in your test coverage.

In this blog post I’m not going to describe how to do that – SSDT are quite well documented anyway. This is about how to deal with dates and times in SQL Server.

A new function

You’ll probably have a few places in your SQL script that call the following:

SELECT GETUTCDATE()

Or, you may even have the following:

SELECT GETDATE()

Which will presumably work well for what you want. Of course, the problem that you have here is, that for unit tests, this presents a variable factor in your test; that is, you’re not always testing the same procedure. Take the following segments of SQL for example:

PROCEDURE MYPROC
AS
BEGIN
	DECLARE @today DATETIME
	DECLARE @hasEntriesAfterToday INT
	
	SET @today = GETUTCDATE()

	SELECT @hasEntriesAfterToday = COUNT(*)
	FROM dbo.MyTable t
	WHERE t.Col1 > @today

	IF (@hasEntriesAfterToday > 0) 	
		select 'test'
END
GO

MyTable contains many entries after today, and my test checks that it returns ‘test’, so the test works, the code works and I’m going to bed.

But what happens in a year’s time?

Let’s say that the last entity in that table is 01/01/2015 (that way the post works in the US, too). As I write this, it is mid-way through June. So, I need to know what will happen on 2nd January 2015. If I do nothing then when it is 2nd January 2015 the test will start to fail, and I won’t know why.

Abstract the date

When faced with this problem, my initial fix was as follows:

CREATE FUNCTION dbo.MyGetDate()
RETURNS DATETIME
AS
BEGIN
	DECLARE @today DATETIME
	SET @today = GETUTCDATE()

	RETURN @today
END

And then simply change the above procedure to call this. That certainly works; however, as soon as you start to reference this function (for example, you set a default value for a date in a table), you’ll find that you’ll get stuck when you try to mock it out; consequently, you need a double layer:

CREATE FUNCTION dbo.MyGetDate2()
RETURNS DATETIME
AS
BEGIN
	DECLARE @today DATETIME	
	SET @today = GETUTCDATE()

	RETURN @today
END
GO

CREATE FUNCTION dbo.MyGetDate()
RETURNS DATETIME
AS
BEGIN
	DECLARE @today DATE	
	SET @today = dbo.MyGetDate2()

	RETURN @today
END
GO

What this then allows you to do is to replace the function of MyGetDate2 without affecting MyGetDate. This is a wrapper function to replace the DateTime:

internal static void OverrideDateTimeTest(SqlConnection cn, SqlTransaction tr, string newDateTime)
{
    string sql =
        "ALTER FUNCTION dbo.MyGetDate2(	" +
        ") RETURNS datetime " +
        "AS " +
        "BEGIN " +
        "DECLARE @value datetime " +
        "SET @value = convert(datetime, '" + newDateTime + "') "  +
        "RETURN @value " +
        "END";
 
    using (SqlCommand cmd = new SqlCommand(sql, cn, tr))
    {
        cmd.Connection = cn;
        cmd.CommandType = CommandType.Text;
        cmd.ExecuteNonQuery();
    }
}

And here’s the test:

[TestMethod]
public void MyTest()
{
    DBWrapper.OverrideDateTimeTest(cn, tr, "2014-06-10 22:30:00.000");
    Assert.AreEqual( …

The best part about this is that IN SQL SERVER DDL STATEMENTS CAN BE ROLLED BACK! Look at the following test:

-- 1
BEGIN TRAN
GO

-- 2
SELECT dbo.MyGetDate()
GO

-- 3
ALTER FUNCTION [dbo].[MyGetDate2]()
RETURNS DATETIME
AS
BEGIN
	DECLARE @today DATETIME
	SET @today = GETUTCDATE()

	RETURN @today
END
GO

-- 4
SELECT dbo.MyGetDate()
GO

-- 5
ALTER FUNCTION [dbo].[MyGetDate2]()
RETURNS DATETIME
AS
BEGIN
	DECLARE @today DATETIME
	SET @today = CONVERT(DATETIME, '2014-06-10 22:30:00.000')

	RETURN @today
END
GO

-- 6
SELECT dbo.MyGetDate()
GO

-- 7
ROLLBACK TRAN
GO

-- 8
SELECT dbo.MyGetDate()
GO

Okay – there’s a fair amount of code, but the stages are as follows (numbered):

1. Start the transaction.
2. Show the existing implementation of MyGetDate2 (in case it’s not what it should be).
3. Change MyGetDate2 to use GetUTCDate(), so it should be the same as before.
4. Check again – should still return the same as 2.
5. Change MgGetDate2 to return hard coded date.
6. Check that it now returns a hard coded date.
7. Rollback the transaction.
8. The transaction is rolled back, and so the function behaves as in 1.

Conclusion

So, we can include a date mock in our test and, should there be a problem, or when we’re finished, it all gets rolled back. Just because I’m always cautious about such things, I’ve created a test that checks that the default implementation returns the current date, but you shouldn’t need this.