Tag Archives: TestMethod

Using MSTest DataRow as a Substitute for NUnit TestCase

I used to believe that Nunit’s TestCase test (that is, an ability to define a test and then simply pass it alternate parameters) was denied MSTest users. It appears that this is, at least now, fallacious.

The following article implies that this is a recent change:

Taking the MSTest Framework forward with “MSTest V2”

This particular example is in a UWP application:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 12)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Will result in a failing test, and:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 16)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Results in a passing one.

If you want additional information relating to the test, you can use this syntax:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6, DisplayName = "First test")]
        [DataRow(8, 2, 3, 13, DisplayName = "Second test")]
        [DataRow(8, 5, 3, 15, DisplayName = "This will fail")]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Given the constant problems that I have with finding the correct NUnit test adaptor, and trying to work out which are the right libraries, I think, despite coming late to this party, MS might actually drag people back to MSTest with this.

Intelli-Test (Part 2)

I recently posted an article which morphed into a discovery of the Intelli-Test feature in VS2015.

My initial findings were relating to creating a basic intelli-test, and then having that create a new unit test for me. However, once you’ve created an intelli-test, you can modify it; here’s the original intelli-test that was created for me:

        [PexGenericArguments(typeof(int))]
        [PexMethod]
        internal void ClearClassTest<T>(T classToClear)
        {
            Program.ClearClass<T>(classToClear);
            // TODO: add assertions to method ProgramTest.ClearClassTest(!!0)
        }

When this creates a test, it looks like this:

[TestMethod]
[PexGeneratedBy(typeof(ProgramTest))]
public void ClearClassTest861()
{
    this.ClearClassTest<int>(0);
}

So, I tried adding some additional parameters:

    class TestClass
    {
        public string Test1 { get; set; }
        public string Test2 { get; set; }
    }

    /// <summary>This class contains parameterized unit tests for Program</summary>
    [PexClass(typeof(Program))]
    [PexAllowedExceptionFromTypeUnderTest(typeof(InvalidOperationException))]
    [PexAllowedExceptionFromTypeUnderTest(typeof(ArgumentException), AcceptExceptionSubtypes = true)]
    [TestClass]
    public partial class ProgramTest
    {
        /// <summary>Test stub for ClearClass(!!0)</summary>
        [PexGenericArguments(typeof(int))]
        [PexGenericArguments(typeof(string))]
        [PexGenericArguments(typeof(float))]
        [PexGenericArguments(typeof(TestClass))]
        [PexMethod]
        internal void ClearClassTest<T>(T classToClear)
        {
            Program.ClearClass<T>(classToClear);
            // TODO: add assertions to method ProgramTest.ClearClassTest(!!0)
        }
    }

Just re-select “Run intelli-test” and it updates the ProgramTest.ClearClassTest.g.cs, generating 6 new tests. To be honest, this was a bit disappointing. I had expected an “intelligent” test – that is, one that tests several outcomes. To simplify what was happening, I tried creating a simple method:

        public static int TestAdd(int num1, int num2)
        {
            return num1 + num2;
        }

Creating the test for this resulted in this:

        [PexMethod]
        internal int TestAddTest(int num1, int num2)
        {
            int result = Program.TestAdd(num1, num2);
            return result;
            // TODO: add assertions to method ProgramTest.TestAddTest(Int32, Int32)
        }

And then to:

[TestMethod]
[PexGeneratedBy(typeof(ProgramTest))]
public void TestAddTest989()
{
    int i;
    i = this.TestAddTest(0, 0);
    Assert.AreEqual<int>(0, i);
}

So, then I considered what this was actually doing. The purpose of it seems to be to execute code; that is, code coverage; so what happens if I create a method like this:

        public static int TestAdd(int num1, int num2)
        {
            if (num1 > num2)
                return num1 + num2;
            else
                return num2 - num1;
        }

And finally, you see the usefulness of this technology. If it only creates a single test, passing in 0,0 then it only tests one code path. The minimum test to cover all code paths is 0,0 and 1,0. That’s exactly what it does; the generated test looks the same:

        /// <summary>Test stub for TestAdd(Int32, Int32)</summary>
        [PexMethod]
        internal int TestAddTest(int num1, int num2)
        {
            int result = Program.TestAdd(num1, num2);
            return result;
            // TODO: add assertions to method ProgramTest.TestAddTest(Int32, Int32)
        }

But the run intelli-test creates two methods:

[TestMethod]
[PexGeneratedBy(typeof(ProgramTest))]
public void TestAddTest989()
{
    int i;
    i = this.TestAddTest(0, 0);
    Assert.AreEqual<int>(0, i);
}

[TestMethod]
[PexGeneratedBy(typeof(ProgramTest))]
public void TestAddTest365()
{
    int i;
    i = this.TestAddTest(1, 0);
    Assert.AreEqual<int>(1, i);
}

In an effort to confuse the system, I changed the base function:

        public static int TestAdd(int num1, int num2)
        {
            num1++;

            if (num1 == num2)
            {
                throw new Exception("cannot be the same");
            }

            if (num1 > num2)
                return num1 + num2;
            else
                return num2 - num1;
            
        }

And re-run the test; it resulted in 3 tests:

intellitest2

Not sure where the numbers came from, but this tests every code path. The exception results in a fail, but you can mark that as a pass by simply right clicking and selecting “Allow”; which changes the test to look like this:

[TestMethod]
[PexGeneratedBy(typeof(ProgramTest))]
[ExpectedException(typeof(Exception))]
public void TestAddTestThrowsException26601()
{
    int i;
    i = this.TestAddTest(502, 503);
}

Summary

This is basically the opposite of the modern testing philosophy, which is test interactions and behaviour; what this does is establish every code path in a function and run it; so it should establish that the code doesn’t crash, but it obviously can’t establish that it does what it’s intended to do, nor can it establish whether it is sensible to pass, for example, a null value. Having said that, if an object can be null, and there’s no defensive code path to deal with it being passed null, then this will be pointed out.

In general, this is intended for people dealing with, and refactoring, legacy code so that they can adopt the “Stangling Vines” development pattern in order to be sure that the code still does what it did before.

Unit Tests Are Not Discoverable

I recently had a situation where I loaded a solution containing a suite of NUnit tests, but the test explorer would not recognise them. The following is a series of checks to make that may cause unit tests to be not visible. Most of these are applicable to all tests:

1. Tests must be declared as public. For example:

        public void MyTestMethod()
        {

2. Tests must be decorated with a test attribute.

For NUnit this is is [Test]:

	        [Test]
	        public void MyNUnitTest()
	        {
	

Or [TestCase]:

	        [TestCase(1)]
	        [TestCase(2)]
	        public void MyParameterisedTest(int testNum)
	        {
	
	

For MSTest this is [TestMethod]:

	        [TestMethod]
	        public void MyTestMethod()
	        {
	

3. If using NUnit – check that the correct version is installed (remember that v3 is not an official release yet – that is, at the time of writing).

The current release test adapter is here

The test adaptor for NUnit3 is here

As usual, this is more for my own reference, but if it helps anyone else then all to the good. Also, if you think of or encounter another then please let me know in the comments and I’ll add it on.

Resource File Testing

Admittedly this does sound like a strange one. How or why would you both to test a resource file? The only reason I could think of was this: what if someone adds a resource, and forgets to localise it?

Without a test such as this, the first way you’d know this is when you ran the program in a localised culture and spotted the default culture appearing. This is something that could potentially go unnoticed for a long time. Consider British and American English, or Spanish and Latin-American Spanish.

To set-up the test, create three resource files:

res1

Create two resources in the base resource file (.resx):

res2

And then in each localised file, create one of these; so, in Resource.en-GB.resx create testphraseone (test phrase one english), and in the Spanish: testphrasetwo (prueba frase dos).

We also need to expose the resource manager:

    public class GetRes
    {
        public static ResourceManager GetResMgr()
        {            
            ResourceManager resMgr = new ResourceManager("resourcetest.Properties.Resource", typeof(resourcetest.Properties.Resource).Assembly);
            return resMgr;
        }

The Test

Now that we have the ResourceManager, we can simply call the GetResourceSet function. The third parameter determines whether to try the parent if the resource doesn’t exist, so setting this to false will force it to get the resource from the file in question.

The test could look like this:

        [TestMethod]
        public void TestMethod1()
        {
            ResourceManager resMgr = resourcetest.GetRes.GetResMgr();
            
            var baseList = resMgr.GetResourceSet(CultureInfo.InvariantCulture, true, true).Cast<DictionaryEntry>();            
            var spanishList = resMgr.GetResourceSet(new CultureInfo("es-ES"), true, false).Cast<DictionaryEntry>();
            var britishList = resMgr.GetResourceSet(new CultureInfo("en-GB"), true, false).Cast<DictionaryEntry>();
            
            var missing = baseList.Where(m => !spanishList.Any(s => string.Compare(s.Key.ToString(), m.Key.ToString()) == 0));
            Assert.AreEqual(0, missing.Count());

            missing = baseList.Where(m => !britishList.Any(s => string.Compare(s.Key.ToString(), m.Key.ToString()) == 0));
            Assert.AreEqual(0, missing.Count());
        }

This should then fail every time it encounters a resource in the base list that is not localised. If you needed to see the reverse, you could simply reverse the statement.

Conclusion

If you work in multiple languages, this can prove to be a very useful test. It also means that you can safely decide to delay the translation without being concerned that you’ll miss anything.