Tag Archives: MSTest

Live Testing in Visual Studio 2017

While working on this project, I decided that I would see how useful the new feature of “Live Testing” is for TDD.

Once you’ve created your suite of tests, turning live testing on gives you some interesting information:

As you can see, there are some slight contradictions here in what it’s telling me. The first test is failing, the test for Count has a line (which indicates not covered). However, code lens is telling me there are two covering tests.

My suspicion here is that live testing is being too clever for its own good; for example, looking at the tests themselves, shows that only the first Enqueue statement is covered:

My thoughts are that the engine works out where the test fails, and so, in the case above, it’s telling me that the tests are not running the line of code that tests the Count functionality. What I’m guessing is that this is because the tests are failing before that.

I can prove this by creating a further unit test:

And, indeed, this now shows coverage:

As I start to add the functionality, I can quickly see that the tests are now passing:

Changing the way code is written

The interesting thing here is that is promotes two things:
1. Defined tests – if a test covers more than one thing, it will show as uncovered initially
2. Tests first – obviously, there need to be tests to show test coverage

Caveats

1. Annoyingly, by default, you continually have to opt in for live unit testing. It only starts when you select to “Start” live unit testing, and this must be done each time you launch VS. This is configurable, and can be switched on by default in options:

2. If the build fails, it shows nothing.

3. Live testing is only available in Visual Studio Enterprise.

NUnit

One thing that only occurred to me as notable while I was writing this is that the tests that I’ve written are in NUnit. So the Live Testing clearly works with none MSTests, and probably works with anything that’ll show up in Test Explorer / Code Lens.

References

https://blogs.msdn.microsoft.com/visualstudio/2017/03/09/live-unit-testing-in-visual-studio-2017-enterprise/

Using MSTest DataRow as a Substitute for NUnit TestCase

I used to believe that Nunit’s TestCase test (that is, an ability to define a test and then simply pass it alternate parameters) was denied MSTest users. It appears that this is, at least now, fallacious.

The following article implies that this is a recent change:

Taking the MSTest Framework forward with “MSTest V2”

This particular example is in a UWP application:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 12)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Will result in a failing test, and:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6)]
        [DataRow(8, 2, 3, 13)]
        [DataRow(8, 5, 3, 16)]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Results in a passing one.

If you want additional information relating to the test, you can use this syntax:

        [DataTestMethod]
        [DataRow(1, 2, 3, 6, DisplayName = "First test")]
        [DataRow(8, 2, 3, 13, DisplayName = "Second test")]
        [DataRow(8, 5, 3, 15, DisplayName = "This will fail")]
        public void AddNumbers(int num1, int num2, int num3, int total)
        {
            Assert.AreEqual(num1 + num2 + num3, total);
        }

Given the constant problems that I have with finding the correct NUnit test adaptor, and trying to work out which are the right libraries, I think, despite coming late to this party, MS might actually drag people back to MSTest with this.

Testing by Interface

Given a standard interface, how to retrieve all implementing classes and run the interface methods.

Unit testing through test driven development is definitely a good idea; but what if you have a number of methods that all effectively do the same thing; that is, each method might do something completely different, but as far as it’s interface goes, it’s identical.

Imagine, for example, that you have a method that calls to the DB, and accepts a number of parameters in, and returns a given parameter. In a unit testing scenario, the DB would be mocked out, and the method called directly from the unit test. Okay, so in this case, you may want some test coverage that your methods call a mocked out DB function, don’t crash, accept a given object, accept null, etc…

Facing the same problem, it occurred to me that it should be possible to write a single test method that would test every existing and future implementation of this, without having to laboriously re-create the test each time I create a method; what’s more, as soon as I create my method name that implements the interface, I get a failing test.

Below is an interface and a test class; it is entirely for the purpose of illustration:

    public class ModelClass
    {
        public string TestProperty { get; set; }
    }


    public interface ITest
    {
        void method1();

        void method2(ModelClass model, int i);
    }

    public class Class1 : ITest
    {
        public void method1()
        {
        }

        public void method2(ModelClass model, int i)
        {
            if (model == null) throw new Exception("test");
            //if (string.IsNullOrWhiteSpace(model.TestProperty)) throw new Exception("Doh");
        }
    }

    public class Class2 : ITest
    {
        public void method1()
        {
            //throw new NotImplementedException();
        }

        public void method2(ModelClass model, int i )
        {
            
        }
    }

    public class Class3
    {
        public void  NonInterfaceMethod(ModelClass model)
        {
            throw new Exception("Doh!");
        }
    }

    public class Class4 : ITest
    {
        public void method1()
        {
            //throw new NotImplementedException();
        }

        public void method2(ModelClass model, int i)
        {
            
        }

        public void test()
        {

        }
    }

 

As you can see, there are a number of interface, and non-interface methods here. There’s nothing particularly interesting, although have a look at Class1.method2(), which should do nothing, but switching the statements should cause a runtime error, if my method works. Also, have a look at Class3.NonInterfaceMethod() – this should never be called, but will throw an exception if it is.

The following is the test code:

        [TestMethod]
        public void TestITestImplementations()
        {
            // Use reflection to get the available methods for the interface
            Type desiredType = typeof(ITest);
            Assembly assembly = desiredType.Assembly;
            var interfaceMethods = desiredType.GetMethods();
            
            // Iterate through each implementation of the interface
            foreach (Type type in assembly.GetTypes())
            {
                if (desiredType.IsAssignableFrom(type) && !type.IsInterface)
                {
                    // Where an implementation is found, instantiate it 
                    // and build a list of available methods to call
                    var classInstance = Activator.CreateInstance(type, null);
                    var methods = type.GetMethods()
                        .Where(m => interfaceMethods.Any(i => i.Name == m.Name)
                            && m.IsPublic
                            && !m.DeclaringType.Equals(typeof(object)));
                    foreach(var method in methods)
                    {
                        // Establish the available parameters and pass them to the call
                        var p = method.GetParameters();
                        object[] p2 = p.Select(a => Activator.CreateInstance(a.ParameterType)).ToArray();

                        try
                        {
                            // Call the method and, where a value should be returned, ensure one is
                            object result = method.Invoke(classInstance, p2);
                            Assert.IsFalse(method.ReturnType != typeof(void) && result == null);
                        }
                        catch(Exception ex)
                        {
                            // Where an error is thrown, print a sensible error
                            Assert.Fail("Call failed: {0}.{1}\nException: {2}", 
                                type.Name, method.Name, ex);
                        }
                    }
                }
            }            
        }

The code above is relatively straight-forward and, if I’m being honest, is only a cursory test. It tests that the methods can be called without throwing an error and, where a value should be returned, checks that it’s not null.

Obviously, it might be perfectly valid that it is null, or an exception might be the desired behaviour. This code is basically just a starting point, but it does provide some very basic test coverage where otherwise, there might be none.

It is also true to say that the code doesn’t deal with overloads, which are not necessary in my particular circumstance.

Unit Tests Are Not Discoverable

I recently had a situation where I loaded a solution containing a suite of NUnit tests, but the test explorer would not recognise them. The following is a series of checks to make that may cause unit tests to be not visible. Most of these are applicable to all tests:

1. Tests must be declared as public. For example:

        public void MyTestMethod()
        {

2. Tests must be decorated with a test attribute.

For NUnit this is is [Test]:

	        [Test]
	        public void MyNUnitTest()
	        {
	

Or [TestCase]:

	        [TestCase(1)]
	        [TestCase(2)]
	        public void MyParameterisedTest(int testNum)
	        {
	
	

For MSTest this is [TestMethod]:

	        [TestMethod]
	        public void MyTestMethod()
	        {
	

3. If using NUnit – check that the correct version is installed (remember that v3 is not an official release yet – that is, at the time of writing).

The current release test adapter is here

The test adaptor for NUnit3 is here

As usual, this is more for my own reference, but if it helps anyone else then all to the good. Also, if you think of or encounter another then please let me know in the comments and I’ll add it on.

Unit Testing Methods With Random Elements (in MVVM Cross)

Okay, quick spoiler for this: you can’t. You can’t, not really; obviously, you can write the test, but unit tests should be predictive, and a random element should not.

Solution

I imagine there are a few ways of solving this. The way shown in this post is specific to MVVM cross, but should work with any system that uses an IoC container. In brief, we’re simply going to mock out the system Random class.

How?

Well, since System.Random is the domain of Microsoft, we’ll start with a wrapper; and since this is MVVM Cross, we’ll make it a service:

    class RandomService : IRandomService
    {
        private static Random _rnd = null;

        public virtual int SelectRandomNumber(int max)
        {
            if (_rnd == null)
            {
                _rnd = new Random();
            }

            return _rnd.Next(max);
        }
    }

Couple of notes on this:
1. I haven’t posted the interface but it’s just the one method.
2. The reason for the Random class being static is that the random seed is taken from the system clock, meaning that if you call this in quick succession, there is a possibility that you would get the same number returned.
3. This is not thread safe.

Okay – all that out of the way, the code is pretty basic. Now let’s call it:

        public static T SelectRandomElement<T>(this IEnumerable<T> enumeration)
        {
            var service = Mvx.Resolve<IRandomService>();            
            int idx = service.SelectRandomNumber(enumeration.Count() + 1);

            return enumeration.ElementAt(idx);
        }

Right, so you’ll recognise the extension method from the last post, but now it retrieves the instance of the random service; here’s where we register that:

        protected override void InitViewModel()
        {
            Mvx.ConstructAndRegisterSingleton<IRandomService, RandomService>();
        }

You can actually register it anywhere you like… before it’s actually called.

Okay, so now we should have unchanged functionality; everything works as before.

The Unit Tests

The first task here is to create the mock RNG:

    class MockRandomService : IRandomService
    {
        static int _lastNumber = 0;

        public int SelectRandomNumber(int max)
        {            
            if (_lastNumber < max)
                return ++_lastNumber;
            else
            {
                _lastNumber = 0;
                return _lastNumber;
            }                
        }
    }
&#91;/sourcecode&#93;

This not allows me to determine what the next random number will be.

<strong>MVVM Cross Unit Testing</strong>

To set-up a test for MVVM Cross using the IoC container, you need to add some additional libraries to the test project first:

<a href="http://pmichaels.net/wp-content/uploads/2014/07/mvvmtest.png"><img src="http://pmichaels.net/wp-content/uploads/2014/07/mvvmtest.png?w=300" alt="mvvmtest" width="300" height="32" class="alignnone size-medium wp-image-618" /></a>


This will add Cirrious.MccmCross.Test.Core:

<a href="http://pmichaels.net/wp-content/uploads/2014/07/refs.png"><img src="http://pmichaels.net/wp-content/uploads/2014/07/refs.png" alt="refs" width="239" height="211" class="alignnone size-full wp-image-620" /></a>

And that is important, because it allows you to declare your test class as follows:


    [TestClass]
    public class ExtensionMethodTests : MvxIoCSupportingTest
    {

Inheriting from MvxIoCSupportingTest allows you to call base.Setup(), which prevents the IoC container from crashing when you call it in a test. Here’s the full unit test code:

[TestClass]
public class ExtensionMethodTests : MvxIoCSupportingTest
{
[TestMethod]
public void TestSelectRandomElement()
{
base.Setup();

Mvx.ConstructAndRegisterSingleton();

List testCollection = new List();

testCollection.Add(1);
testCollection.Add(3);
testCollection.Add(5);
testCollection.Add(7);
testCollection.Add(9);

// Cycle through all elements
for (int i = 0; i <= 5; i++) { int e = testCollection.SelectRandomElement(); Assert.AreNotEqual(e, 0); } } [TestMethod] public void TestSelectRandomElementPredicate() { base.Setup(); Mvx.ConstructAndRegisterSingleton();

List testCollection = new List();

testCollection.Add(1);
testCollection.Add(3);
testCollection.Add(5);
testCollection.Add(7);
testCollection.Add(9);

// Cycle through all elements
for (int i = 0; i <= 5; i++) { int e = testCollection.SelectRandomElement(n => n < 2); Assert.AreNotEqual(e, 1); } } } } [/sourcecode] Conclusion

So, I now have a custom RNG and unit tests that will tell me what happens when I call the method for each element. Obviously these tests are not exhaustive, but they are deterministic.

Resource File Testing

Admittedly this does sound like a strange one. How or why would you both to test a resource file? The only reason I could think of was this: what if someone adds a resource, and forgets to localise it?

Without a test such as this, the first way you’d know this is when you ran the program in a localised culture and spotted the default culture appearing. This is something that could potentially go unnoticed for a long time. Consider British and American English, or Spanish and Latin-American Spanish.

To set-up the test, create three resource files:

res1

Create two resources in the base resource file (.resx):

res2

And then in each localised file, create one of these; so, in Resource.en-GB.resx create testphraseone (test phrase one english), and in the Spanish: testphrasetwo (prueba frase dos).

We also need to expose the resource manager:

    public class GetRes
    {
        public static ResourceManager GetResMgr()
        {            
            ResourceManager resMgr = new ResourceManager("resourcetest.Properties.Resource", typeof(resourcetest.Properties.Resource).Assembly);
            return resMgr;
        }

The Test

Now that we have the ResourceManager, we can simply call the GetResourceSet function. The third parameter determines whether to try the parent if the resource doesn’t exist, so setting this to false will force it to get the resource from the file in question.

The test could look like this:

        [TestMethod]
        public void TestMethod1()
        {
            ResourceManager resMgr = resourcetest.GetRes.GetResMgr();
            
            var baseList = resMgr.GetResourceSet(CultureInfo.InvariantCulture, true, true).Cast<DictionaryEntry>();            
            var spanishList = resMgr.GetResourceSet(new CultureInfo("es-ES"), true, false).Cast<DictionaryEntry>();
            var britishList = resMgr.GetResourceSet(new CultureInfo("en-GB"), true, false).Cast<DictionaryEntry>();
            
            var missing = baseList.Where(m => !spanishList.Any(s => string.Compare(s.Key.ToString(), m.Key.ToString()) == 0));
            Assert.AreEqual(0, missing.Count());

            missing = baseList.Where(m => !britishList.Any(s => string.Compare(s.Key.ToString(), m.Key.ToString()) == 0));
            Assert.AreEqual(0, missing.Count());
        }

This should then fail every time it encounters a resource in the base list that is not localised. If you needed to see the reverse, you could simply reverse the statement.

Conclusion

If you work in multiple languages, this can prove to be a very useful test. It also means that you can safely decide to delay the translation without being concerned that you’ll miss anything.