Tag Archives: Unit Test

Introduction to Unit Tests (with examples in .Net) – Part 3 – Test Frameworks and Manual Mocks

So far, in this series of posts on the basics of unit tests, we’ve spoken about concepts and methodologies, but we’ve steered away from using any specific frameworks or tools. In this post, we’ll investigate what a test framework can do for us.

We’ll continue to work with the code that we created in the previous post, but we’ll address the issues that we still had at the end of that post.

A Recap of the Story So Far

At the end of the previous post, we had the following code:

for (int i = 1; i <= 100; i++)
{
    // Arrange
    Func<string> mockInput = () => "5";
 
    // Act
    string result = RunMethod(mockInput);
 
    // Assert
    if (result == "Well done, you guessed!")
    {
        Console.WriteLine("Test Passed");
        break;
    }
}
 
for (int i = 1; i <= 100; i++)
{
    // Arrange
    Func<string> mockInput = () => "5";
 
    // Act
    string result = RunMethod(mockInput);
 
    // Assert
    if (result.StartsWith("Sorry, that was the wrong number"))
    {
        Console.WriteLine("Test Passed");
        break;
    }
}
 
{
    // Arrange
    Func<string> mockInput = () => "";
 
    // Act
    string result = RunMethod(mockInput);
 
    // Assert
    if (result == "Invalid guess")
    {
        Console.WriteLine("Test Passed");
    }
}

We had yet to introduce any tools or frameworks, but we had managed to test our code. We still had the following issues, however:

1. The tests passed, but we visually have to visually ascertain that.
2. We were outputting to the console needlessly.
3. Our tests were not resilient – a change of a single character in the user output, and the tests would break.
4. The tests were not deterministic – they were dependent on the result of a pseudo random number.

In this post, we’ll address these issues in order (apart from the third one, but we’ll come back to that) : we’ll start with the first.

1. The tests passed, but we visually have to visually ascertain that

How can we ascertain the result of a test without watching to see what happens with the test. One thing we could do is something similar to the following:

int RunTest1()
{
    for (int i = 1; i <= 100; i++)
    {
        // Arrange
        Func<string> mockInput = () => "5";

        // Act
        string result = RunMethod2(mockInput);

        // Assert
        if (result == "Well done, you guessed!")
        {
            Console.WriteLine("Test Passed");
            return 0;
        }
    }
    return 1;
}

int RunTest2()
{
    for (int i = 1; i <= 100; i++)
    {
        // Arrange
        Func<string> mockInput = () => "5";

        // Act
        string result = RunMethod2(mockInput);

        // Assert
        if (result.StartsWith("Sorry, that was the wrong number"))
        {
            Console.WriteLine("Test Passed");
            return 0;
        }
    }
    return 1;
}

int RunTest3()
{
    // Arrange
    Func<string> mockInput = () => "";

    // Act
    string result = RunMethod2(mockInput);

    // Assert
    if (result == "Invalid guess")
    {
        Console.WriteLine("Test Passed");
        return 0;
    }
    return 1;
}

Console.WriteLine(RunTest1());
Console.WriteLine(RunTest2());
Console.WriteLine(RunTest3());

There’s a lot of code here, but all we’ve actually done is wrap the tests up in functions, and then returned a value based on the result of the test. This means that we can write something like this:

if (RunTest1() != 0 | RunTest2() != 0 | RunTest3() != 0)
{
    Console.WriteLine("Some tests failed");
}

In case you didn’t know, the single pipe (|) in C# in a bitwise or – that is, it will execute all conditions regardless of the result and then evaluate the result, a logical or (||) would only run the tests until one failed and then exit the condition.

This approach also helps with the second issue.

2. We were outputting to the console needlessly

We’re outputting to the console for two reasons: the first is to validate the tests; we can now simply remove all of those from the test, since we have an actual value that we can test against. The second reason is that the code itself outputs to the console. As has been mentioned in a previous post, there are ways to redirect the output of the console without mocking it; however, I’m trying to keep this series generic, and that is specific to the .Net console (although I strongly suspect that most languages provide a similar concept).

To get around this, we could replicate what we did in the second post; however, we can also take a slightly different approach and wrap the entire input / output functionality in its own class; for example:

    internal class ConsoleInputOutputWrapper
    {
        public void Output(string text) => Console.WriteLine(text);
        public string GetInput(string prompt)
        {
            Output(prompt);
            return Console.ReadLine();
        }
    }

This idea gives us some additional benefits – as you can see, we already have the prompt and input in a single method; and we could go further – we could do some validation inside the method, too; what if the user doesn’t enter anything:

        public string GetInput(string prompt)
        {            
            while (true)
            {
                Output(prompt);
                string? answer = Console.ReadLine();
                if (!string.IsNullOrWhiteSpace(answer)) return answer;
            }
        }

We can now replace the direct references to Console with references to this:

string RunMethod(Func<string> readData)
{
    var io = new ConsoleInputOutputWrapper();
    int myNumber = Random.Shared.Next(100) + 1;

    io.Output("Guess the number that I'm thinking between 1 - 100");
    string? guess = readData();
    string result = BusinessLogic2(myNumber, guess);
    io.Output(result);
    return result;
}

We haven’t actually changed anything here, though – the console is still being written to. We need to be able to replace the functionality within the system for our test. We can do that by replacing the concrete class with an interface.

Adding an Interface

Adding an interface is much simpler than it may sound. Let’s see what needs to change in our ConsoleInputOutputWrapper class:

internal class ConsoleInputOutputWrapper : IInputOutputWrapper

We’ve implemented an interface that we’ve named IInputOutputWrapper – we’ve named it this because it’s more generic (that is, it doesn’t actually need to be a Console).

The interface just needs to specify the public methods in the class:

    internal interface IInputOutputWrapper
    {
        void Output(string text);
        string GetInput(string prompt);
    }

Whilst this syntax is specific to C#, the concept of an interface is not.

While we’re introducing an interface, and to clean our code a little, we can extract both our RunMethod and BusinessLogic methods into their own class – let’s call it Game:

    internal class Game
    {
        public string RunMethod(Func<string> readData)
        {
            var io = new ConsoleInputOutputWrapper();
            int myNumber = Random.Shared.Next(100) + 1;

            io.Output("Guess the number that I'm thinking between 1 - 100");
            string? guess = readData();
            string result = BusinessLogic(myNumber, guess);
            io.Output(result);
            return result;
        }

        public string BusinessLogic(int myNumber, string guessedNumber)
        {
            if (string.IsNullOrEmpty(guessedNumber))
            {
                return "Invalid guess";
            }

            if (int.Parse(guessedNumber) == myNumber)
            {
                return "Well done, you guessed!";
            }
            else
            {
                return $"Sorry, that was the wrong number, I was thinking of {myNumber}";
            }
        }

    }

This makes things much simpler. We can now create a constructor, and pass in our new interface:

    internal class Game
    {
        private readonly IInputOutputWrapper _inputOutputWrapper;

        public Game(IInputOutputWrapper inputOutputWrapper)
        {
            _inputOutputWrapper = inputOutputWrapper;
        }

Now that we have this instance, we can simply replace the method with a reference to this instead:

        public string RunMethod()
        {            
            int myNumber = Random.Shared.Next(100) + 1;
            
            string guess = _inputOutputWrapper.GetInput("Guess the number that I'm thinking between 1 - 100");
            string result = BusinessLogic(myNumber, guess);
            _inputOutputWrapper.Output(result);
            return result;
        }

We can now update our tests to call this new class, but we can pass in our own version of the IInputOutputWrapper, which may look like this:

    internal class MockInputOutputWrapper : IInputOutputWrapper
    {
        public string GetInput(string prompt)
        {
            return "5";
        }

        public void Output(string text) { }
    }

The test would then look something like this:

    for (int i = 1; i <= 100; i++)
    {
        // Arrange
        var inputOutputWrapper = new MockInputOutputWrapper();
        var sut = new Game(inputOutputWrapper);

        // Act
        string result = sut.RunMethod();

        // Assert
        if (result == "Well done, you guessed!")
        {            
            return 0;
        }
    }
    return 1;

Next, we’ll skip number 3 and jump to 4.

4. The tests were not deterministic – they were dependent on the result of a pseudo random number

We can use the same pattern to create a wrapper for our random number chooser:

    internal class RandomNumberChooser : IRandomNumberChooser
    {
        public int Choose() =>
            Random.Shared.Next(100) + 1;        
    }

We can then mock that out, as before:

internal class MockRandomNumberChooser : IRandomNumberChooser
{
    public int Choose() => 12;
}

This definitely works, but it’s not brilliant. We have a few remaining issues – for example, if we want to test the number are the same, or different, we’ll need two mock classes. There are ways around this, too – for example:

    internal class MockInputOutputWrapper : IInputOutputWrapper
    {
        private readonly string _inputValue;

        public MockInputOutputWrapper(string inputValue) =>
            _inputValue = inputValue;        

        public string GetInput(string prompt) => _inputValue;        

        public void Output(string text) { }
    }

We’ll come back to neater ways to achieve this in a future post, but for now, let’s put all this together and introduce a test framework.

Introducing a Test Framework

Test frameworks give you four basic things (some, in fact most, do more, but these are the absolute basics that you need – otherwise, you might as well roll your own):

1. A return value from the test run to determine whether the tests pass or fail
2. An ability to assert a value is in a given state
3. Some kind of integration into your IDE
4. Method discovery (that is, some way to mark your tests as tests)

For this example, we’ll use xUnit.net. Every language has its own options here – in .Net I’ve used MS Test, Nunit, and xUnit – and they’re all broadly the same; I’ve also seen libraries in Javascript and Python and, again, they mostly do the same stuff.

We’ll need to install the following libraries:

Install-Package Microsoft.Test.Sdk
install-package Xunit
Install-Package Xunit.Runner.Console
Install-Package Xunit.Runner.VisualStudio

This will enable you to create a test such as this:

[Fact]
public void RunMethod_GuessedCorrectly_CorrectTextReturned()
{
    // Arrange
    var inputOutputWrapper = new MockInputOutputWrapper("12");
    var randomNumberChooser = new MockRandomNumberChooser();
    var sut = new Game(inputOutputWrapper, randomNumberChooser);

    // Act
    string result = sut.RunMethod();

    // Assert
    Assert.Equal("Well done, you guessed!", result);
}

We no longer need to run this 100 times, because we can force a correct and incorrect guess:

        [Fact]
        public void RunMethod_GuessedIncorrectly_CorrectTestReturned()
        {
            // Arrange
            var inputOutputWrapper = new MockInputOutputWrapper("13");
            var randomNumberChooser = new MockRandomNumberChooser();
            var sut = new Game(inputOutputWrapper, randomNumberChooser);

            // Act
            string result = sut.RunMethod();

            // Assert
            Assert.StartsWith("Sorry, that was the wrong number", result);
        }

Summary

We’ve now seen how we can manually mock functionality, and how that can help us to accurately test methods; we’ve also introduced a testing framework. In the next post, we’ll discuss mocking frameworks, and how they can make this even easier. We’ll also re-visit the test resilience.

Introduction to Unit Tests (with examples in .Net) – Part 2 – Refactoring and Mocking

This forms the second in a short series of posts on unit tests. You can find the first post in this series here.

In this post, I’ll be expanding the points raised in the first post to cover a slightly more realistic scenario, and offering some tips of how re-factoring might help with the creation of unit tests. We’ll also cover the basic principles behind mocking – I’m intending to cover this in more detail in a future post of this series.

A Quick Recap

You’re welcome (and encouraged) to go back to the first post in the series; however, to summarise, we discussed the Arrange/Act/Assert pattern, and how it can help us structure a unit test; we spoke about the FIRST principles of testing, and thereby the attributes that we should look for in a good unit test.

What we specifically didn’t cover was any testing frameworks, the concept of mocking or any mocking frameworks, or how to write a unit test in a scenario where you’re not simply adding two numbers together.

A More Realistic Unit Test

If we take an example of any level of complexity, we might question some of the points that were made in the first post. After all, very few methods would simply take two numbers and add them together – or, if they do, perhaps we need to reconsider the language that we’re using.

Let’s look at a simple console application:

int myNumber = Random.Shared.Next(100) + 1;

Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
var guess = Console.ReadLine();
if (string.IsNullOrEmpty(guess))
{
    Console.WriteLine("Invalid guess");
    return;
}

if (int.Parse(guess) == myNumber)
{
    Console.WriteLine("Well done, you guessed!");
}
else
{
    Console.WriteLine($"Sorry, that was the wrong number, I was thinking of {myNumber}");
}

If you had to test this code, how would you do it?

In fact, it’s really difficult, because every time you run it, the number is different. This is a simple piece of code, there’s only 3 code paths; arguably, your strategy could be: run it once and enter a blank value, run it once and enter a non-numeric value, run it 100 more times and hope that you’ll get the number right once.

Writing a Test

As before, let’s start with the manual test that we’ve just described; arguably, we could simply automate this. We’d probably do something like this:

        public static void RunTest()
        {
            // Run method here - check that a blank entry works

            // Run method here - check that a numeric entry works

            for (int i = 0 ; i < 100; i++)
            {
                // Run method here - exit this loop once we've determined that we have a correct and incorrect guess

            }
        }

In fact, running that exact test would be possible – we could simply redirect the console input and output; however, for the purposes of this post, we’ll bypass that method (as it is quite specific to writing a .Net Console app), and we’ll re-factor our code a little.

Refactoring

We can refactor it by splitting the method into two; one method that accepts the input, and one that runs the business logic:

void RunMethod()
{
    int myNumber = Random.Shared.Next(100) + 1;

    Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
    var guess = Console.ReadLine();
    BusinessLogic(myNumber, guess);
}

void BusinessLogic(int myNumber, string guessedNumber)
{
    if (string.IsNullOrEmpty(guessedNumber))
    {
        Console.WriteLine("Invalid guess");
        return;
    }

    if (int.Parse(guessedNumber) == myNumber)
    {
        Console.WriteLine("Well done, you guessed!");
    }
    else
    {
        Console.WriteLine($"Sorry, that was the wrong number, I was thinking of {myNumber}");
    }
}

All we’ve done here is split the method into two methods – the code is exactly the same as it was before. However, now we can run the code in our test without worrying about the input:

        public static void RunTest()
        {
            // Check that a blank entry works
            BusinessLogic(3, "");

            // Check that a non-numeric entry works
            BusinessLogic(3, "aardvark");

            for (int i = 1; i <= 100; i++)
            {
                // Check that false and true numbers work
                BusinessLogic(i, "2");
            }
        }

It still feels a lot like we’re only testing half of the code. We aren’t testing the calling method.

Mocking

Let’s refactor a little further. Instead of using the console, we’ll simply create our own method that performs the same task:

private static string? GetInput() => Console.ReadLine();    

Again, no real change here, we’re just wrapping the code that accepts input in a method that we control.

Now that we’ve done this, we can use our own method to accept input, instead of the Console methods. There’s a number of ways we could do this but, perhaps the easiest, is to pass the GetInput method into the RunMethod method as a parameter:

public static void RunMethod(Func<string> readData)
    {

        int myNumber = Random.Shared.Next(100) + 1;

        Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
        string? guess = readData();
. . .

Here, we’re simply changing two things: we’re accepting a delegate into our main method, and then we’re calling that, instead of the Console.ReadLine().

What’s the point of doing that? Well, now that we control that function as a parameter, we can mock the function, and replace it with our own functionality.

In fact, we are not technically discussing a mock here, but a stub. For the purpose of this post, we’ll simply group them together with the working definition that a mock is: “anything that replaces functionality for the purpose of testing”. I intend to re-visit this in a future post and go into more detail on the difference between the two.

Let’s jump to our test.

Arrange

In the test, we can now replace this functionality with a specific value:

        public static void RunTest()
        {
             // Arrange
            Func<string> mockInput = () => "5"; . . .

We’ve now established the input of the method, the next step is to be able to assert that the test worked. In fact, that’s very difficult with the code as it currently is, because we just display output to the user. To finish this post off, we’ll refactor this as little as we can; imagine we take the business logic function and change it to be like this:

string BusinessLogic(int myNumber, string guessedNumber)
{
    if (string.IsNullOrEmpty(guessedNumber))
    {        
        return "Invalid guess";
    }

    if (int.Parse(guessedNumber) == myNumber)
    {
        return "Well done, you guessed!";
    }
    else
    {
        return $"Sorry, that was the wrong number, I was thinking of {myNumber}";
    }
}

All we’ve changed here is that we’re returning the string, instead of outputting it. We can then change the calling method to do the output:

string RunMethod(Func<string> readData)
{
    int myNumber = Random.Shared.Next(100) + 1;

    Console.WriteLine("Guess the number that I'm thinking between 1 - 100");
    string? guess = readData();
    string result = BusinessLogic(myNumber, guess);
    Console.WriteLine(result);
    return result;
}

Same idea again, a very small change of writing the output, and returning the result again. The functionality hasn’t changed, but now we have something to test against.

Assert

We can now write out test method to look something like this:

for (int i = 1; i <= 100; i++)
{
    // Arrange
    Func<string> mockInput = () => "5";

    // Act
    string result = RunMethod(mockInput);

    // Assert
    if (result == "Well done, you guessed!")
    {
        Console.WriteLine("Test Passed");
        break;
    }
}

for (int i = 1; i <= 100; i++)
{
    // Arrange
    Func<string> mockInput = () => "5";

    // Act
    string result = RunMethod(mockInput);

    // Assert
    if (result.StartsWith("Sorry, that was the wrong number"))
    {
        Console.WriteLine("Test Passed");
        break;
    }
}

{
    // Arrange
    Func<string> mockInput = () => "";

    // Act
    string result = RunMethod(mockInput);

    // Assert
    if (result == "Invalid guess")
    {
        Console.WriteLine("Test Passed");
    }
}

There’s three distinct tests here and, unless unlucky, they’ll all pass. There’s definitely some work left to do here, as we still have the following problems:

1. Although the tests can pass, we have to visually ascertain that.
2. We’re outputting to the console needlessly.
3. Our tests are not resilient – if I change a single character in the user output, the tests will break.
4. The tests are not deterministic – they are dependent on the result of a pseudo random number.

In the next post, we’ll address these issues: we’ll introduce a test framework, and further refactor this code such that we can be confident that cosmetic changes will not break the tests.

Cyclomatic Complexity – What it is, why you should care, and how to reduce it using the Strategy Pattern

Cyclomatic complexity is one of those terms that makes you think you missed something when you were learning programming. However, the concept is a really simple one. Cyclomatic complexity is simply the number of paths through your code. There are more detailed explanations if you scan the web (involving edges and nodes), but for this post, we’ll just work with that.

Code Metrics

This article is not about .Net per se – what I’m writing here applies to any OO language; however, for the purpose of illustration, I’ll be using C#, and the Code Metrics Window in Visual Studio.

Cyclomatic Complexity

Given what we’ve just said, we can take the following program:

int x = 1;

And we can determine that the cyclomatic complexity is 1 – that is, there is only one way that this code can execute – no branches, no loops, just one statement. So, the cyclomatic complexity is 1:

Okay, let’s now add a single branch:

There are now two ways this code can execute (in actuality, there is only one, but cyclomatic complexity doesn’t follow the actual logical course – so let’s agree on two for the sake of argument).

We can now add a second condition:

The complexity now goes to three.

Okay, so that’s all very interesting, but what does that actually mean – why is it useful to know this number? The main answer to this is testability: if you know there are 3 possible execution paths for the code, then you know that you need a minimum of 3 tests to cover those paths. There’s a second part to that, which is that a method that requires one – three tests might be easy to change or debug, one that requires ten – twenty tests is definitely not.

So, the question is, how can we reduce this figure?

Reducing Cyclomatic Complexity

There are, obviously, more answers to this than would fit in this post, but here I’m going to focus on bringing in a strategy pattern.

Let’s consider the following code:

    // Night
    switch (thingToAutomate)
    {
        case "Door":
            Console.WriteLine("Lock");
            break;

        case "Window":
            Console.WriteLine("Close");
            break;

        case "TV":
            Console.WriteLine("Turn off");
            break;

        case "Lights":
            Console.WriteLine("Turn off");
            break;
    }

This method had a cyclomatic complexity of 5 – there are 4 options, but also the possibility that none are true.

The essence behind the strategy pattern is just that you assign functionality using polymorphism. If we consider the code above, doing that would actually increase the number of lines of code, and thereby increase the cyclomatic complexity. However, what if, in addition to the automation routine at night, we needed one for the morning:

// Morning
switch (thingToAutomate)
{
    case "Door":
        Console.WriteLine("Unlock");
        break;

    case "Window":
        Console.WriteLine("Open");
        break;

    case "TV":        
    case "Lights":        
        break;
}

Okay, now our cyclomatic complexity increases (to 9) – and will increase every time we need to vary our behaviour based on the thing that we’re automating. Instead, let’s consider the following:

    internal interface IThingToAutomateStrategy
    {
        void Morning();
        void Night();
    }

Let’s now imagine that we implement the interface for each thing:

    internal class Door : IThingToAutomateStrategy
    {
        public void Morning()
        {
            Console.WriteLine("Unlock");
        }

        public void Night()
        {
            Console.WriteLine("Lock");
        }
    }

    . . .

Admittedly, this does increase the lines of code, but we end up with simpler code, and it has a lower, overall, cyclomatic complexity:

    internal class Automation
    {
        public IThingToAutomateStrategy AutomateStrategy { get; set; }

        public void AutomateMorning()
        {
            AutomateStrategy.Morning();            
        }

        public void AutomateNight()
        {
            AutomateStrategy.Night();
        }
    }

We can then use this in our program like this:

IThingToAutomateStrategy automateStrategy;

switch (thingToAutomate)
{
    case "Door":
        automateStrategy = new Door();
        break;

    case "Window":
        automateStrategy = new Window();
        break;

    case "TV":
        automateStrategy = new TV();
        break;

    case "Lights":
        automateStrategy = new Lights();
        break;
}

automateStrategy.Morning();
automateStrategy.Night();

The cyclomatic complexity of this is back to 5; but the best part is that it doesn’t increase. Imagine the following new method:

    internal interface IThingToAutomateStrategy
    {
        void Morning();
        void Darkness();
        void Night();
    }

And the implementation:

    internal class Lights : IThingToAutomateStrategy
    {
        public void Darkness()
        {
            Console.WriteLine("Switch on");
        }

        public void Morning()
        {
            // Nothing to do here
        }

        public void Night()
        {
            Console.WriteLine("Turn Off");
        }
    }

It’s worth pointing out that we’re in breach of the ISP here – but since we’re only doing it to make a point, we’ll agree to let it slide.

Adding this to the code flow doesn’t affect the cyclomatic complexity score of that code file at all:

IThingToAutomateStrategy automateStrategy;

switch (thingToAutomate)
{
    case "Door":
        automateStrategy = new Door();
        break;

    case "Window":
        automateStrategy = new Window();
        break;

    case "TV":
        automateStrategy = new TV();
        break;

    case "Lights":
        automateStrategy = new Lights();
        break;
}

automateStrategy.Morning();
automateStrategy.Darkness();
automateStrategy.Night();

It’s worth noting that it does increase the overall complexity, as it counts as a single additional code path per thing.

References

https://docs.microsoft.com/en-us/visualstudio/code-quality/code-metrics-cyclomatic-complexity?WT.mc_id=DT-MVP-5004601

https://docs.microsoft.com/en-us/visualstudio/code-quality/how-to-generate-code-metrics-data?WT.mc_id=DT-MVP-5004601

https://codinghelmet.com/articles/reduce-cyclomatic-complexity-composite-design-pattern

https://www.c-sharpcorner.com/UploadFile/shinuraj587/strategy-pattern-in-net/

https://codewithshadman.com/strategy-pattern-csharp/

https://github.com/pcmichaels/StrategyExample

Testing an Asp.Net Web App Using Integration Testing

I’ve recently been playing around with a tool called Scrutor. I’m using this in a project and it’s working really well; however, I came across an issue (not related to the tool per se). I had created an interface, but hadn’t yet written a class to implement it. Scrutor realised this was the case and started moaning at me. Obviously, I hadn’t written any unit tests around the non-existent class, but I did have a reasonably good test coverage for the rest of the project; however the project wouldn’t actually run.

To be clear, what I’m saying here is that, despite the test suite that I had running successfully, the program wouldn’t even start. This feels like a very askew state of affairs.

Some irrelevant background, I had a very strange issue with my Lenovo laptop, whereby, following a firmware update, the USB-C ports just stopped working – including to accept charge – so my laptop died. Sadly, I hadn’t followed good practice, with commits, and so part of my code was lost.

I’ve previously played around with the concept of integration tests in Asp.Net Core+, so I thought that’s probably what I needed here. There are a few articles and examples out there, but I couldn’t find anything that worked with Asp.Net 6 – so this is that.

In this post, we’ll walk through the steps necessary to add a basic test to your Asp.Net 6 web site. Note that this is not comprehensive – some dependencies will trip this up (e.g. database access); however, it’s a start. The important thing is that the test will fail where there are basic set-up and configuration issues with the web app.

The Test Project

The first step is to configure a test project. Obviously, your dependencies will vary based on what tools you decide to use, but the following will work for Xunit:

<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="6.0.5" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.2.0" />		
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.console" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" />

(See this post on Xunit libraries for details on the basic Xunit dependency list for .Net 6.)

The key here is to set-up the Web Application Factory:

var appFactory = new WebApplicationFactory<Program>();
var httpClient = appFactory.CreateClient();

We’ll come back to some specific issues with this exact code shortly, but basically, we’re setting up the in-memory test harness for the service (which in this case, is our web-app). You can obviously do this for an API in exactly the same manner. The rest of our test then looks like this:

using var response = await httpClient.GetAsync("/");

Assert.True(response.IsSuccessStatusCode);

If your test fails, and you want a fighting chance of working out why, you may wish to replace the assertion with this:

var content = await response.Content.ReadAsStringAsync();

That’s basically it; however, as that currently stands, you’ll start getting errors (some that you can see, and some that you cannot). It makes sense to make the HttpClient static, or at least raise it to the class level, as you only need to actually create it once.

Accessing the Main Project

The other issue that you’ll get here is that, because we’re using .Net 6 top level statements in Program.cs, it will tell you that it’s inaccessible. In fact, top level code does generate an implicit class, but it’s internal. This can be worked around my simply adding the following to the end of your Program.cs code:

public partial class Program { } // so you can reference it from tests

(See the references below for details of where this idea came from.)

Summary

In this post, we’ve seen how you can create an integration test that will assert that, at the very least, your web app runs. This method is much faster than constantly having to actually run your project. It obviously makes no assertions about how it runs, and whether what it’s doing is correct.

References

Example of testing top level statements

GitHub Issue reporting error with top level statements being tested

Stack Overflow question on how to access Program.cs from in program using top level statements

Tutorial video on integration tests

Unit Testing a Console Application

I’ve previously written about some Unusual things to do with a Console Application, including creating a game in a console application.

This post covers another unusual thing to want to do, but I was recently writing a console application, and wondered how you could test it. That is, without mocking the Console out completely. It turns out that, not only is this possible, it’s actually quite straightforward.

The key here are the methods Console.SetIn and Console.SetOut. These allow you to redirect the console input and output. Let’s take the Hello World example – to unit test this, the first thing to do is to redirect the Console.Out:

var writer = new StringWriter();        
Console.SetOut(writer); 

You can now unit test this by simply checking the StringWriter:

        [Fact]
        public void HelloWorldTest()
        {
            // Arrange
            var writer = new StringWriter();        
            Console.SetOut(writer); 

            // Act
            RunHelloWorld();

            // Assert
            var sb = writer.GetStringBuilder();
            Assert.Equal("Hello, World!", sb.ToString().Trim());
        }

You can similarly test an input; let’s take the following method:

        public static void GetName()
        {
            Console.WriteLine("What is your name?");
            string name = Console.ReadLine();
            Console.WriteLine($"Hello, {name}");            
        }

We can test both the input and the output of this method:

        [Fact]
        public void GetNameTest()
        {
            // Arrange
            var writer = new StringWriter();        
            Console.SetOut(writer); 

            var textReader = new StringReader("Susan");
            Console.SetIn(textReader);

            // Act
            GetName();

            // Assert
            var sb = writer.GetStringBuilder();
            var lines = sb.ToString().Split(Environment.NewLine, StringSplitOptions.TrimEntries);
            Assert.Equal("Hello, Susan", lines[1]);

        }

I’m not saying it’s necessarily good practice to unit test, what is essentially, logging, but it’s interesting to know that it’s possible!

Git Bisect with Automated Tests

Some time ago, I saw a talk at DDD North about git bisect (it may well have been this one). I blogged about it here. I can honestly say that it’s one of, if not the, most useful thing I’ve ever learnt in 10 minutes!

However, the problem with it is that you, essentially, have to tell it what’s good and what’s bad. In this post, I’ll be detailing how you can write automated tests to determine this, and then link them in.

Using existing tests to determine where something broke

In this example, I’ll be using this repository (feel free to do the same). The code in the repository is broken, but it hasn’t always been, and there are some tests within the repository that clearly weren’t run before check-in, and are now broke (I know this, because I purposely broke the code – although this does happen in real life, and often with good intentions – or at least not bad).

We’re using xUnit here; but I’m confident that any test framework would do the same. The trick is the dotnet test command; from the docs on that command:

If all tests are successful, the test runner returns 0 as an exit code; otherwise if any test fails, it returns 1.

As with the previous post, we need to start with a good and bad commit; for the purpose of this post, we’ll assume the current commit is bad, and the first ever commit was good.

git log

Will give a list of commits:

We need the first, which, for this repo, is:

3cbd757dd4e92d8ab2424c6a1e46a73bef23e056

Now we need to go through the process of setting up git bisect; the process is: you tell git that you wish to start a bisect:

git bisect start

Next, you tell git which commit is bad. In our case, that’s the current one:

git bisect bad

Finally, you tell it which was the last known good one – in our case, the first:

git bisect good 3cbd757dd4e92d8ab2424c6a1e46a73bef23e056

Now that we’re in a bisect, you could just tell git each time which is good and which bad (see the previous post on how you might do that), but here you can simply tell it to run the test:

git bisect run dotnet test GitBisectDemo/

This will then iterate through the commits and come back with the breaking commit:

That’s great, but in most cases you didn’t actually have a breaking test – something has stopped working, and you don’t know why or when. In these cases, you can write a new breaking test, and then give that to git bisect for it to tell you the last time that test passed.

Create a new test to determine where something broke

Firstly, the new test must not be checked in to source control, as this works by checking out code from previous releases. Then create your new test; for example:

namespace GitBisectDemo.Tests
{
    public class CalculationTests2
    {
        [Fact]
        public void DoCalculation_ReturnsCorrectValue()
        {
            // Arrange
            var calculationEngine = new CalculationEngine();

            // Act
            float result = calculationEngine.DoCalculation(2, 3);

            // Assert
            Assert.True(result > 4);
        }

    }
}

This is a new class, and it’s not checked into source control.

Executing a specific test from the command line

We now want to execute just one test, and you can do that using dotnet test like so:

dotnet test GitBisectDemo/ --filter "FullyQualifiedName=GitBisectDemo.Tests.CalculationTests2.DoCalculation_ReturnsCorrectValue"

You need to give it the full namespace and class name; we can now incorporate that into our git bisect:

git bisect start
git bisect bad
git bisect good 3cbd757dd4e92d8ab2424c6a1e46a73bef23e056

These are the same as before.

Note: if, at any time, you wish to cancel the bisect, it’s git bisect reset

Now, we feed the filtered test run into git bisect:

git bisect run dotnet test GitBisectDemo/ --filter  "FullyQualifiedName=GitBisectDemo.Tests.CalculationTests2.DoCalculation_ReturnsCorrectValue"

And we get a result when the new test would have broken.

That’s two cases covered. The final case is the situation whereby the thing that has broken cannot be determined by an automated test; say, for example, that an API call isn’t working correctly, or a particular process has slowed down. In this situation, we can have git bisect call out to an external executable.

Custom Console App

The first step here is to return a value (exit code) from the console app. In fact, this is deceptively simple:

static int Main(string[] args)
{
    var calculationEngine = new CalculationEngine();
    float result = calculationEngine.DoCalculation(3, 1);

    return (result == 4) ? 0 : -1;
}

Notice that all we’ve done here is change the Main signature to return an int. This console app could now be calling an external API, running a performance test, or anything that has a verifiable result.

Publish the console app

Because we’re calling this from another location, we’ll need to publish this test as a self-contained console app:

dotnet publish -r win-x64

Run the test

Again, the same set-up:

git bisect start
git bisect bad
git bisect good 3cbd757dd4e92d8ab2424c6a1e46a73bef23e056

Finally, we call the console app to run the test:

git bisect run GitBisectDemo/GitBisectDemo.ConsoleTest/bin/Debug/netcoreapp3.1/win-x64/GitBisectDemo.ConsoleTest.exe

References

https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test

https://stackoverflow.com/questions/155610/how-do-i-specify-the-exit-code-of-a-console-application-in-net

Manually Adding DbContext for an Integration Test

In EF Core, there is an extension method that allows you to add a DBContext, called AddDBContext. This is a really useful method, however, in some cases, you may find that it doesn’t work for you. Specifically, if you’re trying to inject a DBContext to use for unit testing, it doesn’t allow you to access the DBContext that you register.

Take the following code:

services.AddDbContext<MyDbContext>(options =>
                options.UseSqlServer());
         

I’ve previously written about using UseInMemoryDatabase. However, this article covered unit tests only – that is, you are able to instantiate a version of the DBContext in the unit test, and use that.

As a reminder of the linked article, if you were to try to write a test that included that DBContext, you might want to use an in memory database; you might, therefore, build up a DBContextOptions like this:

var options = new DbContextOptionsBuilder<MyDbContext>()
                .UseInMemoryDatabase(Guid.NewGuid().ToString())
                .EnableSensitiveDataLogging()
                .Options;
var context = new MyDbContext(options);

But in a scenario where you’re writing an integration test, you may need to register this with the IoC. Unfortunately, in this case, AddDbContext can stand in your way. The alternative is that you can simply register the DbContext yourself:

var options = new DbContextOptionsBuilder<MyDbContext>()
                .UseInMemoryDatabase(Guid.NewGuid().ToString())
                .EnableSensitiveDataLogging()
                .Options;
var context = new MyDbContext(options);
AddMyData(context);
services.AddScoped<MyDbContext>(_ => context);

AddMyData just adds some data into your database; for example:

private void AddTestUsers(MyDbContext context)
{            
    MyData data = new MyData()
    {
        value1 = "test",
        value2 = "1"                
    };
    context.MyData.Add(subject);
    context.SaveChanges();
}

This allows you to register your own, in memory, DbContext in your IoC.

Unit Testing With Entity Framework and Entity Framework Core 2.1

Entity Framework Core 2.1 comes with a nifty little feature: an In Memory Database setting. What this means, is that with a single option setting, your tests can interact directly with the database (or at least EF’s impression of the database) but not actually touch any physical database. In other words, you can write unit tests for data access; an example:

// Arrange
DbContextOptions<ApplicationDbContext> options = new DbContextOptionsBuilder<ApplicationDbContext>()
    .UseInMemoryDatabase(Guid.NewGuid().ToString())
    .EnableSensitiveDataLogging()                
    .Options;

using (var context = new ApplicationDbContext(options))
{
    context.Database.EnsureDeleted();
	ResourceCategory resourceCategory = new ResourceCategory()
    {
        Name = "TestCategory"
    }
 
    // Act
    _applicationDbContext.ResourceCategories.Add(resourceCategory);
    _applicationDbContext.SaveChanges();
};	
 
// Assert                
Assert.Equal("TestCategory", context.ResourceCategories.First().Name);               

To just quickly explain what this is doing: we have a DbContext called ApplicationDbContext and we’re building a set of options on top of that context. We’re then instantiating the context and cleaning the in memory database. Finally, we’re adding a new piece of data to the context and then asserting that it has been added.

Told you it was nifty.

But what about if you’re still using Entity Framework 6?

Glad you asked.

Out of the box, EF does not come with this kind of functionality; however, I recently came across (and contributed) to a NuGet library that provides just such a facility. It provides a wrapper for both Moq and Nsubstitute. The GitHub Repo is here.

Short Walks – XUnit Warning

As with many of these posts – this is more of a “note to self”.

Say you have an assertion that looks something like this in your Xunit test:

Assert.True(myEnumerable.Any(a => a.MyValue == "1234"));

In later versions (not sure exactly which one this was introduced it), you’ll get the following warning:

warning xUnit2012: Do not use Enumerable.Any() to check if a value exists in a collection.

So, Xunit has a nice little feature where you can use the following syntax instead:

Assert.Contains(myEnumerable, a => a.MyValue == "1234");

Short Walks – NSubstitute – Subclassing and Partial Substitutions

I’ve had this issue a few times recently. Each time I have it, after I’ve worked out what it was, it makes sense, but I keep running into it. The resulting frustration is this post – that way, it’ll come up the next time I search for it on t’internet.

The Error

The error is as follows:

“NSubstitute.Exceptions.CouldNotSetReturnDueToNoLastCallException: ‘Could not find a call to return from.”

Typically, it seems to occur in one of two circumstances: substituting a concrete class and partially substituting a concrete class; that is:

var myMock = Substitute.ForPartsOf<MyClass>();

Or:

var myMock = Substitute.For<MyClass>();

Why?

If you were to manually mock out an interface, how would you do that? Well, say you had IMyClass, you’d just do something like this:

public class MyClassMock : IMyClass 
{
	// New and imaginative behaviour goes here
}

All’s good – you get a brand new implementation of the interface, and you can do anything with it. But what would you do if you were trying to unit test a method inside MyClass that called another method inside MyClass; for example:

public class MyClass : IMyClass
{
	public bool Method1()
{
		int rowCount = ReadNumberOfRowsInFileOnDisk();
		Return rowCount > 10;
	}
	
	public int ReadNumberOfRowsInFileOnDisk()
	{
		// Opens a file, reads it, and returns the number of rows
	}
}

(Let’s not get bogged down in how realistic this scenario is, or whether or not this is good practice – it illustrates a point)

If you want to unit test Method1(), but don’t want to actually read a file from the disk, you’ll want to replace ReadNumberOfRowsInFileOnDisk(). The only real way that you can do this is to subclass the class; for example:

public class MyClassMock : MyClass

You can now test the behaviour on MyClass, via MyClassMock; however, you still can’t* override the method ReadNumberOfRowsInFileOnDisk() because it isn’t virtual. If you make the method virtual, you can override it in the subclass.

The same is true with NSubstitute – if you want to partially mock a class in this way, it follows the same rules as you must if you would to roll your own.

Footnotes

* There may, or may not be one or two ways to get around this restriction, but let’s at least agree that they are, at best, unpleasant.