Category Archives: .Net

Testing an Asp.Net Web App Using Integration Testing

I’ve recently been playing around with a tool called Scrutor. I’m using this in a project and it’s working really well; however, I came across an issue (not related to the tool per se). I had created an interface, but hadn’t yet written a class to implement it. Scrutor realised this was the case and started moaning at me. Obviously, I hadn’t written any unit tests around the non-existent class, but I did have a reasonably good test coverage for the rest of the project; however the project wouldn’t actually run.

To be clear, what I’m saying here is that, despite the test suite that I had running successfully, the program wouldn’t even start. This feels like a very askew state of affairs.

Some irrelevant background, I had a very strange issue with my Lenovo laptop, whereby, following a firmware update, the USB-C ports just stopped working – including to accept charge – so my laptop died. Sadly, I hadn’t followed good practice, with commits, and so part of my code was lost.

I’ve previously played around with the concept of integration tests in Asp.Net Core+, so I thought that’s probably what I needed here. There are a few articles and examples out there, but I couldn’t find anything that worked with Asp.Net 6 – so this is that.

In this post, we’ll walk through the steps necessary to add a basic test to your Asp.Net 6 web site. Note that this is not comprehensive – some dependencies will trip this up (e.g. database access); however, it’s a start. The important thing is that the test will fail where there are basic set-up and configuration issues with the web app.

The Test Project

The first step is to configure a test project. Obviously, your dependencies will vary based on what tools you decide to use, but the following will work for Xunit:

<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="6.0.5" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.2.0" />		
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.console" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" />

(See this post on Xunit libraries for details on the basic Xunit dependency list for .Net 6.)

The key here is to set-up the Web Application Factory:

var appFactory = new WebApplicationFactory<Program>();
var httpClient = appFactory.CreateClient();

We’ll come back to some specific issues with this exact code shortly, but basically, we’re setting up the in-memory test harness for the service (which in this case, is our web-app). You can obviously do this for an API in exactly the same manner. The rest of our test then looks like this:

using var response = await httpClient.GetAsync("/");

Assert.True(response.IsSuccessStatusCode);

If your test fails, and you want a fighting chance of working out why, you may wish to replace the assertion with this:

var content = await response.Content.ReadAsStringAsync();

That’s basically it; however, as that currently stands, you’ll start getting errors (some that you can see, and some that you cannot). It makes sense to make the HttpClient static, or at least raise it to the class level, as you only need to actually create it once.

Accessing the Main Project

The other issue that you’ll get here is that, because we’re using .Net 6 top level statements in Program.cs, it will tell you that it’s inaccessible. In fact, top level code does generate an implicit class, but it’s internal. This can be worked around my simply adding the following to the end of your Program.cs code:

public partial class Program { } // so you can reference it from tests

(See the references below for details of where this idea came from.)

Summary

In this post, we’ve seen how you can create an integration test that will assert that, at the very least, your web app runs. This method is much faster than constantly having to actually run your project. It obviously makes no assertions about how it runs, and whether what it’s doing is correct.

References

Example of testing top level statements

GitHub Issue reporting error with top level statements being tested

Stack Overflow question on how to access Program.cs from in program using top level statements

Tutorial video on integration tests

A Cleaner Program.cs / Startup.cs with Scrutor

I’ve previously written about the Scrutor library. However, this post covers something that has long irritated me about using an IoC container. Typically, when you have a fairly complex site, you’ll end up with dozens of statements like the following:

builder.Services.AddScoped<ISearchService, SearchService>();
builder.Services.AddScoped<IResourceDataAccess, ResourceDataAccess>();

It turns out that one of the other things that Scrutor can do for you is to work out which dependencies you need to register. For example, let’s consider the two classes above; let’s say that the first is in the main assembly of the project:

builder.Services.Scan(scan => scan
    .FromCallingAssembly()
        .AddClasses(true)
            .AsMatchingInterface()
            .WithScopedLifetime());

So, what does this do?

Well, FromCallingAssembly points it at the main assembly (that is, the one which you’re calling this registration from). FromClasses(true) then returns a list of all public classes and interfaces.

Finally, AsMatchingInterface matches classes with their interfaces, assuming a one-to-one pairing (if you don’t have that then there are other options to cope with that); and WithScopedLifetime will register them as scoped.

That worked well, but when I ran it, I realised that the second class (ResourceDataAccess) hadn’t registered. The reason being that it wasn’t from the calling assembly, but lived in a referenced project. An easy way to fix this was:

builder.Services.Scan(scan => scan
    .FromCallingAssembly()
        .AddClasses(true)
            .AsMatchingInterface()
            .WithScopedLifetime()
    .FromAssemblyOf<IResourceDataAccess>()
        .AddClasses(true)
            .AsMatchingInterface()
            .WithScopedLifetime());

Notice that we can simply return to the start following the With…Lifetime(), and this time, we tell it to register any classes found in the same assembly as IResourceDataAccess.

If we look at the list of assemblies, we can see this has worked:

What this means is that, each time you add a new class, you don’t have to add a registration in the startup / program file. This is perhaps a good and bad thing: arguably, if the list gets so large that it’s noticeable, then you might have gauged your decomposition incorrectly.

Unit Testing a Console Application

I’ve previously written about some Unusual things to do with a Console Application, including creating a game in a console application.

This post covers another unusual thing to want to do, but I was recently writing a console application, and wondered how you could test it. That is, without mocking the Console out completely. It turns out that, not only is this possible, it’s actually quite straightforward.

The key here are the methods Console.SetIn and Console.SetOut. These allow you to redirect the console input and output. Let’s take the Hello World example – to unit test this, the first thing to do is to redirect the Console.Out:

var writer = new StringWriter();        
Console.SetOut(writer); 

You can now unit test this by simply checking the StringWriter:

        [Fact]
        public void HelloWorldTest()
        {
            // Arrange
            var writer = new StringWriter();        
            Console.SetOut(writer); 

            // Act
            RunHelloWorld();

            // Assert
            var sb = writer.GetStringBuilder();
            Assert.Equal("Hello, World!", sb.ToString().Trim());
        }

You can similarly test an input; let’s take the following method:

        public static void GetName()
        {
            Console.WriteLine("What is your name?");
            string name = Console.ReadLine();
            Console.WriteLine($"Hello, {name}");            
        }

We can test both the input and the output of this method:

        [Fact]
        public void GetNameTest()
        {
            // Arrange
            var writer = new StringWriter();        
            Console.SetOut(writer); 

            var textReader = new StringReader("Susan");
            Console.SetIn(textReader);

            // Act
            GetName();

            // Assert
            var sb = writer.GetStringBuilder();
            var lines = sb.ToString().Split(Environment.NewLine, StringSplitOptions.TrimEntries);
            Assert.Equal("Hello, Susan", lines[1]);

        }

I’m not saying it’s necessarily good practice to unit test, what is essentially, logging, but it’s interesting to know that it’s possible!

Using Scrutor to Implement the Decorator Pattern

I recently came across a very cool library, thanks to this video by Nick Chapsas. The library is Scrutor. In this post, I’m going to run through a version of the Open-Closed Principle that this makes possible.

An Overly Complex Hello World App

Let’s start by creating a needlessly complex app that prints Hello World. Instead of simply printing Hello World we’ll use DI to inject a service that prints it. Let’s start with the main program.cs code (in .Net 6):

using Microsoft.Extensions.DependencyInjection;
using scrutortest;

var serviceCollection = new ServiceCollection();

serviceCollection.AddSingleton<ITestLogger, TestLogger>();

var serviceProvider = serviceCollection.BuildServiceProvider();

var testLogger = serviceProvider.GetRequiredService<ITestLogger>();
testLogger.Log("hello world");

Impressive, eh? Here’s the interface that we now rely on:

internal interface ITestLogger
{
    public void Log(string message);
}

And here is our TestLogger class:

    internal class TestLogger : ITestLogger
    {
        public void Log(string message)
        {
            Console.WriteLine(message);
        }
    }

If you implement this, and run it, you’ll see that it works fine – almost as well as the one line version. However, let’s imagine that we now have a requirement to extend this class. After every message, we need to display —OVER— for… some reason.

Extending Our Overly Complex App to be Even More Pointless

There’s a few ways to do this: you can obviously just change the class itself, but that breaches the Open-Closed Principle. That’s where the Decorator Pattern comes in. Scrutor allows us to create a new class that looks like this:

    internal class TestLoggerExtended : ITestLogger
    {
        private readonly ITestLogger _testLogger;

        public TestLoggerExtended(ITestLogger testLogger)
        {
            _testLogger = testLogger;
        }

        public void Log(string message)
        {
            _testLogger.Log(message);
            _testLogger.Log("---OVER---");
        }
    }

There’s a few things of note here: firstly, we’re implementing the same interface as the main / first class; secondly, we’re Injecting said interface into our constructor; and finally, in the Log method, we’re calling the original class. Obviously, if you just register this in the DI container as normal, bad things will happen; so we use the Scrutor Decorate method:

using Microsoft.Extensions.DependencyInjection;
using scrutortest;

var serviceCollection = new ServiceCollection();

serviceCollection.AddSingleton<ITestLogger, TestLogger>();
serviceCollection.Decorate<ITestLogger, TestLoggerExtended>();

var serviceProvider = serviceCollection.BuildServiceProvider();

var testLogger = serviceProvider.GetRequiredService<ITestLogger>();
testLogger.Log("hello world");

If you now run this, you’ll see that the functionality is very similar to inheritance, but you haven’t coupled the two services directly:

Chaos Monkey – Part 4 – Creating an Asp.Net 6 Application that Caches an Error

This is a really strange post, but it’s a line up for a different post; however, I felt it made sense to be a post in its own right – it follows on from a trend I have of creating things that break on purpose. For example, here’s a post from a few years ago where I discussed how you might force a machine to run out of memory.

In this case, I’m creating a simple application that runs fine, but at a random point, it generates an error, which it caches, and then is broken until the application is restarted.

Why?

I’m working on some alerting and resilience experiments at the minute, and having an unstable application is useful for those tests. Also, this is not an unusual scenario – I mean, obviously, writing an application that purposes crashes after it’s broken, and from then on, is unusual; but having an application that does this somewhere in your estate may not be so unusual.

How

I’ve set-up a bog standard Asp.Net MVC 6 application. I then installed the following package:

Install-Package System.Runtime.Caching

Finally, I changed the default Privacy controller action to potentially crash:

public IActionResult Privacy()
{
    string result = Crash();
    return View(model: result);
}

Here, I’m feeding a string into the privacy view as its model. The Crash method has a 1 in 10 chance of caching an error:

        private string Crash()
        {
            if (!_memoryCache.TryGetValue("Error", out string errorCache))
            {
                if (_random.Next(10) == 1)
                {
                    _memoryCache.Set("Error", "Now broken!");
                    return "Now broken";
                }
            }
            else
            {
                throw new Exception("Some exception");
            }

            return "Working fine";
        }

I then just display the model in the view (privacy.cshtml):

@model string
@{
    ViewData["Title"] = "Privacy Policy";
}
<h1>@ViewData["Title"]</h1>
<h1>@Model</h1>

<p>Use this page to detail your site's privacy policy.</p>

Now, if you run it, somewhere between 2 and 15 times, you’re likely to see it break, and need to restart to fix.

Beginner’s Guide to Docker – Part 4 – Viewing Docker Logs

In this post (part of a series around docker basics) I’ll be disucssing how you can view logs for running docker containers.

As usual with my posts, there’s very little here that you can’t get from the the docs – although I’d like to think these posts are slightly more accessible (especially for myself).

Viewing Logs

In fact, the process of viewing logs is remarkably easy; once you’ve launched the container, you need the ID:

docker ps

Once you have the container ID, you can feed it into the docker container logs command:

docker container logs --follow dcd204c97421

The follow argument basically leaves a session open, and updates with any new log entries (leave it out for a static view of the logs).

The docker desktop app actually provides the same functionality out of the box (but provides you with a nice search and formatting facility).

Writing Logs

Obviously, this varies from language to language, so I’ll stick to .Net here. It’s actually very straightforward:

builder.Services.AddLogging();

You can now log in the same way as you would for any other .net application; simply inject into the constructor:

        public MyService(
            ILogger<MyService> logger)
        {
            _logger = logger;
        }

And then log as you would normally:

_logger.LogWarning("something happened");

References

Constructor Injection in .Net

Docker Container Logs

Running Hangfire with .Net 6 and Lite Storage

One of the things that I’ve been looking into recently is Hangfire. I’ve used this in the past, but not for a while, and so I wanted to get familiar with it again. I imagine there’ll be a couple more posts on the way, but in this one, I’m going to detail how to get Hangfire running using .Net 6, and a Lite DB Storage. I did initially intend to create this with a SQLite DB; however, when I did, I found that the Hangfire messages were sending multiple times; after some searching, I came across this GitHub issue, which suggested the Hangfire.Lite storage.

In this particular article, we’ll look specifically at running Hangfire with an Asp.Net MVC project. In .Net 6, the default template looks like this:

The first step here is that you’ll need some additional refences; here’s the csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">

	<PropertyGroup>
		<TargetFramework>net6.0</TargetFramework>
		<Nullable>enable</Nullable>
		<ImplicitUsings>enable</ImplicitUsings>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="Hangfire.AspNetCore" Version="1.7.27" />
		<PackageReference Include="Hangfire.Core" Version="1.7.27" />
		<PackageReference Include="Hangfire.LiteDB" Version="0.4.1" />
	</ItemGroup>

</Project>

In addition to the Hangfire Core and AspNetCore projects, there’s the LiteDB package.

The next step is, in program.cs:

using Hangfire;
using Hangfire.LiteDB;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddControllersWithViews();

// HF
builder.Services.AddHangfire(configuration =>
{
    configuration.UseLiteDbStorage("./hf.db");    
});
builder.Services.AddHangfireServer();
// HF - End

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Home/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}
. . .

That’s actually it – that’s all you need to add Hangfire! However, it won’t, technically speaking, do anything. To test, edit the HomeCrontroller.cs:

        public IActionResult Index()
        {
            var id = BackgroundJob.Enqueue(() => DoSomething());
            return View();
        }

        public void DoSomething()
        {
            Console.WriteLine("Test");
            Console.WriteLine($"test2 {Guid.NewGuid()}");
        }

When you run the project, you should see Hangfire fire up and start printing outputs:

When you run it, don’t be too impatient – but it should print the test output.

References

https://github.com/HangfireIO/Hangfire/issues/1025

https://docs.microsoft.com/en-us/aspnet/core/fundamentals/routing?view=aspnetcore-6.0

https://docs.hangfire.io/en/latest/getting-started/index.html

Manually Parsing a Json String Using System.Text.Json

In this post from 2016 I gave some details as to how you could manually parse a JSON string using Newtonsoft JSON.NET. What I don’t think I covered in that post, was why you may wish to do that, nor did I cover how you could do that using System.Text.Json (although since the library was only introduced in .Net Core 3 – circa 2019, that part couldn’t really be helped!)

Why?

Let’s start with why you would want to parse a JSON string manually – I mean, the serialisation functions in, pretty much, any JSON library are really good.

The problem is coupling: by using serialisation, you’re coupling your data to a given shape, and very tightly coupling it to that shape, too. So much so, that a common pattern if you’re passing data between two services, and using serialisation, it to share the model class between the services. This sounds quite innocuous at first glance, but let’s consider a few factors (I’m assuming we’re talking exclusively about .Net, but I imagine the points are valid outside of that, too):

1. Why are you serialising and de-serialising the data to begin with?
2. Single Responsibility Principle.
3. Microservices.

Let’s start with (1). Your answer may be different, but typically, if you’re passing data as a string, it’s because you’re trying to remove a dependency to a given complex type. After all, a string can be passed anywhere: an API call, a message broker, even held in a database.

What has this got to do with the SRP (2)? Well, the SRP is typically used to describe the reason that a module has to change (admittedly it is slightly mis-named). Let’s see how the two modules may interact:

Now, let’s look at the interaction with a common module:

As you can see, they both have a dependency on a single external (external to the service) dependency. If the CustomerModel changes, then both services may also need to change, but they also need to change for alterations for business rules that relate to the module itself: so they now have two reasons to change.

Of course, you don’t have to have a common dependency like this; you could structure your system like this:

However, you don’t solve your problem – in fact, you arguably make it worse: if you change CustomerModel referenced by Service 1 you potentially break Service 2, so you now need to change CustomerModel referenced by Service 2, and Service 2!

Just to clarify what I’m saying here: there may be valid reasons for both of these designs – but if you use them, then be aware that you’re coupling the services to each other; which brings us to point (3): if you’re trying to create a Service Oriented Architecture of any description, then this kind of coupling may hinder your efforts.

The Solution

A quick caveat here: whatever you do in a system, the parts of that system will be coupled to some extent. For example, if you have built a Microservice Architecture where your system is running a nuclear reactor, and then you decide to change one of the services from monitoring the cooling tower to, instead, mine bit-coins, you’re going to find that there is an element of coupling. Some of that will be business coupling (i.e. the cooling tower may overheat), but some will be technical – a downstream service may depend on the monitoring service to assert that something is safe.

Apologies, I know absolutely nothing about nuclear reactors; or, for that matter, about mining bit-coin!

All that said, if you manually parse the JSON that’s received, you remove some dependency on the shape of the data.

The following reads a JSON document, and iterates through an array:

            using var doc = JsonDocument.Parse(json);
            var element = doc.RootElement;

            foreach (var eachElement in element.EnumerateArray())
            {
                string name = eachElement.GetProperty("Name").GetString();
                decimal someFigure = eachElement.GetProperty("SomeFigure").GetDecimal();

                if (someFigure > 500)
                {
                    Console.WriteLine($"{name} has more than 500!");
                }
            }

As you can see, if the property name of SomeFigure changed, the code would break; however, there may be a dozen more fields in each element that could change, and we wouldn’t care.

Compiling Code Files in C# Using Roslyn

I started investigating this for a unit test. I wanted to compile a separate project, and then copy the output into the test directory. As it happens, by the time I’d figured this out in Roslyn, I’d worked out it probably wasn’t right for the test that I was writing; however, given the amount of effort involved in actually piecing together the tiny amount of documentation available, I definitely thought it was worthy of documenting.

This particular post will focus on reading code files and compiling them into an executable or library. I may play around with this some more in the future: I suspect this might have quite a lot of mileage in the unit test space.

Dependencies

Let’s start with the dependencies:

    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="3.11.0" />
    <PackageReference Include="Microsoft.CodeDom.Providers.DotNetCompilerPlatform" Version="3.6.0" />

It’s worth bearing in mind that there are a few other libraries knocking about from previous incarnations of Roslyn, and similar tools; however, it’s my understanding that these are not in the Microsoft namespace. I probably spent the first couple of hours trying to get this to work with CSharpCodeProvider. If you found your way here searching for this error:

System.PlatformNotSupportedException: ‘Operation is not supported on this platform.’

Then you’re in good company.

Code

The following code will produce a library called qwerty.dll; it requires a variable giving the sourceFilesLocation and another, destinationLocation:

	// 1
            DirectoryInfo d = new DirectoryInfo(sourceFilesLocation); 
            string[] sourceFiles = d.EnumerateFiles("*.cs", SearchOption.AllDirectories)                
                .Select(a => a.FullName).ToArray();
            
	// 2
            List<SyntaxTree> trees = new List<SyntaxTree>();
            foreach (string file in sourceFiles)
            {
                string code = File.ReadAllText(file);
                SyntaxTree tree = CSharpSyntaxTree.ParseText(code);
                trees.Add(tree);
            }
            
	// 3
            MetadataReference mscorlib =
                    MetadataReference.CreateFromFile(typeof(object).Assembly.Location);
            MetadataReference codeAnalysis =
                    MetadataReference.CreateFromFile(typeof(SyntaxTree).Assembly.Location);
            MetadataReference csharpCodeAnalysis =
                    MetadataReference.CreateFromFile(typeof(CSharpSyntaxTree).Assembly.Location);

            MetadataReference[] references = { mscorlib, codeAnalysis, csharpCodeAnalysis };

	// 4
            var compilation = CSharpCompilation.Create("qwerty.dll",
                   trees,
                   references,
                   new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
            var result = compilation.Emit(Path.Combine(destinationLocation, "qwerty.dll"));

Let’s break the code down (you may notice that there are 4 comments, that to the untrained eye, appear useless).

1. The first part scans the source files directory and returns a list of files.
2. We then build a syntax tree list by reading all the files and calling the ParseText method on each; they are added to a list of parsed files.
3. We now build up a list of references needed to compile the code.
4. Finally, we create the dll, passing in the files that we built up in 2 & 3. .Create creates an in-memory version of the dll, so we then need to call .Emit to write it to disk.

Summary and caveats

This is a very simplistic example – as I’ve said, I need to spend some more time looking at some of the more advanced cases, but if you have a very simple library, this will compile and produce the DLL for you.

References

https://stackoverflow.com/questions/54364785/c-sharp-compile-c-sharp-code-at-runtime-with-roslyn

https://blog.dangl.me/archive/integration-testing-in-memory-compiled-code-with-roslyn/

https://www.tugberkugurlu.com/archive/compiling-c-sharp-code-into-memory-and-executing-it-with-roslyn

https://docs.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/get-started/syntax-transformation

https://stackoverflow.com/questions/32769630/how-to-compile-a-c-sharp-file-with-roslyn-programmatically