Monthly Archives: August 2020

Change the Default Asp.Net Core Layout to Use Feature Folders

One of the irritating things about the Asp.Net Core default project is that the various parts of your system are arranged by type, as opposed to function. For example, if you’re working on the Accounts page, you’re likely going to want to change the view, the controller and, perhaps, the model; you are, however, unlikely to want to change the Sales Order controller as a result of your change: so why have the AccountsController and SalesOrderController in the same place, but away from the AccountsView?

If you create a new Asp.Net Core MVC Web App:

Then you’ll get a layout like this (in fact, exactly like this):

If your web app has two or three controllers, and maybe five or six views, then this works fine. When you start getting a larger, more complex app, you’ll find that you’re scrolling through your solution trying to find the SalesOrderController, or the AccountsView.

One way to alleviate this, is to re-organise your project to reference features in vertical slices. For example:

There’s not much to either of these, but let’s just put them in for the sake of completeness; the View:

@{
    ViewData["Title"] = "Wibble Page";
}

<div class="text-center">
    <h1 class="display-4">Wibble</h1>    
</div>

And the controller:

namespace WebApplication1.Wibble
{
    public class WibbleController : Controller
    {
        public IActionResult Index()
        {
            return View();
        }
    }
}

The problem here is that the engine won’t know where to look for the views. We can change that by changing the ConfigureServices method in Startup.cs:

        public void ConfigureServices(IServiceCollection services)
        {
            . . . 
            services.Configure<RazorViewEngineOptions>(options =>
            {
                options.ViewLocationFormats.Clear();
                options.ViewLocationFormats.Add($"/Wibble/{{0}}{RazorViewEngine.ViewExtension}");
                options.ViewLocationFormats.Add($"/Views/Shared/{{0}}{RazorViewEngine.ViewExtension}");
            });
        }

Let’s also change the default controller action (in the Configure method of Startup.cs):

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }

            . . . 
            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Wibble}/{action=Index}/{id?}");
            });
        }

There’s more than a few libraries that will handle this for you (here’s one by the late Scott Allen), but it’s always nice to be able to do such things manually before you resort to a third-party library.

Manually Creating a Test Harness in .Net

Let me start by saying that this post isn’t intended to try to replace Selenium, or Cypress, or whatever UI testing tools you may choose to use. In fact, it’s something that I did for manual testing, although it’s not difficult to imagine introducing some minor automation.

The Problem

Imagine that you have a solution that requires some data – perhaps it requires a lot of data, because you’re testing some specific performance issue, or perhaps you just want to see what the screen looks like when you have a lot of data. Let’s also imagine that you’re repeatedly running your project for one reason or another, and adding data, or whatever.

My idea here was that I could create a C# application that scripts this process, but because it’s an internal application, I could give it access to the data layer directly.

The Solution

Basically, the solution to this (and to many things) was a console app. Let’s take a solution that implements a very basic service / repository pattern:

From this, we can see that we have a pretty standard layout, and essentially, what we’re trying to do is insert some data into the database. It’s a bonus if we can add some test coverage while we’re at it (manual test coverage is still test coverage – it just doesn’t show up on your stats). So, if you’re using a REST type pattern, you might want to use the controller endpoints to add the data; but for my purpose, I’m just going to add the data directly into the data access layer.

Let’s see what the console app looks like:

        static async Task Main(string[] args)
        {
            IConfiguration configuration = new ConfigurationBuilder()
                  .AddJsonFile("appsettings.json", true, true)
                  .Build();

            // Ensure the DB is populated
            var dataAccess = new TestDataAccess(configuration.GetConnectionString("ConnectionString"));
            if (dataAccess.GetDataCount() == 0)
            {
                var data = new List<MyData>();

	     // Generate 100 items of data
                for (int j = 0; j <= 100; j++)
                {
		var dataItem = CreateTestItem();
                      data.Add(dataItem);
                }
                dataAccess.AddDataRange(data);
            }

            // Launch the site            
            await RunCommand("dotnet", "build");
            await RunCommand("dotnet", "run", 5);

            System.Diagnostics.Process.Start(@"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe", @"https://localhost:5001");
        }

Okay, so let’s break this down: there’s essentially three sections to this: configuration, adding the data, and running the app.

Configuration

We’ll start with the configuration:

       IConfiguration configuration = new ConfigurationBuilder()
                  .AddJsonFile("appsettings.json", true, true)
                  .Build();

        // Ensure the DB is populated
        var dataAccess = new TestDataAccess(configuration.GetConnectionString("ConnectionString"));

Because we’re using a console app, we’ll need to get the configuration; you could copy the appsettings.json, but my suggestion would be to add a link; that is, add an existing item, and select that item from the main project, then choose to “Add As Link” (this is not a new feature):

This means that you’ll be able to change the config file, and it will be reflected in the test harness.

Creating the data

There’s not too much point in me covering what’s behind TestDataAccess – suffice to say that it encapsulates the data access layer; which, as a minimum, requires the connection string.

It’s also worth pointing out that we check whether there is any data there before running it. Depending on your specific use-case, you may choose to remove this.

Building, running, and launching the app

Okay, so we’ve now added our data, we now want to build the main application – thanks to the command line capabilities of .Net Core, this is much simpler than it was when we used to have to try and wrangle with MSBuild!

    // Launch the site            
    await RunCommand("dotnet", "build");
    await RunCommand("dotnet", "run", 5);

    await RunCommand(@"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe", @"https://localhost:5001");

RunCommand probably needs a little more attention, but before we look at that, let’s think about what we’re trying to do:

1. Build the application
2. When the application has built, run the application
3. Once the application is running, navigate to the site

RunCommand looks like this:

        private static async Task RunCommand(string command, string? args = null, int? waitSecs = -1)
        {
            Console.WriteLine($"Executing command: {command} (args: {args})");

            Process proc = new System.Diagnostics.Process();
            proc.StartInfo.WorkingDirectory = @"..\..\..\..\MyApp.App";
            proc.StartInfo.FileName = command;
            proc.StartInfo.Arguments = args ?? string.Empty;

            proc.Start();

            if ((waitSecs ?? -1) == -1)
            {
                proc.WaitForExit();
            }
            else
            {
                if (waitSecs! == 0) return;
                await Task.Delay(waitSecs!.Value * 1000);
            }
        }

“But this looks inordinately complicated for a simple wrapper for running a process!” I hear you say.

It is a bit, but the essence is this: when running the build command, we need to wait for it to complete, when running the run command, we can’t wait for it to complete, because it never will; but we do need to move to the next thing; and when we launch the site, we don’t really care whether it waits or not after that.

Summary

I appreciate that some of you may be regretting spending the time reading through this, as all I’ve essentially done is script some data creation and run an application; but I imagine there are some people out there, like me, that want to see (visually) what their app looks like with different data shapes.

Feature Flags in Asp.Net Core – Advanced Features – Targeting a Specific Audience using Feature Filters

In this post, I introduced the concept, and Microsoft’s implementation, of Feature Flags. In fact, there’s a lot more to both than I covered in this initial post. In this post, I want to cover how you could use this to turn a particular feature on for a single user, or a group of users. You can even specify groups for those users, and allow all, or some of those users to see the feature.

It’s worth noting that, at the time of writing, this functionality is currently only in preview; you’ll need the following NuGet package (or later) for it to work:

<PackageReference Include="Microsoft.FeatureManagement.AspNetCore" Version="2.2.0-preview" />

Before we get into how to use this, let’s have a quick look at the source. The interesting method is EvaluateAsync. Essentially this method returns a boolean indicating whether or not the feature is available; you could simply return true and the feature would always be enabled; but let’s see an abridged version of this method:

        public Task<bool> EvaluateAsync(FeatureFilterEvaluationContext context, ITargetingContext targetingContext)
        {
            . . .

            //
            // Check if the user is being targeted directly
            if (targetingContext.UserId != null &&
                settings.Audience.Users != null &&
                settings.Audience.Users.Any(user => targetingContext.UserId.Equals(user, ComparisonType)))
            {
                return Task.FromResult(true);
            }

            //
            // Check if the user is in a group that is being targeted
            if (targetingContext.Groups != null &&
                settings.Audience.Groups != null)
            {
                foreach (string group in targetingContext.Groups)
                {
                    GroupRollout groupRollout = settings.Audience.Groups.FirstOrDefault(g => g.Name.Equals(group, ComparisonType));

                    if (groupRollout != null)
                    {
                        string audienceContextId = $"{targetingContext.UserId}\n{context.FeatureName}\n{group}";

                        if (IsTargeted(audienceContextId, groupRollout.RolloutPercentage))
                        {
                            return Task.FromResult(true);
                        }
                    }
                }
            }

            //
            // Check if the user is being targeted by a default rollout percentage
            string defaultContextId = $"{targetingContext.UserId}\n{context.FeatureName}";

            return Task.FromResult(IsTargeted(defaultContextId, settings.Audience.DefaultRolloutPercentage));
        }

So, the process is that it looks for specific users and, if it finds them, they can see the feature; if it cannot find them then it looks through the groups (we’ll come back to IsTargeted later), and finally reverts to the DefaultRolloutPercentage – again, we’ll look into what that is later on.

Let’s start with a single user

If you have a look at the previous post, you’ll see that the Feature Management system is being added using the following syntax:

services.AddFeatureManagement();

In order to add one of the pre-defined filters, we’ll need to add to this like so:

            services.AddFeatureManagement()
                .AddFeatureFilter<TargetingFilter>();

You’ll also need the following class importing:

using Microsoft.FeatureManagement.FeatureFilters;

If you run this now, you’ll get the following error:

System.InvalidOperationException: ‘Unable to resolve service for type ‘Microsoft.FeatureManagement.FeatureFilters.ITargetingContextAccessor’ while attempting to activate ‘Microsoft.FeatureManagement.FeatureFilters.TargetingFilter’.’

The reason being that you need to identify a ITargetingContextAccessor. What exactly this looks like is up to the implementer, but you’ll find an example implementation here.

We’ll come back to this shortly in the groups section.

Let’s now have a look at what our appsettings.json might look like:

  "FeatureManagement": {
    "MyFeature": {
      "EnabledFor": [
        {
          "Name": "Targeting",
          "Parameters": {
            "Audience": {
              "Users": [
                "[email protected]"
              ]
            }
          }
        }
      ]      
    }

If we have a look at the default HttpContextTargetingContextAccessor (see the link above for the ITargetingContextAccessor), we’ll see that the UserId is being set there:

            TargetingContext targetingContext = new TargetingContext
            {
                UserId = user.Identity.Name,
                Groups = groups
            };

This isn’t particularly controversial – at least the User Id part isn’t; however, it doesn’t have to be this; for example, you could get the family_name claim, and return that – and then you could target your feature to anyone named Smith. It’s a bit of a silly example, but the point is that you can customise how this works (in fact, you can write a completely custom filter, which I’ll probably cover in a later post).

This part of the Feature Management is not to be underestimated: you could release a feature, in live, to only one or two Beta Testers. However, it is quite specific; that’s where Groups come in.

Groups

Groups are slightly more interesting. You can specify which groups are in and out using the following configuration:

  "FeatureManagement": {

    "MyFeature": {
      "EnabledFor": [
        {
          "Name": "Targeting",
          "Parameters": { 
            "Audience": {
              "Users": [],
              "Groups": [
                {
                  "Name": "Group1",
                  "RolloutPercentage": 80
                },
                {
                  "Name": "Group2",
                  "RolloutPercentage": 40
                }
              ],
              "DefaultRolloutPercentage": 20
            }
          }
        }
      ]
    },

The RolloutPercentage here indicates what proportion of the group that you wish to be included.

Do you remember the IsTargeted method from earlier? This is what it looks like:

        private bool IsTargeted(string contextId, double percentage)
        {
            byte[] hash;

            using (HashAlgorithm hashAlgorithm = SHA256.Create())
            {
                hash = hashAlgorithm.ComputeHash(Encoding.UTF8.GetBytes(contextId));
            }

            //
            // Use first 4 bytes for percentage calculation
            // Cryptographic hashing algorithms ensure adequate entropy across hash
            uint contextMarker = BitConverter.ToUInt32(hash, 0);

            double contextPercentage = (contextMarker / (double)uint.MaxValue) * 100;

            return contextPercentage < percentage;
        }

To an extent, this is a random selection, but it uses the User ID and feature name to calculate the hash for that selection (that’s what gets passed into contextId), meaning that the same user will see the same thing each time. You may also find when playing with this that for small numbers, it doesn’t really match the expectation; for example, at 40%, you would expect around two out of five users to see the feature, but when I ran my test, all the five users could see the feature. Larger numbers work better, although the fact that this is tied to the User Id makes it a little tricky to test (you can’t simply launch the site and press Ctrl-F5 until it switches over).

Again, it’s worth pointing out that what a group is, is determined by you (or at least the creator of the HttpContextTargetingContextAccessor). This means that you can base this on a claim, on the first letter of the username, the time of day, anything you like. I haven’t tried it, but I suspect you could put a DB query in here, too. That’s probably not the best idea, because it gets called a lot, but I believe it’s possible.

Default Rollout Percentage

Here we have a catch-all – if the user is not in the group, and not identified as a user, this will allow you to expose your feature to a percentage of the user base. Again, this isn’t something you can easily check by refreshing your page, as it’s based on a hash of the user, group, and feature name. In fact, this won’t work very well at all if you’re not using any kind of identiy.

References

http://dontcodetired.com/blog/post/Using-the-Microsoft-Feature-Toggle-Library-in-ASPNET-Core-(MicrosoftFeatureManagement)

https://github.com/microsoft/FeatureManagement-Dotnet

CORS Errors while playing with the HTML Canvas

In this post, I wrote about how you might draw a graph using the HTML canvas; my next step was to make that more efficient; however, upon trying to import a Javascript module into my script:

import animatedGraphics from './animatedGraphics';

I started getting this error:

Access to script at ‘file:///C:/repos/webanimations/animated-columns-optimised/columns.js’ from origin ‘null’ has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https.

After a bit of digging, I uncovered a feature of ES6 that prevents opening this directly in a browser.

Hosting the site locally

The issue is caused because the file is being opened directly; so there seemed to be a couple of ways around this: one is to disable the security in Chrome, although try as I might, I couldn’t manage to get it to give up the ghost: I tried various combinations around the –disable-web-security flag of Chrome.

The second option is to host the site locally. For a brief moment I considered using something like IIS Express; but fortunately, I came across this tool that hosts a site locally for you.

It can be installed as an npm package:

npm install --global http-server

Once installed, you just navigate to the relevant directory, and type http-server:

C:\repos\webanimations\animated-columns-optimised>http-server
Starting up http-server, serving ./
Available on:
  http://192.168.1.79:8080
  http://127.0.0.1:8080
  http://172.17.230.225:8080
Hit CTRL-C to stop the server

You can then navigate to your specific page; for example:

http://127.0.0.1:8080/columns

And no more CORS error (doesn’t quite work yet, but that’s a whole different story).

Drawing Custom Graphs in HTML and Javascript

While recently playing with the HTML Canvas again, it occurred to me that this power could be used for evil (or statistics are it is commonly known these days).

In this post, I’m going to draw an animated chart using the HTML Canvas and Javascript.

Let’s start with the HTML:

<!DOCTYPE html>
<html>
<head>
    <link rel="stylesheet" type="text/css" href="test.css">
    <script src="test.js"></script>
</head>
<body onload="doDrawing()">
    <canvas id="canvas">
    </canvas>
</body>
</html>

Again, as with my previous post (linked above), I’ll remind you that this will not cope with screen resizing. The CSS, again, isn’t much to write home about:

* { 
    margin:0; 
    padding:0; 
}
canvas {  
    display: block;    
}

Javascript

Finally, we come to the Javascript: this is a bit more involved, so I’ll break it down into the individual functions; we’ll start with the top level one:

const doDrawing = () => {
    var c = document.getElementById("canvas");
    
    c.width = window.innerWidth;
    c.height = window.innerHeight;
    var ctx = c.getContext("2d");    
    drawGraphTop(50, 5, 500);
    drawColumn(100, 500, 4, 'January');
    drawColumn(150, 350, 5, 'February');
    drawColumn(200, 150, 6, 'March');
}

All we’re doing here is getting a handle to the canvas, setting the height and width so that we fill the screen, and then calling some functions to do the drawing. The helper method to get the context is trivial, but for completeness:

function getContext() {
    var c = document.getElementById("canvas");    
    var ctx = c.getContext("2d");        
    return ctx;
}

There’s not much to explain here, we’re just getting the context from the canvas and returning it – thereby saving 3 lines of code each time we do that. The drawGraphTop function is a little more interesting:

const drawGraphTop = (top, interval, width) => {
    let ctx = getContext();
    ctx.beginPath();
    ctx.rect(1, top, width, 2);
    let stage = width / interval;
    for (let i = 0; i <= interval; i++) {
        ctx.rect(i * stage, top, 2, 5);
    }
    ctx.stroke();
}

The idea here is that we display a flat, horizontal line across the top of the screen, with markers. There is no line method on the canvas context, so a flat rectangle is the best we can do.

The more complex method is the drawColumn method (technically, it isn’t actually drawing columnns – but they still feel like columns – at least, more than rows):

const drawColumn = (top, target, speed, label) => {
    let ctx = getContext();
    let x = 10;    
    let directionHorizontal = speed;

    let intervalHandle = setInterval(() => {
        const buffer = 15;
        const height = 30;
        let showText = false;
        ctx.beginPath();
        ctx.clearRect(1, top - 1, target + buffer + Math.abs(directionHorizontal) + 2, height + 2);

        if (directionHorizontal === 1 && x <= target + buffer) {
            
        } else if (directionHorizontal > 0 && x >= target + buffer) {
            directionHorizontal = -1;
        } else if (directionHorizontal < 0 && x <= target) {
            clearInterval(intervalHandle);
            directionHorizontal = 0;
            showText = true;            
        }

        x += directionHorizontal;
        ctx.rect(1, top, x, height);
        ctx.stroke();

        if (showText) {                        
            ctx.fillText(label, 10, top + 20);
        }
    }, 1);
    
}

There is quite a lot to this; let’s focus on the interval; we take a handle to the interval, so that we can cancel it when we’ve finished drawing.

The first thing we do with the context is call beginPath – this allows us to group a series of updates into a single screen update; then we’re clearing an area, just wide of, the size of the rectangle.

We then have a conditional check – if we’re heading right, and have yet to reach the target (which has a small buffer appended for the purpose of animation), then this drops through to the code below; if we’ve reached that then we change direction; and when we’re back to the actual target, we cancel the update and set a flag to make the text appear.

Subsequently, we draw the (horizontal) column and, if we’ve finished, display the text.

Output and Caveats

It’s worth bearing in mind that the way this code is structured is probably not the best for performance – ideally, you would have a sort of game loop and draw, and then update the entire screen in one go.

Code

You can find the code for this here.