Monthly Archives: July 2021

An Introduction to Playwright

Pretty much the only end-to-end automation tool that I’ve used to date has been Selenium. I heard about Playwright in passing, and thought I’d have a play. This may end up being the start of a series of posts, because I’m very impressed so far.

Installation

You’ll need node.js installed, and then to install Playwright, it’s just:

npm -i -D playwright     

Running

You can actually have Playwright create the test (or at least some of it) for you.

 npx playwright codegen 

Launches a browser and an inspector window. Once you’ve create the test, you’ll end up with a script like this:

const { test, expect } = require('@playwright/test');
test('test', async ({ page }) => {
  // Go to https://www.pmichaels.net/
  await page.goto('https://www.pmichaels.net/');
  // Go to https://www.pmichaels.net/about/
  await page.goto('https://www.pmichaels.net/about/');
  // Click a:has-text("About")
  await page.click('a:has-text("About")');
  expect(page.url()).toBe('https://www.pmichaels.net/about/');

});

You can then run the test using the following command:

npx playwright test

There are some very interesting concepts at play here: playwright will automatically wait for elements to be available, it’s also clever enough to work out if there’s some kind of CSS at work, and will wait for that to complete, too.

Caveats

This one caught me out: it appears that Playwright insists on the tests being named *.spec.js, if they are not, you’ll get:

No tests found

The other thing to note about Playwright is that it’s headless by default – this can be overridden, but it fits much better into the testing idea.

Configure Cloudflare to work with your Azure Domain Registration

In this previous post I showed how you can set-up a custom domain in Azure and link that to your app service. Here, I’ll cover how you can add Cloudflare as a reverse proxy to your Azure site.

Start with Cloudflare

For the purpose of this post, I’ll assume that you have a Cloudflare account. I believe that everything in this post can be accomplished on the free tier. The first step is to add a new site to Cloudflare:

It’ll ask you for the URL of your site (you must own the root domain of your site). When you add your new site, you’ll be presented with this:

In order to make the changes that Cloudflare suggests, you’ll need to jump to Azure.

Over to Azure

In the Azure Portal, navigate to your App Service Domain (see the previous post referenced above for how to create this):

In the domain registration, you’ll need to select Manage DNS records:

Here, you can see the DNS entries for the domain:

As we saw from the Cloudflare recommendation, we’ll need to change these; although annoyingly, you can’t change them in this blade.

Advanced Management Portal

To change the nameservers, in the App Service Domain, go to Advanced Management Portal:

In here, select your domain:

Select Manage DNS:

When you update this, it may take up to a day to change (although it can take a few minutes).

References

https://docs.microsoft.com/en-us/azure/dns/dns-operations-recordsets-portal

Create and link a custom domain to an Azure App Service

I’ve recently been playing around with Cloudflare workers. As part of this, I wanted to experiment using them against an Azure App Service, but to do this, you need a custom domain. In this post, I’ll cover how to create a new domain, and then how to register that against an app service and add a TLS certificate, all without leaving the Azure Portal.

Register a New Domain

To register a new domain, you need to select the App Service Domains resource:

In the App Service Domains, you can create a new domain:

Once you’ve created the domain, you can register it against the App Service.

Register the Domain Against an Azure App Service

This assumes that you have an App Service to register against a domain. The example that I’m using here is just a new templated MVC app that was deployed directly to Azure.

Go into the Azure App Service and select Custom domains:

Inside the Custom domains blade, select Add custom domain. As shown in the diagram above, you’ll be asked to enter the domain, and to validate that domain. You’ll then select Add custom domain.

That’s actually all there is to it; you’ve now registered the domain against the app service. However, if you try to navigate to the domain, you’ll see that it doesn’t have a valid certificate – looking in the Custom domains blade, we can see why:

Let’s now add a certificate. This used to be a process that involved a lot of faffing about, and it had been greatly simplified.

To Add a TLS Certificate

Select the TLS / SSL settings blade:

Then select Create AppService Managed Certificate, as shown in the diagram above. This should present you with something similar to the following:

Click Create and this will create the certificate for you. This might be a good point to go and get a brew, as it takes a fair while.

When it’s finally finished, you’ll see something like this:

The last step is to register the certificate with the domain.

Register the Certificate with the Domain

We’re on the home straight now. Go back to Custom Domains:

As you can see, the domain is showing as Not Secure. Select Add Binding (as shown above), and the following dialog (or something similar) should appear:

If you now select Add Binding that should register the certificate against your domain.

References

https://www.youtube.com/watch?v=bXP6IvNYISw

Mutation Testing

Some time ago, I heard Dan Clarke from the Unhandled Exception podcast mention Mutation testing – the latest episode on this can be found here. I thought this definitely warranted some investigation.

If you skip to the bottom of this post, you’ll see some links to the official docs for Stryker, and to a video that details exactly how to use it.

What is Mutation Testing

The hypothesis here is that, if you’ve written a test, you can test that test by changing an element of the code under test – if the change breaks your test then your test is valid, if it does not, then your test is not.

I’m not completely sure I accept this theory, but I can see its uses. In this post, I’m experimenting with a Calculator class.

Installation

For the purpose of this, I’ll assume that you have some code to test. If you don’t, then you can download the code that I used my my tests here.

You’ll need a terminal window – you can use Windows Terminal, or any other terminal window of your choice; I’ve recently started using the Developer Power Shell (it’s kind of the Visual Studio equivalent of the VS Code Terminal):

The first thing you’ll need to do (unless you’re using other .Net Tools in your project) is to install the manifest:

dotnet new tool-manifest

To install Stryker, use the following command:

dotnet tool install dotnet-stryker

Tests and Usage

The tool cannot work without tests – remember that the purpose of it is to tell you if the tests are useful, not if the tests are there (although you do get some coverage stats from it). Here’s my code:

   public static class Calculator
    {
        public static decimal Add(decimal x, decimal y) =>
            x + y;

        public static decimal Subtract(decimal x, decimal y) =>
            x - y;

    }

And here’s the tests that I have:

        [Fact]
        public void Calculator_Add_ReturnsCorrect()
        {
            // Arrange            

            // Act
            decimal result = CalculatorApp.Calculator.Add(3, 6);

            // Assert
            Assert.Equal(9, result);
        }

As you can see, we’re looking at, at most, 50% test coverage. Let’s run the mutation tool and see what happens:

If you open the URL, you’ll get a coverage report, including any mutants that survived (we’ll come back to the later):

What this is telling us is that we don’t have particularly good test coverage, but what we do have has not survived mutation.

Let’s fill out the test coverage to 100%:

        [Fact]
        public void Calculator_Subtract_ReturnsCorrect()
        {
            // Arrange            

            // Act
            decimal result = CalculatorApp.Calculator.Subtract(1, 0);

            // Assert
            Assert.Equal(1, result);

        }

Admittedly, this took some gaming of the system, but when you run this, it survives:

Why did that test survive, by the first one didn’t? And what does ‘survived’ mean? Well, you can actually get it to tell you what it does during the mutation by selecting the file in question, and clicking “Expand All”:

What this tells you is that it replaced the code in the Subtract method with 1 (i.e. just return 1), and with x + y, (rather than x – y). The mutation would be ‘killed’ if, upon this change, at least one test failed. All I had to do was to find a test that would survive both scenarios (hence 1 – 0.

Summary

Stryker looks like a really cool and useful tool, but it definitely has its limitations. It identifies test coverage, and any test coverage that isn’t definitive (where there is no assert statement, or where the assert statement is ambiguous; for example, asserting that an exception is not thrown).

I’ve still to run it on a reasonable sized code-base; which I fully intend to do, but I’m not sure that I’d necessarily build this into a CI/CD pipeline (unless you genuinely fear that your developers are gaming the code-coverage stats).

I also have reservations as to whether a code base with 100% test coverage, and 0 surviving mutants is a healthy one, or one in a straight-jacket. Having said that, I definitely think this is a useful tool – it gives you information about your code base that you didn’t have before.

References

https://www.youtube.com/watch?v=DiIFM4Iluzw

https://stryker-mutator.io/docs/stryker-net/Introduction/

Conditionally Creating Resources in Terraform

I’ve recently been learning and blogging about Terraform (the latest of which you can find here).

In this post, I’m going to cover the conditional creation of a resource, using the count variable.

Disclaimer

As with most of the stuff that finds its way into my blog, this is from finer minds than my own. It’s also worth noting, as with all the stuff that finds its way here, the main audience is future me.

Count

All of the resources in Terraform have an inbuild property of count – what that means is that you can do something like this:

resource "azurerm_resource_group" "rg" {
  count = 2
  name     = "myTFResourceGroup${count.index}"
  location = "westus2"
}

This will create two resource groups, called myTFResourceGroup0 and myTFResourceGroup1. Admittedly, this has limited uses; however, it becomes much more useful when you use something like this:

resource "azurerm_resource_group" "rg" {
  count                      = var.myvar == "some_setting" ? 1 : 0 

So now, using the variables associated with the configuration that you run, you can either create the resource, or not (a count of 0 meaning that it will not be created).

Error

So that’s all well and good, but let’s imagine that you’re using this resource; for example:

resource "azurerm_app_service_plan" "app-service-plan" {
  name                = "pcm-app-service-plan"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku {
    tier = "Standard"
    size = "S1"
  }
}

(See this post for an explanation of this)

If you’re using the resource group, you’ll start to get an error:

Because azurerm_resource_group.rg has “count” set, its attributes must be
accessed on specific instances.

What this is basically telling you is that, because you’ve specified a count, you now have to tell it which version of the resource you’re referring to. In our case, we know that if the resource group exists at all, there will only be one; so we can use this:

resource "azurerm_app_service_plan" "app-service-plan" {
  name                = "pcm-app-service-plan"
  location            = azurerm_resource_group.rg[0].location
  resource_group_name = azurerm_resource_group.rg[0].name
  sku {
    tier = "Standard"
    size = "S1"
  }
}

However, there’s still a problem here. Let’s imagine that our myvar setting is not set to “some_setting” – well in that case, the resource group will not create; however, the app service plan will, because no such check exists. The upshot of this is that you’ll need to ensure that anything that uses a resource that has a count, must itself, have a count (and on the same logic).