Category Archives: Azure

Introduction to Azure Chaos Studio

Some time ago, I investigated the concept of chaos engineering. The principle behind Chaos Engineering is a very simply one: since your software is likely to encounter hostile conditions in the wild, why not introduce those conditions while (and when) you can control them, and then deal with the fallout then, instead of at 3am on a Sunday.

At the time, I was trying to deal with an on-site issue where the connection seemed to be randomly dropping. In the end, I solved this by writing something similar to Polly – albeit a much simpler version.

Microsoft have recently released a preview of something called Chaos Studio. It’s very much in its infancy now, but what is there looks very interesting.

The product is essentially divided into two sections: targets and experiments. Targets represent the thing that you intend to wrought chaos upon, and experiments are how that chaos will be wrought.


For this test, I’m going to use a VM. That’s mainly because what you can do with this product is currently limited to VMs, AKS, and Redis.

Create a VM and Check Availability

The first step is to create a VM. To be honest, it doesn’t matter what the VM is, because all we’ll be doing is switching it off. Start by checking the availability – you should be able to do that in Logs – and you should notice 100% availability, unless something has gone catastrophically wrong with your deployment.


The next step is to configure our target. In chaos studio, select Targets and pick the new VM:

Not that you’ve enabled the targets, you’ll need to grant permission to the chaos studio for the VMs. Inside the VM blade, select Access Control:

If you don’t grant this access, you’ll get a permissions error when you run the experiment. The next step is to create the experiment. In Chaos Studio, select Experiments and then Create:

This will bring up a screen similar to the following:

Let’s discuss a little the concepts here: we have step, branch, and fault. A step is a sequential action that you will execute, whilst a branch is a parallel action; that is, actions in different branches can happen at the same time. A fault is what you actually do – so the fault is the chaos! Let’s add a fault:

This asks me two things, what do I want the fault to happen on (you can only select targets that have previously been created) and what do I want the fault to be. In my case, I’ve created a two step process that turns the machine off, waits a minute, then turns it off again:

Now that the experiment is created, you can start it. You get a warning at this point that basically says “it’s your foot, and you’re currently pointing a high powered rifle at it!”:

If you now run this, and it’s worth bearing in mind that there’s no simulation here – if you do this on production infrastructure it will shut it down for you, then you’ll see the update of it running:

You can drill down into the details to see exactly what it’s doing, what stage, etc.:

The experiment kills the machine for 1 minute, then waits for a minute, then kills it again. If you have a look at the availability graph, you should be able to see that:


So far, I’m pretty impressed with this tool. When they’ve finished (and by that, I mean, they’ve given the ability to create your own chaos, and have expanded the targets to cover the entire Azure ecosystem), it’s going to be a really interesting testing tool.


Azure Friday Introduction to Chaos Studio

Configure Cloudflare to work with your Azure Domain Registration

In this previous post I showed how you can set-up a custom domain in Azure and link that to your app service. Here, I’ll cover how you can add Cloudflare as a reverse proxy to your Azure site.

Start with Cloudflare

For the purpose of this post, I’ll assume that you have a Cloudflare account. I believe that everything in this post can be accomplished on the free tier. The first step is to add a new site to Cloudflare:

It’ll ask you for the URL of your site (you must own the root domain of your site). When you add your new site, you’ll be presented with this:

In order to make the changes that Cloudflare suggests, you’ll need to jump to Azure.

Over to Azure

In the Azure Portal, navigate to your App Service Domain (see the previous post referenced above for how to create this):

In the domain registration, you’ll need to select Manage DNS records:

Here, you can see the DNS entries for the domain:

As we saw from the Cloudflare recommendation, we’ll need to change these; although annoyingly, you can’t change them in this blade.

Advanced Management Portal

To change the nameservers, in the App Service Domain, go to Advanced Management Portal:

In here, select your domain:

Select Manage DNS:

When you update this, it may take up to a day to change (although it can take a few minutes).


Create and link a custom domain to an Azure App Service

I’ve recently been playing around with Cloudflare workers. As part of this, I wanted to experiment using them against an Azure App Service, but to do this, you need a custom domain. In this post, I’ll cover how to create a new domain, and then how to register that against an app service and add a TLS certificate, all without leaving the Azure Portal.

Register a New Domain

To register a new domain, you need to select the App Service Domains resource:

In the App Service Domains, you can create a new domain:

Once you’ve created the domain, you can register it against the App Service.

Register the Domain Against an Azure App Service

This assumes that you have an App Service to register against a domain. The example that I’m using here is just a new templated MVC app that was deployed directly to Azure.

Go into the Azure App Service and select Custom domains:

Inside the Custom domains blade, select Add custom domain. As shown in the diagram above, you’ll be asked to enter the domain, and to validate that domain. You’ll then select Add custom domain.

That’s actually all there is to it; you’ve now registered the domain against the app service. However, if you try to navigate to the domain, you’ll see that it doesn’t have a valid certificate – looking in the Custom domains blade, we can see why:

Let’s now add a certificate. This used to be a process that involved a lot of faffing about, and it had been greatly simplified.

To Add a TLS Certificate

Select the TLS / SSL settings blade:

Then select Create AppService Managed Certificate, as shown in the diagram above. This should present you with something similar to the following:

Click Create and this will create the certificate for you. This might be a good point to go and get a brew, as it takes a fair while.

When it’s finally finished, you’ll see something like this:

The last step is to register the certificate with the domain.

Register the Certificate with the Domain

We’re on the home straight now. Go back to Custom Domains:

As you can see, the domain is showing as Not Secure. Select Add Binding (as shown above), and the following dialog (or something similar) should appear:

If you now select Add Binding that should register the certificate against your domain.


Receiving a Message Using Azure.Messaging.ServiceBus

Azure.Messaging.ServiceBus is the latest SDK library that allows you to interface with Azure Service Bus.

In this post I wrote about receiving a message in Azure Service Bus using the Microsoft.Azure.ServiceBus library. Here, I’ll cover the method of receiving a message using Azure.Messaging.ServiceBus.

The first step is to create a ServiceBusClient instance:

_serviceBusClient = new ServiceBusClient(connectionString);

Once you’ve created this, the subsequent classes are created from there. This library draws a distinction between a message receiver and a message processor – the latter being event driven.

Receiving a Message

To receive a message:

            var messageReceiver = _serviceBusClient.CreateReceiver(QUEUE_NAME);            
            var message = await messageReceiver.ReceiveMessageAsync();

            //string messageBody = Encoding.UTF8.GetString(message.Body);
            string messageBody = message.Body.ToString();

It’s worth noting here that it is no longer necessary to decode the message body explicitly.

Processing a Message

This is the new version of registering a handler for the event, and it has a few additional features. Let’s see the code:

            var processor = _serviceBusClient.CreateProcessor(QUEUE_NAME);
            processor.ProcessMessageAsync += handleMessage;
            processor.ProcessErrorAsync += ExceptionHandler;

            await processor.StartProcessingAsync();                        

            await Task.Delay(2000);
            await processor.StopProcessingAsync();

We won’t worry too much about the events themselves for now, but the important events are StartProcessingAsync and StopProcessingAsync. Note that here we have a 2 second delay – this means that we will receive messages for two seconds, and then stop; obviously the start and stop don’t need to be in the same method.


Azure Service Bus SDK Libraries

I’ve written pretty extensively on the Microsoft.Azure.ServiceBus SDK. In this post, I’m just covering the fact that this library is on its way to deprecation (don’t panic, its predecessor has been hanging around since 2011!)

Let’s see what these libraries are and some links.


This library does look like it’s on its way to being deprecated. It supports .Net Framework only.

The NuGet package is here, but it’s closed source:


This library was introduced to support .Net Core.

The NuGet package is here:

The code for this is open source:


If you read Sean Feldman’s article here (which this was heavily based on), you’ll see that this seems to be due to some restructuring of teams. The code has changed, and MS say it’s more consistent (although what with, I’m unsure).

The NuGet Package is here:

The source code for this is here:


Terraform – Autoscale an App Service

I’ve recently been writing about Terraform – mainly because I’m learning it from scratch, and playing about with tech and then writing about it is basically my way of learning.

In this post, I’m going to build on this previous post on creating an App Service, by adding a Scale Out feature to it.

This is the App Service that we created in the referenced post:

In the image, you’ll see Scale Out. Note that it says (App Service Plan): in fact, this is just a link to the App Service Plan Scale Out. We can access it from here – let’s see what that looks like:

As we can see, there’s a single instance of the App Service, and it’s managed manually. What we’re going to do is change that so that the App Service is auto-scaled.

The Terraform script here is broadly taken from the example here. However, that applies to a VM Scale Set, whereas we’re applying it to an App Service Plan.

resource "azurerm_monitor_autoscale_setting" "example" {
  name                = "myAutoscaleSetting"
  resource_group_name =
  location            = azurerm_resource_group.rg.location
  target_resource_id  =
  profile {
    name = "default"
    capacity {
      default = 1
      minimum = 1
      maximum = 10
    rule {
      metric_trigger {
        metric_name        = "CpuPercentage"
        metric_resource_id =
        time_grain         = "PT1M"
        statistic          = "Average"
        time_window        = "PT5M"
        time_aggregation   = "Average"
        operator           = "GreaterThan"
        threshold          = 90
      scale_action {
        direction = "Increase"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT1M"
    rule {
      metric_trigger {
        metric_name        = "CpuPercentage"
        metric_resource_id =
        time_grain         = "PT1M"
        statistic          = "Average"
        time_window        = "PT5M"
        time_aggregation   = "Average"
        operator           = "LessThan"
        threshold          = 10
      scale_action {
        direction = "Decrease"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT1M"

Some key points:

– The example uses “Percentage CPU”, whereas for an App Service, this gets switched to CpuPercentage.
– The resource IDs that are referred to are that of the App Service Plan.

Finally, if we apply that, we can see the autoscale:


Terraform – Provisioning an Azure App Service

In my previous post on Getting started with Terraform I covered a very quick, and vague explanation of what Terraform is, and what it does. In this post, I’m going to cover the explanation of what the various syntax looks like; I’m also going to provision some infrastructure in the form of an App Service.

Before we get into the what we’ll need to create an app service, let’s first analyse the config that we used in the previous post:

# Configure the Azure provider
terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.26"

provider "azurerm" {
  features {}

resource "azurerm_resource_group" "rg" {
  name     = "myTFResourceGroup"
  location = "ukwest"

Let’s breakdown exactly what we’re seeing here for the resource:

Now that we’ve broken that down, it makes sense that, if we want to deploy an App Service, that we simply need to know what the correct type of the app service is. There’s probably a list of these somewhere.

Let’s have a look at the config for the App Service:

# App Service
resource "azurerm_app_service_plan" "app-service-plan" {
  name                = "pcm-app-service-plan"
  location            = azurerm_resource_group.rg.location
  resource_group_name =
  sku {
    tier = "Standard"
    size = "S1"
resource "azurerm_app_service" "app-service" {
  name                = "pcm-app-service"
  location            = azurerm_resource_group.rg.location
  resource_group_name =
  app_service_plan_id =

Again, let’s break this down – starting with the plan:

Finally, let’s have a look at the app service itself – there’s not too much difference here:

If we now run

terraform.exe plan

Then we’ll see that it intends to create an app service plan and app service within that plan; running:

terraform.exe apply

Will execute that and generate our new resources.

Read the Dead Letter Queue

I’ve been writing about and speaking about Azure Service Bus a lot recently.

In this post, I’m going to focus on the Dead Letter Queue in more detail.

What is the Dead Letter Queue, and what has it ever done for me?

To describe what the dead letter queue does, I invite you to think about an assembly line for a car. The car in question has just come through to have a bonnet fitted (hood for any American readers). However, the guy that’s fitting the bonnet can’t get it to sit right in the hinges; he tries and tries, but it won’t fit. After a while, he goes to get his superviser, and they both try. They draft the workers in from all over the plant, but can’t get the bonnet fitted.

Meanwhile, the entire assembly line has stopped. The person that fits the steering wheel is behind the bonnet fitter, and there’s no space for him to move the car that he’s just fitted the wheel to; the dashboard fitter can’t pass onto the steering wheel, and so on.

(I have no knowledge of what a car assembly line looks like, outside of the film Christine, so apologies if this is incorrect).

A message that can’t be processed is often called a poison message, and it causes exactly this problem. The Service Bus can’t deliver any messages until this message has gone, and this message can’t go, because there’s something wrong with it. The solution is to have a dedicated queue that holds these messages: it’s called a Dead Letter Queue – it’s kind of like a holding bay for the car.

Why would a message be “poison”

There are a few reasons that a message can be considered “poison” and dead lettered; some of the most common are:

– Each queue has a maximum delivery count, if it’s exceeded – that is, we’ve tried too many times to process it
– The message can be explicitly marked as bad by the client
– The size of the message is bigger than the allocated maximum size
– The message has been “auto-forwarded” too many times

Essentially, the system tries to work out whether this message is staying around too long and causing issues with the system. It’s important to know, though, that the dead letter queue is just another queue. The message isn’t lost – just side-lined.

Dead Lettering

Let’s see how we can force a message into a dead letter queue. The easiest way to do this is to explicitly just Dead Letter the message; for example:

            var messageReceiver = new MessageReceiver(connectionString, QUEUE_NAME);
            var message = await messageReceiver.ReceiveAsync();

            await messageReceiver.DeadLetterAsync(message.SystemProperties.LockToken, "Really bad message");

Here, we’ve read the message, and then told Service Bus to just Dead Letter it. In real life, you may choose to do this on rare occasions, but I imagine its main use is for testing.

Abandon the Message

Another way to cause a message to be dead lettered is to exceed the Max Delivery Count. You can do this by “abandoning” the message multiple times; for example:

var messageReceiver = new MessageReceiver(connectionString, QUEUE_NAME);
var message = await messageReceiver.ReceiveAsync();

string messageBody = Encoding.UTF8.GetString(message.Body);

Console.WriteLine($"Message {message.MessageId} ({messageBody}) had a delivery count of {message.SystemProperties.DeliveryCount}");
await messageReceiver.AbandonAsync(message.SystemProperties.LockToken);

Here, we’re reading the message, and rather than completing it, we’re abandoning it. It’s worth bearing in mind that this is what happens when you abandon a message. It’s also what happens when you read a message and just implicitly abandon it (i.e., you read it on a PeekLock and then do nothing): the AbandonAsync method doesn’t actually change the functionality of the code above – it does change the speed, though.

Reading The Dead Letter Queue

Now that we’ve dead-lettered a message, we can read the Dead Letter Queue.

            var deadletterPath = EntityNameHelper.FormatDeadLetterPath(QUEUE_NAME);
            var deadLetterReceiver = new MessageReceiver(connectionString, deadletterPath, ReceiveMode.PeekLock);
            var message = await deadLetterReceiver.ReceiveAsync();

            string messageBody = Encoding.UTF8.GetString(message.Body);

            Console.WriteLine("Message received: {0}", messageBody);
            if (message.UserProperties.ContainsKey("DeadLetterReason"))
                Console.WriteLine("Reason: {0} ", message.UserProperties["DeadLetterReason"]);
            if (message.UserProperties.ContainsKey("DeadLetterErrorDescription"))
                Console.WriteLine("Description: {0} ", message.UserProperties["DeadLetterErrorDescription"]);

The code above sets up a MessageReceiver for the dead letter queue. The delivery count inside the dead letter queue does not increase, but it does retain the number that it had from the original queue. Effectively, all you can do with a Dead Letter message is to complete it.


When a message is dead lettered, the properties DeadLetterReason and DeadLetterErrorDescription may get added to the message. If you forcibly dead letter the message then you have the option to add this: if you choose not to then it will not be present (hence the checks around the properties), but mostly, these will be available.

Re-submitting a Message and Transactions

We’ve now seen how to cause a message to Dead Letter, and read the Dead Letter queue; next we’re going to investigate re-submitting the message.

As a quick side not – you can’t really re-submit a message – as you’ll see, what we actually do is to complete the dead letter message, and send a copy back to the queue.

            var serviceBusConnection = new ServiceBusConnection(connectionString);

            var deadletterPath = EntityNameHelper.FormatDeadLetterPath(QUEUE_NAME);
            var deadLetterReceiver = new MessageReceiver(serviceBusConnection, deadletterPath, ReceiveMode.PeekLock);
            var queueClient = new QueueClient(serviceBusConnection, QUEUE_NAME, ReceiveMode.PeekLock, RetryPolicy.Default);

            var deadLetterMessage = await deadLetterReceiver.ReceiveAsync();

            using var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);

            var resubmitMessage = deadLetterMessage.Clone();

            await queueClient.SendAsync(resubmitMessage);
            await deadLetterReceiver.CompleteAsync(deadLetterMessage.SystemProperties.LockToken);            


There’s a few points to note in the above code:


FormatDeadLetterPath gives you the entity path for the dead letter queue, based on an entity.

Transaction Scope

The scope ensures that everything between its creation and completion happens as a single transaction. That is, if part of that fails, the whole thing fails. For example, you could add a throw new exception between the send and the complete, and the new message will not send.

We’re using the new C# 8 using statement – that is, it will apply to everything between it, and the end of the method.


There are several overloads for most of these methods, and typically, you can pass a connection string into the constructor – for example, MessageReceiver could be called like this:

new MessageReceiver(connectionString, QUEUE_NAME);

Typically, you can use this and it works exactly the same as if you established your own connection and passed that through; however, with a transaction, everything needs to share a connection. If they do not, then you may see an error such as this:

Transaction hasn’t been declared yet, or has already been discharged

Hence we’re creating the connection upfront.


Add Storage Queue Message

I’ve written quite extensively in the past about Azure, and Azure Storage. I recently needed to add a message to an Azure storage queue, and realised that I had never written a post about that, specifically. As with many Azure focused .Net activities, it’s not too complex; but I do like to have my own notes on things.

If you’ve arrived at this post, you may find it’s very similar to the Microsoft documentation.

How to add a message

The first step is to install a couple of NuGet packages:

Install-Package Microsoft.Azure.Storage.Common
Install-Package Microsoft.Azure.Storage.Queue

My preference for these kinds of things is to create a helper: largely so that I can mock it out for testing; however, even if you fundamentally object to the concept of testing, you may find such a class helpful, as it keeps all your code in one place.

 public class StorageQueueHelper
        private readonly string _connectionString;
        private readonly string _queueName;

        public StorageQueueHelper(string connectionString, string queueName)
            _connectionString = connectionString;
            _queueName = queueName;

        public async Task AddNewMessage(string messageBody)
            var queue = await GetQueue();

            CloudQueueMessage message = new CloudQueueMessage(messageBody);
            await queue.AddMessageAsync(message);

        private async Task<CloudQueue> GetQueue()
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);
            CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
            CloudQueue queue = queueClient.GetQueueReference(_queueName);
            await queue.CreateIfNotExistsAsync();

            return queue;

The class above works for a single queue, and storage account. Depending on your use case, this might not be appropriate.

The GetQueue() method here is a bit naughty, as it actually changes something (or potentially changes something). Essentially, all it’s doing is connecting to a cloud storage account, and then getting a reference to a queue. We know that the queue will exist, because we’re forcing it to (CreateIfNotExistsAsync()).

Back in AddNewMessage(), once we have the queue, it’s trivial to simply create the message and add it.