Category Archives: Azure

Configure Cloudflare to work with your Azure Domain Registration

In this previous post I showed how you can set-up a custom domain in Azure and link that to your app service. Here, I’ll cover how you can add Cloudflare as a reverse proxy to your Azure site.

Start with Cloudflare

For the purpose of this post, I’ll assume that you have a Cloudflare account. I believe that everything in this post can be accomplished on the free tier. The first step is to add a new site to Cloudflare:

It’ll ask you for the URL of your site (you must own the root domain of your site). When you add your new site, you’ll be presented with this:

In order to make the changes that Cloudflare suggests, you’ll need to jump to Azure.

Over to Azure

In the Azure Portal, navigate to your App Service Domain (see the previous post referenced above for how to create this):

In the domain registration, you’ll need to select Manage DNS records:

Here, you can see the DNS entries for the domain:

As we saw from the Cloudflare recommendation, we’ll need to change these; although annoyingly, you can’t change them in this blade.

Advanced Management Portal

To change the nameservers, in the App Service Domain, go to Advanced Management Portal:

In here, select your domain:

Select Manage DNS:

When you update this, it may take up to a day to change (although it can take a few minutes).

References

https://docs.microsoft.com/en-us/azure/dns/dns-operations-recordsets-portal

Create and link a custom domain to an Azure App Service

I’ve recently been playing around with Cloudflare workers. As part of this, I wanted to experiment using them against an Azure App Service, but to do this, you need a custom domain. In this post, I’ll cover how to create a new domain, and then how to register that against an app service and add a TLS certificate, all without leaving the Azure Portal.

Register a New Domain

To register a new domain, you need to select the App Service Domains resource:

In the App Service Domains, you can create a new domain:

Once you’ve created the domain, you can register it against the App Service.

Register the Domain Against an Azure App Service

This assumes that you have an App Service to register against a domain. The example that I’m using here is just a new templated MVC app that was deployed directly to Azure.

Go into the Azure App Service and select Custom domains:

Inside the Custom domains blade, select Add custom domain. As shown in the diagram above, you’ll be asked to enter the domain, and to validate that domain. You’ll then select Add custom domain.

That’s actually all there is to it; you’ve now registered the domain against the app service. However, if you try to navigate to the domain, you’ll see that it doesn’t have a valid certificate – looking in the Custom domains blade, we can see why:

Let’s now add a certificate. This used to be a process that involved a lot of faffing about, and it had been greatly simplified.

To Add a TLS Certificate

Select the TLS / SSL settings blade:

Then select Create AppService Managed Certificate, as shown in the diagram above. This should present you with something similar to the following:

Click Create and this will create the certificate for you. This might be a good point to go and get a brew, as it takes a fair while.

When it’s finally finished, you’ll see something like this:

The last step is to register the certificate with the domain.

Register the Certificate with the Domain

We’re on the home straight now. Go back to Custom Domains:

As you can see, the domain is showing as Not Secure. Select Add Binding (as shown above), and the following dialog (or something similar) should appear:

If you now select Add Binding that should register the certificate against your domain.

References

https://www.youtube.com/watch?v=bXP6IvNYISw

Receiving a Message Using Azure.Messaging.ServiceBus

Azure.Messaging.ServiceBus is the latest SDK library that allows you to interface with Azure Service Bus.

In this post I wrote about receiving a message in Azure Service Bus using the Microsoft.Azure.ServiceBus library. Here, I’ll cover the method of receiving a message using Azure.Messaging.ServiceBus.

The first step is to create a ServiceBusClient instance:

_serviceBusClient = new ServiceBusClient(connectionString);

Once you’ve created this, the subsequent classes are created from there. This library draws a distinction between a message receiver and a message processor – the latter being event driven.

Receiving a Message

To receive a message:

            var messageReceiver = _serviceBusClient.CreateReceiver(QUEUE_NAME);            
            var message = await messageReceiver.ReceiveMessageAsync();

            //string messageBody = Encoding.UTF8.GetString(message.Body);
            string messageBody = message.Body.ToString();

It’s worth noting here that it is no longer necessary to decode the message body explicitly.

Processing a Message

This is the new version of registering a handler for the event, and it has a few additional features. Let’s see the code:

            var processor = _serviceBusClient.CreateProcessor(QUEUE_NAME);
            processor.ProcessMessageAsync += handleMessage;
            processor.ProcessErrorAsync += ExceptionHandler;

            await processor.StartProcessingAsync();                        

            await Task.Delay(2000);
            await processor.StopProcessingAsync();

We won’t worry too much about the events themselves for now, but the important events are StartProcessingAsync and StopProcessingAsync. Note that here we have a 2 second delay – this means that we will receive messages for two seconds, and then stop; obviously the start and stop don’t need to be in the same method.

References

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues

Azure Service Bus SDK Libraries

I’ve written pretty extensively on the Microsoft.Azure.ServiceBus SDK. In this post, I’m just covering the fact that this library is on its way to deprecation (don’t panic, its predecessor has been hanging around since 2011!)

Let’s see what these libraries are and some links.

WindowsAzure.ServiceBus

This library does look like it’s on its way to being deprecated. It supports .Net Framework only.

The NuGet package is here, but it’s closed source:

https://www.nuget.org/packages/WindowsAzure.ServiceBus

Microsoft.Azure.ServiceBus

This library was introduced to support .Net Core.

The NuGet package is here:

https://www.nuget.org/packages/Microsoft.Azure.ServiceBus

The code for this is open source:

https://github.com/Azure/azure-service-bus-dotnet

Azure.Messaging.ServiceBus

If you read Sean Feldman’s article here (which this was heavily based on), you’ll see that this seems to be due to some restructuring of teams. The code has changed, and MS say it’s more consistent (although what with, I’m unsure).

The NuGet Package is here:

https://www.nuget.org/packages/Azure.Messaging.ServiceBus

The source code for this is here:

https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/servicebus

References

https://markheath.net/post/migrating-to-new-servicebus-sdk

https://weblogs.asp.net/sfeldman/the-future-of-asb-dotnet-sdk

Terraform – Autoscale an App Service

I’ve recently been writing about Terraform – mainly because I’m learning it from scratch, and playing about with tech and then writing about it is basically my way of learning.

In this post, I’m going to build on this previous post on creating an App Service, by adding a Scale Out feature to it.

This is the App Service that we created in the referenced post:

In the image, you’ll see Scale Out. Note that it says (App Service Plan): in fact, this is just a link to the App Service Plan Scale Out. We can access it from here – let’s see what that looks like:

As we can see, there’s a single instance of the App Service, and it’s managed manually. What we’re going to do is change that so that the App Service is auto-scaled.

The Terraform script here is broadly taken from the example here. However, that applies to a VM Scale Set, whereas we’re applying it to an App Service Plan.

resource "azurerm_monitor_autoscale_setting" "example" {
  name                = "myAutoscaleSetting"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  target_resource_id  = azurerm_app_service_plan.app-service-plan.id
  profile {
    name = "default"
    capacity {
      default = 1
      minimum = 1
      maximum = 10
    }
    rule {
      metric_trigger {
        metric_name        = "CpuPercentage"
        metric_resource_id = azurerm_app_service_plan.app-service-plan.id
        time_grain         = "PT1M"
        statistic          = "Average"
        time_window        = "PT5M"
        time_aggregation   = "Average"
        operator           = "GreaterThan"
        threshold          = 90
      }
      scale_action {
        direction = "Increase"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT1M"
      }
    }
    rule {
      metric_trigger {
        metric_name        = "CpuPercentage"
        metric_resource_id = azurerm_app_service_plan.app-service-plan.id
        time_grain         = "PT1M"
        statistic          = "Average"
        time_window        = "PT5M"
        time_aggregation   = "Average"
        operator           = "LessThan"
        threshold          = 10
      }
      scale_action {
        direction = "Decrease"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT1M"
      }
    }
  }  
}

Some key points:

– The example uses “Percentage CPU”, whereas for an App Service, this gets switched to CpuPercentage.
– The resource IDs that are referred to are that of the App Service Plan.

Finally, if we apply that, we can see the autoscale:

References

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/monitor_autoscale_setting

https://stackoverflow.com/questions/58657096/error-creating-auto-scaling-rule-for-app-service-using-terraform-in-azure

Terraform – Provisioning an Azure App Service

In my previous post on Getting started with Terraform I covered a very quick, and vague explanation of what Terraform is, and what it does. In this post, I’m going to cover the explanation of what the various syntax looks like; I’m also going to provision some infrastructure in the form of an App Service.

Before we get into the what we’ll need to create an app service, let’s first analyse the config that we used in the previous post:

# Configure the Azure provider
terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.26"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "rg" {
  name     = "myTFResourceGroup"
  location = "ukwest"
}

Let’s breakdown exactly what we’re seeing here for the resource:

Now that we’ve broken that down, it makes sense that, if we want to deploy an App Service, that we simply need to know what the correct type of the app service is. There’s probably a list of these somewhere.

Let’s have a look at the config for the App Service:

# App Service
resource "azurerm_app_service_plan" "app-service-plan" {
  name                = "pcm-app-service-plan"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku {
    tier = "Standard"
    size = "S1"
  }
}
resource "azurerm_app_service" "app-service" {
  name                = "pcm-app-service"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.app-service-plan.id
}

Again, let’s break this down – starting with the plan:

Finally, let’s have a look at the app service itself – there’s not too much difference here:

If we now run

terraform.exe plan

Then we’ll see that it intends to create an app service plan and app service within that plan; running:

terraform.exe apply

Will execute that and generate our new resources.

Read the Dead Letter Queue

I’ve been writing about and speaking about Azure Service Bus a lot recently.

In this post, I’m going to focus on the Dead Letter Queue in more detail.

What is the Dead Letter Queue, and what has it ever done for me?

To describe what the dead letter queue does, I invite you to think about an assembly line for a car. The car in question has just come through to have a bonnet fitted (hood for any American readers). However, the guy that’s fitting the bonnet can’t get it to sit right in the hinges; he tries and tries, but it won’t fit. After a while, he goes to get his superviser, and they both try. They draft the workers in from all over the plant, but can’t get the bonnet fitted.

Meanwhile, the entire assembly line has stopped. The person that fits the steering wheel is behind the bonnet fitter, and there’s no space for him to move the car that he’s just fitted the wheel to; the dashboard fitter can’t pass onto the steering wheel, and so on.

(I have no knowledge of what a car assembly line looks like, outside of the film Christine, so apologies if this is incorrect).

A message that can’t be processed is often called a poison message, and it causes exactly this problem. The Service Bus can’t deliver any messages until this message has gone, and this message can’t go, because there’s something wrong with it. The solution is to have a dedicated queue that holds these messages: it’s called a Dead Letter Queue – it’s kind of like a holding bay for the car.

Why would a message be “poison”

There are a few reasons that a message can be considered “poison” and dead lettered; some of the most common are:

– Each queue has a maximum delivery count, if it’s exceeded – that is, we’ve tried too many times to process it
– The message can be explicitly marked as bad by the client
– The size of the message is bigger than the allocated maximum size
– The message has been “auto-forwarded” too many times

Essentially, the system tries to work out whether this message is staying around too long and causing issues with the system. It’s important to know, though, that the dead letter queue is just another queue. The message isn’t lost – just side-lined.

Dead Lettering

Let’s see how we can force a message into a dead letter queue. The easiest way to do this is to explicitly just Dead Letter the message; for example:

            var messageReceiver = new MessageReceiver(connectionString, QUEUE_NAME);
            var message = await messageReceiver.ReceiveAsync();

            await messageReceiver.DeadLetterAsync(message.SystemProperties.LockToken, "Really bad message");

Here, we’ve read the message, and then told Service Bus to just Dead Letter it. In real life, you may choose to do this on rare occasions, but I imagine its main use is for testing.

Abandon the Message

Another way to cause a message to be dead lettered is to exceed the Max Delivery Count. You can do this by “abandoning” the message multiple times; for example:

var messageReceiver = new MessageReceiver(connectionString, QUEUE_NAME);
var message = await messageReceiver.ReceiveAsync();

string messageBody = Encoding.UTF8.GetString(message.Body);

Console.WriteLine($"Message {message.MessageId} ({messageBody}) had a delivery count of {message.SystemProperties.DeliveryCount}");
await messageReceiver.AbandonAsync(message.SystemProperties.LockToken);

Here, we’re reading the message, and rather than completing it, we’re abandoning it. It’s worth bearing in mind that this is what happens when you abandon a message. It’s also what happens when you read a message and just implicitly abandon it (i.e., you read it on a PeekLock and then do nothing): the AbandonAsync method doesn’t actually change the functionality of the code above – it does change the speed, though.

Reading The Dead Letter Queue

Now that we’ve dead-lettered a message, we can read the Dead Letter Queue.

            var deadletterPath = EntityNameHelper.FormatDeadLetterPath(QUEUE_NAME);
            var deadLetterReceiver = new MessageReceiver(connectionString, deadletterPath, ReceiveMode.PeekLock);
            
            var message = await deadLetterReceiver.ReceiveAsync();

            string messageBody = Encoding.UTF8.GetString(message.Body);

            Console.WriteLine("Message received: {0}", messageBody);
            if (message.UserProperties.ContainsKey("DeadLetterReason"))
            {
                Console.WriteLine("Reason: {0} ", message.UserProperties["DeadLetterReason"]);
            }
            if (message.UserProperties.ContainsKey("DeadLetterErrorDescription"))
            {
                Console.WriteLine("Description: {0} ", message.UserProperties["DeadLetterErrorDescription"]);
            }

The code above sets up a MessageReceiver for the dead letter queue. The delivery count inside the dead letter queue does not increase, but it does retain the number that it had from the original queue. Effectively, all you can do with a Dead Letter message is to complete it.

DeadLetterReason

When a message is dead lettered, the properties DeadLetterReason and DeadLetterErrorDescription may get added to the message. If you forcibly dead letter the message then you have the option to add this: if you choose not to then it will not be present (hence the checks around the properties), but mostly, these will be available.

Re-submitting a Message and Transactions

We’ve now seen how to cause a message to Dead Letter, and read the Dead Letter queue; next we’re going to investigate re-submitting the message.

As a quick side not – you can’t really re-submit a message – as you’ll see, what we actually do is to complete the dead letter message, and send a copy back to the queue.

            var serviceBusConnection = new ServiceBusConnection(connectionString);

            var deadletterPath = EntityNameHelper.FormatDeadLetterPath(QUEUE_NAME);
            var deadLetterReceiver = new MessageReceiver(serviceBusConnection, deadletterPath, ReceiveMode.PeekLock);
            
            var queueClient = new QueueClient(serviceBusConnection, QUEUE_NAME, ReceiveMode.PeekLock, RetryPolicy.Default);

            var deadLetterMessage = await deadLetterReceiver.ReceiveAsync();

            using var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);

            var resubmitMessage = deadLetterMessage.Clone();

            resubmitMessage.UserProperties.Remove("DeadLetterReason");
            resubmitMessage.UserProperties.Remove("DeadLetterErrorDescription");
            
            await queueClient.SendAsync(resubmitMessage);
            await deadLetterReceiver.CompleteAsync(deadLetterMessage.SystemProperties.LockToken);            

            scope.Complete();            

There’s a few points to note in the above code:

FormatDeadLetterPath

FormatDeadLetterPath gives you the entity path for the dead letter queue, based on an entity.

Transaction Scope

The scope ensures that everything between its creation and completion happens as a single transaction. That is, if part of that fails, the whole thing fails. For example, you could add a throw new exception between the send and the complete, and the new message will not send.

We’re using the new C# 8 using statement – that is, it will apply to everything between it, and the end of the method.

ServiceBusConnection

There are several overloads for most of these methods, and typically, you can pass a connection string into the constructor – for example, MessageReceiver could be called like this:

new MessageReceiver(connectionString, QUEUE_NAME);

Typically, you can use this and it works exactly the same as if you established your own connection and passed that through; however, with a transaction, everything needs to share a connection. If they do not, then you may see an error such as this:

Transaction hasn’t been declared yet, or has already been discharged

Hence we’re creating the connection upfront.

References

https://blogs.infosupport.com/implementing-a-retry-pattern-for-azure-service-bus-with-topic-filters/

https://stackoverflow.com/questions/38784331/how-to-peek-the-deadletter-messages

https://github.com/Azure/azure-service-bus/pull/91

Add Storage Queue Message

I’ve written quite extensively in the past about Azure, and Azure Storage. I recently needed to add a message to an Azure storage queue, and realised that I had never written a post about that, specifically. As with many Azure focused .Net activities, it’s not too complex; but I do like to have my own notes on things.

If you’ve arrived at this post, you may find it’s very similar to the Microsoft documentation.

How to add a message

The first step is to install a couple of NuGet packages:

Install-Package Microsoft.Azure.Storage.Common
Install-Package Microsoft.Azure.Storage.Queue

My preference for these kinds of things is to create a helper: largely so that I can mock it out for testing; however, even if you fundamentally object to the concept of testing, you may find such a class helpful, as it keeps all your code in one place.

 public class StorageQueueHelper
{
        private readonly string _connectionString;
        private readonly string _queueName;

        public StorageQueueHelper(string connectionString, string queueName)
        {
            _connectionString = connectionString;
            _queueName = queueName;
        }

        public async Task AddNewMessage(string messageBody)
        {
            var queue = await GetQueue();

            CloudQueueMessage message = new CloudQueueMessage(messageBody);
            await queue.AddMessageAsync(message);
        }

        private async Task<CloudQueue> GetQueue()
        {
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);
            CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
            CloudQueue queue = queueClient.GetQueueReference(_queueName);
            await queue.CreateIfNotExistsAsync();

            return queue;
        }
}

The class above works for a single queue, and storage account. Depending on your use case, this might not be appropriate.

The GetQueue() method here is a bit naughty, as it actually changes something (or potentially changes something). Essentially, all it’s doing is connecting to a cloud storage account, and then getting a reference to a queue. We know that the queue will exist, because we’re forcing it to (CreateIfNotExistsAsync()).

Back in AddNewMessage(), once we have the queue, it’s trivial to simply create the message and add it.

References

https://docs.microsoft.com/en-us/azure/storage/queues/storage-tutorial-queues

Using Kudu to Edit a Deployed app.settings file

When you deploy an Azure App Service, there are occasions when you may need to change the running values. Exactly how you may do this depends heavily on how the service was deployed, and how you are managing your variables. You can typically overwrite variables inside the app service; however, the running appsettings.json and web.config will be deployed with the app, and you can edit these directly (whether you should or not is a different question).

It’s your foot

These instructions let you change the deployed files on your App Service. Doing so may result in the behaviour of the site changing.

On with the show

Launch Kudu

When you select this, you’ll be given the option to select “Go”, or … not. Select “Go”.

This will take you to the Kudu console.

From here, select “CMD”; this will take you to a hybrid screen with a command console and a file navigator:

Navigate to d:\home\site\wwwroot:

Too Many Files

Initially, you may get the following error:

There are n items in this directory, but maxViewItems is set to 99. You can increase maxViewItems by setting it to a larger value in localStorage.

To get around this, select F12 and in the console window type:

window.localStorage['maxViewItems'] = 1000

After you’ve changed this, refresh the page (F5).

App Settings

To change the variables, you’ll need to locate the appsettings.json in the list (it’s alpha-numerically sorted, so it should be near the top). (Unfortunately, you can’t edit this from the command line).

When you find the file, click the edit button:

And then change the file:

When done, select Save and then restart the app service.

References

https://www.poppastring.com/blog/kudu-error-with-maxviewitems-in-localstorage