Tag Archives: Azure Service Bus

What can you do with a logic app? Part three – Creating a Logic App Client

One of the things that are missing from Azure Logic apps is the ability to integrate human interaction. Microsoft do have their own version of an interactive workflow (PowerApps), which is (obviously) far better than what you can produce by following this post.

In this post, we’ll create a very basic client for a logic app. Obviously, with some thought, this could easily be extended to allow a fully functional, interactive, workflow system.

Basic Logic App

Let’s start by designing our logic app. The app in question is going to be a very simple one. It’s format is going to be that it will add a message to a logging queue (just so it has something to do), then we’ll ask the user a question; and we’ll do this by putting a message onto a topic: left or right. Based on the user’s response, we’ll either write a message to the queue saying left, or right. Let’s have a look at our Logic App design:

It’s worth pointing out a few things about this design:
1. The condition uses the expression base64ToString() to convert the encoded message into plain text.
2. Where the workflow picks up, it uses a peek-lock, and then completes the message at the end. It looks like it’s a ‘feature’ of logic apps that an automatic complete on this trigger will not actually complete the message (plus, this is actually a better design).

Queues and Topics

The “Log to message queue” action above is putting an entry into a queue; so a quick note about why we’re using a queue for logging, and a topic for the interaction with the user. In a real life version of this system, we might have many users, but they might all want to perform the same action. Let’s say that they all are part of a sales process, and the actions are actually actions along that process; adding these to a queue maintains their sequence. Here’s the queue and topic layout that I’m using for this post:

Multiple Triggers

As you can see, we actually have two triggers in this workflow. The first starts the workflow (so we’ll drop a message into the topic to start it), and the second waits for a second message to go into the topic.

To add a trigger part way through the workflow, simply add an action, search and select “Triggers”:

Because we have a trigger part way through the workflow, what we have effectively issued here is an await statement. Once a message appears in the subscription, the workflow will continue where it left off:

As soon as a message is posted, the workflow carries on:

Client Application

For the client application, we could simply use the Service Bus Explorer (in fact, the screenshots above were taken from using this to simulate messages in the topic). However, the point of this post is to create a client, and so we will… although we’ll just create a basic console app for now.

We need the client to do two things: read from a topic subscription, and write to a topic. I haven’t exactly been here before, but I will be heavily plagiarising from here, here, and here.

Let’s create a console application:

Once that’s done, we’ll need the service bus client library: Install it from here.

The code is generally quite straight-forward, and looks a lot like the code to read and write to queues. The big difference is that you don’t read from a topic, but from a subscription to a topic (a topic can have many subscriptions):

class Program
{
    
    static async Task Main(string[] args)
    {
        MessageHandler messageHandler = new MessageHandler();
        messageHandler.RegisterToRead("secondstage", "sub1");
 
        await WaitForever();
    }
 
    private static async Task WaitForever()
    {
        while (true) await Task.Delay(5000);
    }
}
public class MessageHandler
{
    private string _connectionString = "service bus connection string details";
    private ISubscriptionClient _subscriptionClient;
    public void RegisterToRead(string topicName, string subscriptionName)
    {            
        _subscriptionClient = new SubscriptionClient(_connectionString, topicName, subscriptionName);
 
        MessageHandlerOptions messageHandlerOptions = new MessageHandlerOptions(ExceptionReceived)
        {
            AutoComplete = false,
            MaxAutoRenewDuration = new TimeSpan(1, 0, 0)
        };
 
        _subscriptionClient.RegisterMessageHandler(ProcessMessage, messageHandlerOptions);
 
    }
 
    private async Task ProcessMessage(Message message, CancellationToken cancellationToken)
    {
        string messageText = Encoding.UTF8.GetString(message.Body);
 
        Console.WriteLine(messageText);
        string leftOrRight = Console.ReadLine();
 
        await _subscriptionClient.CompleteAsync(message.SystemProperties.LockToken);
 
        await SendResponse(leftOrRight, "userinput");
    }
 
    private async Task SendResponse(string leftOrRight, string topicName)
    {
        TopicClient topicClient = new TopicClient(_connectionString, topicName);
        Message message = new Message(Encoding.UTF8.GetBytes(leftOrRight));
        await topicClient.SendAsync(message);
    }
 
    private Task ExceptionReceived(ExceptionReceivedEventArgs arg)
    {
        Console.WriteLine(arg.Exception.ToString());
        return Task.CompletedTask;
    }
}

If we run it, then when the logic app reaches the second trigger, we’ll get a message from the subscription and ask directions:

Based on the response, the logic app will execute either the right or left branch of code.

Summary

Having worked with workflow systems in the past, one recurring feature of them is that they start to get used for things that don’t fit into a workflow, resulting in a needlessly over-complex system. I imagine that Logic Apps are no exception to this rule, and in 10 years time, people will roll their eyes at how Logic Apps have been used where a simple web service would have done the whole job.

The saving grace here is source control. The workflow inside a Logic App is simply a JSON file, and so it can be source controlled, added to a CI pipeline, and all the good things that you might expect. Whether or not a more refined version of what I have described here makes any sense is another question.

There are many downsides to this approach: firstly, you are fighting against the Service Bus by asking it to wait for input (that part is a very fixable problem with a bit of an adjustment to the messages); secondly, you would presumably need some form of timeout (again, a fixable problem that will probably feature in a future post). The biggest issue here is that you are likely introducing complex conditional logic with no way to unit test; this isn’t, per se, fixable; however, you can introduce some canary logic (again, this will probably be the feature of a future post).

References

https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-limits-and-config

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-how-to-use-topics-subscriptions

https://stackoverflow.com/questions/28127001/the-lock-supplied-is-invalid-either-the-lock-expired-or-the-message-has-alread

Reading Azure Service Bus Queue Names from the Config File

In this post, I wrote about how you might read a message from the service bus queue. However, with Azure Functions (and WebJobs), comes the ability to have Microsoft do some of this plumbing code for you.

I have a queue here (taken from the service bus explorer):

I can read this in an Azure function; let’s create a new Azure Functions App:

This time, we’ll create a Service Bus Queue Triggered function:

Out of the box, that will give you this:

public static class Function1
{
    [FunctionName("Function1")]
    public static void Run([ServiceBusTrigger("testqueue", AccessRights.Listen, Connection = "")]string myQueueItem, TraceWriter log)
    {
        log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");
    }
}

There’s a few things that we’ll probably want to change here. The first is “Connection”. We can remove that parameter altogether, and then add a row to the local.settings.json file (which can be overridden later inside Azure). Out of the box, you get AzureWebJobsStorage and AzureWebJobsDashboard, which both accept a connection string to a Azure Storage Account. You can also add AzureWebJobsServiceBus, which accepts a connection string to the service bus:

"Values": {
    "AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=teststorage1…",
    "AzureWebJobsDashboard": "DefaultEndpointsProtocol=https;AccountName=teststorage1…",
    "AzureWebJobsServiceBus": "Endpoint=sb://pcm-servicebustest.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=…"
  }

If you run the job, it will now pick up any outstanding entries in that queue. But, what if you don’t know the queue name; for example, what if you find out the queue name is different. To illustrate the point; here, I’m looking for “testqueue1”, but the queue name (as you saw earlier) is “testqueue”:

public static class Function1
{
    [FunctionName("Function1")]
    public static void Run([ServiceBusTrigger("testqueue1", AccessRights.Listen)]string myQueueItem, TraceWriter log)
    {
        log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");
    }
}

Obviously, if you’re looking for a queue that doesn’t exist, bad things happen:

To fix this, I have to change the code… which is broadly speaking a bad thing. What we can do, is configure the queue name in the config file; like this:

"Values": {
    "AzureWebJobsStorage": " . . . ",
    . . .,
    "queue-name":  "testqueue"
  }

And we can have the function look in the config file by changing the queue name:

[FunctionName("Function1")]
public static void Run([ServiceBusTrigger("%queue-name%", AccessRights.Listen)]string myQueueItem, TraceWriter log)
{
    log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");
}

The pattern of supplying a variable name in the format “%variable-name%” seems to work across other triggers and bindings for Azure Functions.

Deployment

That’s now looking much better, but what happens when the function gets deployed? Let’s see:

We can now see that the function is deployed:

At the minute, it won’t do anything, because it’s looking for a queue name in a setting that only exists locally. Let’s fix that:

Remember to save the changes.

Looking at the logs confirms that this now runs correctly.

Setting up an e-mail Notification System using Logic Apps

One of the new features of the Microsoft’s Azure offering are Logic Apps: these are basically a workflow system, not totally dis-similar to Windows Workflow (WF so as not to get sued by panda bears). I’ve worked with a number of workflow systems in the past, from standard offerings to completely bespoke versions. The problem always seems to be that, once people start using them, they become the first thing you reach for to solve every problem. Not to say that you can’t solve every problem using a workflow (obviously, it depends which workflow and what you’re doing), but they are not always the best solution. In fact, they tend to be at their best when they are small and simple.

With that in mind, I thought I’d start with a very straightforward e-mail alert system. In fact, all this is going to do is to read an service bus queue and send an e-mail. I discussed a similar idea here, but that was using a custom written function.

Create a Logic App

The first step is to create a new Logic App project:

There are three options here: create a blank logic app, choose from a template (for example, process a service bus message), or define your own with a given trigger. We’ll start from a blank app:

Trigger

Obviously, for a workflow to make sense, it has to start on an event or a schedule. In our case, we are going to run from a service bus entry, so let’s pick that from the menu that appears:

In this case, we’ll choose Peek-Lock, so that we don’t lose our message if something fails. I can now provide the connection details, or simply pick the service bus from a list that it already knows about:

It’s not immediately obvious, but you have to provide a connection name here:

If you choose Peek-Lock, you’ll be presented with an explanation of what that means, and then a screen such as the following:

In addition to picking the queue name, you can also choose the queue type (as well as listening to the queue itself, you can run your workflow from the dead-letter queue – which is very useful in its own right, and may even be a better use case for this type of workflow). Finally, you can choose how frequently to poll the queue.

If you now pick “New step”, you should be faced with an option:

In our case, let’s provide a condition (so that only queue messages with “e-mail” in the message result in an e-mail):

Before progressing to the next stage – let’s have a look at the output of this (you can do this by running the workflow and viewing the “raw output”):

Clearly the content data here is not what was entered. A quick search revealed that the data is Base64 encoded, so we have to make a small tweak in advanced mode:

Okay – finally, we can add the step that actually sends the e-mail. In this instance, I simply picked Outlook.com, and allowed Azure access to my account:

The last step is to complete the message. Because we only took a “peek-lock”, we now need to manually complete the message. In the designer, we just need to add an action:

Then tell it that we want to use the service bus again. As you can see – that’s one of the options in the list:

Finally, it wants the name of the queue, and asks for the lock token – which it helpfully offers from dynamic content:

Testing

To test this, we can add a message to our test queue using the Service Bus Explorer:

I won’t bother with a screenshot of the e-mail, but I will show this:

Which provides a detailed overview of exactly what has happened in the run.

Summary

Having a workflow system built into Azure seems like a two edged sword. On the one hand, you could potentially use it to easily augment functionality and quickly plug holes; however, on the other hand, you might find very complex workflows popping up all over the system, creating an indecipherable architecture.

Using Azure Functions to Send an E-mail Alert from a Service Bus

In this post, I discussed creating an Azure service bus that sends an e-mail as an action once a message has expired; and in this post, I covered Azure functions and setting a basic one up.

These two pieces of functionality seem to be crying out to be together. After all, if your functionality to send an e-mail is in the cloud, you don’t have to worry about your server being down (which, if your message has expired, is a real possibility).

Create the Azure Function

The first thing to do is to create the Azure function to send an e-mail. Remember that we’ll be hooking into the service bus, and so we’ll create the function a little differently.

The first few steps are the same, though:

The new function is here:

We’ll create a custom function again:

Although this looks familiar from the last post, the next part does differ slightly. This time, we’ll set up a Service Bus Trigger:

This requires the connection string to your service bus…

As you can see above, the service bus connection is blank, and there are no possible entries… onto App Settings:

App Settings

On the App Settings tab, you can configure settings that pertain to your Azure Function App. Select “Manage App Settings”. Here we can set-up a connection string:

Now, we should be able to see that from the Function:

Does it work?

What does this function do out of the box?

Well, having populated the queue with 50 messages that time out after 30 seconds, the function kicked in and started logging that it was picking up messages after 30 seconds – so that’s a promising sign!

The messages are processed and removed from the dead letter queue. This process happens so quickly, it’s easy (as I did) to interpret this as a bug (i.e. messages are not being dead-lettered). However, as we can see from the function logs – they are.

This did, however, leave me with a concern that the messages were being disposed of before they had been successfully processed. To check this, I changed the function slightly:

So, it crashes correctly:

And here, safe and sound, are 50 freshly dead-lettered messages:

Function Code

Now we have a function, we need to make it send an e-mail… so we’ll need some code. Let’s start with what we created here.

using System;
using System.Threading.Tasks;
using System.Net.Mail;

public static void Run(string myQueueItem, TraceWriter log)
{
    log.Info($"Start C# ServiceBus queue trigger function processed message: {myQueueItem}");

    System.Net.Mail.MailMessage message = new System.Net.Mail.MailMessage();
    message.To.Add("[email protected]");
    message.Subject = "Message in queue has expired";
    message.From = new System.Net.Mail.MailAddress("[email protected]");
    message.Body = messageText;
    System.Net.Mail.SmtpClient smtp = new System.Net.Mail.SmtpClient("smtp.live.com");
    smtp.Port = 587;
    smtp.UseDefaultCredentials = false;
    smtp.Credentials = new System.Net.NetworkCredential("[email protected]", "[email protected]sw0rd");
    smtp.EnableSsl = true;
    smtp.Send(message);

    log.Info($"End C# ServiceBus queue trigger function processed message: {myQueueItem}");
}


This doesn’t work:

2017-06-27T16:47:56.928 Function started (Id=1188dbdb-4963-4e55-af5c-4be1f71a1ca5)
2017-06-27T16:47:56.928 Start C# ServiceBus queue trigger function processed message: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA32
2017-06-27T16:47:56.928 Function completed (Failure, Id=1188dbdb-4963-4e55-af5c-4be1f71a1ca5, Duration=0ms)
2017-06-27T16:47:57.147 Exception while executing function: Functions.ServiceBusQueueTriggerCSharp1. mscorlib: Exception has been thrown by the target of an invocation. f-ServiceBusQueueTriggerCSharp1__-1971403142: Cannot complete.
2017-06-27T16:47:57.557 Exception while executing function: Functions.ServiceBusQueueTriggerCSharp1. mscorlib: Exception has been thrown by the target of an invocation. f-ServiceBusQueueTriggerCSharp1__-1971403142: Cannot complete.

Debugging Azure

A quick side note on debugging Azure. There are a number of resources with details of how this should work on the web, and I’ll probably have a later post of my own experiences, but it’s a pretty flaky experience, and I ended up using trial and error to determine the issue.

Working code

using System;
using System.Threading.Tasks;

public static void Run(string myQueueItem, TraceWriter log)
{
    log.Info($"Start C# ServiceBus queue trigger function processed message: {myQueueItem}");

    System.Net.Mail.MailMessage message = new System.Net.Mail.MailMessage();
    
    message.To.Add("[email protected]");    
    message.Subject = "Message in queue has expired";    
    message.From = new System.Net.Mail.MailAddress("[email protected]");
    message.Body = myQueueItem;
        
    System.Net.Mail.SmtpClient smtp = new System.Net.Mail.SmtpClient("smtp.live.com");
    smtp.Port = 587;
    smtp.UseDefaultCredentials = false;
    smtp.Credentials = new System.Net.NetworkCredential("[email protected]", "[email protected]");
    smtp.EnableSsl = true;
    smtp.Send(message);

    log.Info($"End C# ServiceBus queue trigger function processed message: {myQueueItem}");
}

So, the problem was just that I was referencing an unknown variable (messageText). I’m unsure exactly why I needed to travel to the mountains of Mordor to determine this – a simple error message in the online text would have sufficed.

The other issue that I faced was a security challenge; however, once I’d persuaded Azure that this really was me, everything sprung into life:

Credit Considerations

Unlike in previous posts where I’ve identified the Azure cost to be negligible, functions are the fastest way to use up credit I have found so far. Especially functions such as I’ve created here. I left the (non-working) function above active, but failing all night, and it used up over £40 worth of credit, continually trying, and failing, to process the dead-letter queue… I think the lights might even have dimmed in Redmond for a split second! The moral of the story is is: be careful when you’re debugging this – you can’t just leave at the end of the night with a function that doesn’t work, but is still active.

Summary

This concept is extremely compelling. I can have a service bus queue that is processed and monitored by an Azure function. If aliens land and steal the entire office, all the servers, dev PCs and programmers, this function will continue to run. There is obviously a mindset shift here, and it doesn’t make sense to move everything into this kind of model, but consider the possibilities; imagine a system that books holidays: it processes the customer request and adds it to a queue; the aeroplane booking system picks that from the queue and books the ticket on the plane, the car hire system takes the message to book a car, once they’re all complete they add respective messages to say so (but remain agnostic of each other), finally, if any one part of the system fails, an Azure function could sit there monitoring and cancel the whole lot. I’ve never worked in this kind of industry, so there’s a lot that I’ve probably not considered, but the essence is that you can have active functionality on (even catastrophic) failure – which is a brand new concept.

References

https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus

https://stackoverflow.com/questions/10043219/view-content-of-an-azure-service-bus-queue

Service Bus Explorer:

https://code.msdn.microsoft.com/Service-Bus-Explorer-f2abca5a

http://markheath.net/post/remote-debugging-azure-functions

Sending e-mails:

https://stackoverflow.com/questions/25216202/smtp-live-com-mailbox-unavailable-the-server-response-was-5-7-3-requested-ac

Azure Service Bus – Send an e-mail on Message Timeout

A message queue has, in its architecture, two main points of failure; the first is the situation where a message is added to a queue, but never read (or at least not read within a specified period of time); this is called a Dead Letter, and it is the subject of this post. The second is where the message is corrupt, or it breaks the reading logic in some way; that is known as a Poison Message.

There are a number of reasons that a message might not get read in the specified time: the service reading and processing the messages might not be keeping up with the supply, it might have crashed, the network connection might have failed.

One possible thing to do at this stage, is to have a process that automatically notifies someone that a message has ended up in the dead letter queue.

Step One – specify a timeout

Here’s how you would specify a timeout on the message specifically:

           BrokeredMessage message = new BrokeredMessage(messageBody)
            {
                MessageId = id,
                TimeToLive = new TimeSpan(0, 5, 0)
            };

Or, you can create a default on the queue from the QueueDescription (typically this would be done when you initially create the queue:

                QueueDescription qd = new QueueDescription("TestQueue")
                {
                    DefaultMessageTimeToLive = new TimeSpan(0, 5, 0)
                };
                nm.CreateQueue(qd);

Should these values differ, the shortest time will be taken.

What happens to the message by default?

I’ve added a message to the queue using the default timeout of 5 minutes; here it is happily sitting in the queue:

Looking at the properties of the queue, we can determine that the “TimeToLive” is, indeed, 5 minutes:

In addition, you can see that, by default, the flag telling Service Bus to move the message to a dead letter queue is not checked. This means that the message will not be moved to the dead letter queue.

5 Minutes later:

Nothing has happened to this queue, except time passing. The message has now been discarded. It seems an odd behaviour; however, as with ReadAndDelete Locks there may be reasons that this behaviour is required.

Step Two – Dead Letters

If you want to actually do something with the expired message, the key is a concept called “Dead Lettering”. The following code will direct the Service Bus to put the offending message into the “Dead Letter Queue”:

                QueueDescription qd = new QueueDescription("TestQueue")
                {
                    DefaultMessageTimeToLive = new TimeSpan(0, 5, 0),
                    EnableDeadLetteringOnMessageExpiration = true
                };
                nm.CreateQueue(qd);

Here’s the result for the same test:

Step Three – Doing something with this…

Okay – so the message hasn’t been processed, and it’s now sat in a queue specially designed for that kind of thing, so what can we do with it? One possible thing is to create a piece of software that monitors this queue. This is an adaptation of the code that I originally created here:

        static void Main(string[] args)
        {
            System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
            sw.Start();

            if (!InitialiseClient())
            {
                Console.WriteLine("Unable to initialise client");
            }
            else
            {
                while (true)
                {
                    string message = ReadMessage("TestQueue/$DeadLetterQueue");

                    if (string.IsNullOrWhiteSpace(message)) break;
                    Console.WriteLine($"{DateTime.Now}: Message received: {message}");
                }
            }

            sw.Stop();
            Console.WriteLine($"Done ({sw.Elapsed.TotalSeconds}) seconds");
            Console.ReadLine();
        }

        private static bool InitialiseClient()
        {
            Uri uri = ServiceManagementHelper.GetServiceUri();
            TokenProvider tokenProvider = ServiceManagementHelper.GetTokenProvider(uri);

            NamespaceManager nm = new NamespaceManager(uri, tokenProvider);
            return nm.QueueExists("TestQueue");
        }

        private static string ReadMessage(string queueName)
        {
            QueueClient client = QueueManagementHelper.GetQueueClient(queueName, true);

            BrokeredMessage message = client.Receive();
            if (message == null) return string.Empty;
            string messageBody = message.GetBody<string>();

            //message.Complete();

            return messageBody;
        }

If this was all that we had to monitor the queue, then somebody’s job would need to be to watch this application. That may make sense, depending on the nature of the business; however, we could simply notify the person in question that there’s a problem. Now, if only the internet had a concept of an offline messaging facility that works something akin to the postal service, only faster…

        static void Main(string[] args)
        {
            System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
            sw.Start();

            if (!InitialiseClient())
            {
                Console.WriteLine("Unable to initialise client");
            }
            else
            {
                while (true)
                {
                    string message = ReadMessage("TestQueue/$DeadLetterQueue");

                    if (string.IsNullOrWhiteSpace(message)) break;
                    Console.WriteLine($"{DateTime.Now}: Message received: {message}");

                    Console.WriteLine($"{DateTime.Now}: Send e-mail");
                    SendEmail(message);
                }
            }

            sw.Stop();
            Console.WriteLine($"Done ({sw.Elapsed.TotalSeconds}) seconds");
            Console.ReadLine();
        }

        private static void SendEmail(string messageText)
        {
            System.Net.Mail.MailMessage message = new System.Net.Mail.MailMessage();
            message.To.Add("[email protected]");
            message.Subject = "Message in queue has expired";
            message.From = new System.Net.Mail.MailAddress("[email protected]");
            message.Body = messageText;
            System.Net.Mail.SmtpClient smtp = new System.Net.Mail.SmtpClient("smtp.live.com");
            smtp.Port = 587;
            smtp.UseDefaultCredentials = false;
            smtp.Credentials = new System.Net.NetworkCredential("[email protected]", "passw0rd");
            smtp.EnableSsl = true;
            smtp.Send(message);
        }

In order to prevent a torrent of mails, you might want to put a delay in this code, or even maintain some kind of list so that you only send one mail per day.

References

https://docs.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.queuedescription.enabledeadletteringonmessageexpiration?view=azureservicebus-4.0.0#Microsoft_ServiceBus_Messaging_QueueDescription_EnableDeadLetteringOnMessageExpiration

https://www.codit.eu/blog/2015/01/automatically-expire-messages-in-azure-service-bus-how-it-works/

https://stackoverflow.com/questions/9851319/how-to-add-smtp-hotmail-account-to-send-mail

Implicitly Acknowledging a Message from Azure Service Bus

In this post I discussed receiving, processing and acknowledging a message using the Azure Service Bus. There are two ways to acknowledge a message received from the queue (which are common to all message broker systems that I’ve used so far). That is, you either take the message, process it, and then go back to the broker to tell it you’re done (explicit acknowledgement); or, you remove the queue and then process it (implicit acknowledgement).

Explicit Acknowledgement / PeekLock

If the message is not processed within a period of time, then it will be unlocked and returned to the queue to be picked up by the next client request.

The code for this is as follows (it is also the default behaviour in Azure Service Bus):

QueueClient queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName, ReceiveMode.PeekLock);

Remember that, with this code, if you don’t call:

message.Complete();

Then you will repeatedly read the same message over and over again.

Implicit Acknowledgement / ReadAndDelete

Here, if the message is not processed within a period of time, or fails to process, then it is likely lost. So, why would you ever use this method of acknowledgement? Well, speed is the main reason; because you don’t need to go back to the server, you potentially increase the whole transaction speed; furthermore, there is clearly work involved for the broker in maintaining the state of a message on the queue, expiring the message lock, etc.

The code for the implicit acknowledgement is:

QueueClient queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName, ReceiveMode.ReceiveAndDelete);

References

https://docs.microsoft.com/en-us/rest/api/servicebus/peek-lock-message-non-destructive-read

A C# Programmer’s Guide to Queues and Sending a Message with Azure Service Bus

I have previously written about message queue systems. The big two, as far as I can see, are Active MQ and RabbitMQ.

Microsoft have always had MSMQ*, but it’s not really a message broker as such (I believe that you can get similar behaviour using NServiceBus, but have never tried that myself). However, with Azure comes the Azure Service Bus.

The first thing that you need to do is set-up an Azure account. Note that Microsoft offer Azure as a paid service, and so this is not free. However, they also offer free trials and free Azure credit if you have an MSDN.

Log on to:

https://portal.azure.com

Namespace

Namespaces are an important concept in Azure. They basically allow you to split a single Azure account across many functions, but what that means is that everything you do relates to a specific namespace.

To add one, first, pick a pricing tier:

Make sure that your Namepsace isn’t taken:

You’ll then get an alert to say it worked:

If you refresh, you should now see your namespace:

Create Test Project

I always try to start with a console app when trying new stuf. Add NuGet reference:

It is my understanding that, as with ActiveMQ and RabbitMQ, these client libraries are an abstraction over a set of HTTP Post calls. In the case of Azure, I believe that, behind the scenes, it uses WCF to handle all this.

Using the Namespace

Using a message queue system such as RabbitMQ or ActiveMQ, you need a message queue server, and a URL that relates to it. However, one of the things Azure allows you to do is to abstract that; for example:

        static void Main(string[] args)
        {
            Console.WriteLine($"Getting service bus URI...");
            Uri uri = ServiceBusEnvironment.CreateAccessControlUri("pcm-servicebustest");
            Console.WriteLine($"Service Bus URI: {uri.ToString()}");
            Console.ReadLine();
        }

Tells me what the URI of the message queue broker is:

Adding a message to a queue

In order to do anything with a message queue in Azure, you need a token; effectively, this provides a level of security

Tokens

Get the key:

You can store these details in the app/web.config, or you can use them programmatically:

        private static TokenProvider GetTokenProvider(Uri uri)
        {
            Console.WriteLine($"Getting token...");
            TokenProvider tp = TokenProvider.CreateSharedAccessSignatureTokenProvider("RootManageSharedAccessKey", "JWh82nkstIAi4w5tW6MEj7GKQfoiZlwBYjHx9wfDqdA=");                                                

            Console.WriteLine($"Token {tp.ToString()}");
            return tp;
        }

Queues

Putting the above calls together, we can now create a queue in Azure:

        private static void CreateNewQueue(Uri uri, TokenProvider tokenProvider)
        {
            Console.WriteLine($"Creating new queue...");
            NamespaceManager nm = new NamespaceManager(uri, tokenProvider);

            Console.WriteLine($"Created namespace manager for {nm.Address}");
            if (nm.QueueExists("TestQueue"))
            {
                Console.WriteLine("Queue already exists");
            }
            else
            {
                Console.WriteLine("Creating new queue");
                QueueDescription qd = nm.CreateQueue("TestQueue");
            }
        }

Incidentally, the act of creating a queue appears to have cost £0.24 GBP. If you have MSDN, you should get £40 GBP credit each month (at the time of writing).

Now we have a queue, let’s put some messages on it.

Adding a message

        private static void AddNewMessage(string id, string messageBody, string queueName)
        {
            BrokeredMessage message = new BrokeredMessage(messageBody)
            {
                MessageId = id
            };

            string connectionString = GetConnectionString();
            
            QueueClient queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName);
            queueClient.Send(message);
        }

The Connection String can be found here:

We can now see that a message has, indeed, been added to the queue:

At this time, this is about as much as you can see from this portal.

Errors

These are some errors that I encountered during the creation of this post, and their solutions.

System.UnauthorizedAccessException

System.UnauthorizedAccessException: ‘The token provider was unable to provide a security token while accessing ‘https://pcm-servicebustest-sb.accesscontrol.windows.net/WRAPv0.9/’. Token provider returned message: ‘The remote name could not be resolved: ‘pcm-servicebustest-sb.accesscontrol.windows.net”.’

The cause is not an invalid secret

That’s because this line:

TokenProvider tp = TokenProvider.CreateSharedSecretTokenProvider("RootManageSharedAccessKey", "jjdsjdsjk");

Gives the error:

System.ArgumentException: ‘The ‘issuerSecret’ is invalid.’

The fix…

This code is littered throughout the web:

TokenProvider tp = TokenProvider.CreateSharedSecretTokenProvider("RootManageSharedAccessKey", "jjdsjdsjk");

But the correct code was:

TokenProvider tp = TokenProvider.CreateSharedAccessSignatureTokenProvider("RootManageSharedAccessKey", "JWh82nkstIAi4w5tW6MEj7GKQfoiZlwBYjHx9wfDqdA=");                                                

System.ArgumentNullException: ‘Queue name should be specified as EntityPath in connectionString.’

Or: 40400: Endpoint not found.

Microsoft.ServiceBus.Messaging.MessagingEntityNotFoundException: ‘40400: Endpoint not found., Resource:sb://pcm-servicebustest.servicebus.windows.net/atestqueue. TrackingId:48de75d7-fb01-4fa9-b72e-20a5dc090a8d_G11, SystemTracker:pcm-servicebustest.servicebus.windows.net:aTestQueue, Timestamp:5/25/2017 5:23:27 PM

Means (obviously) that the following code:

QueueClient.CreateFromConnectionString(connectionString, queueName);

Either doesn’t have the queue name, or it is wrong.

References

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-exceptions

https://blogs.msdn.microsoft.com/brunoterkaly/2014/08/07/learn-how-to-create-a-queue-place-and-read-a-message-using-azure-service-bus-queues-in-5-minutes/

https://stackoverflow.com/questions/18558299/servicebus-throws-401-unauthorized-error

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-queues-topics-subscriptions

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-how-to-use-topics-subscriptions

https://msdn.microsoft.com/en-us/library/jj542433.aspx?f=255&MSPPError=-2147217396

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-multi-tier-app-using-service-bus-queues

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues

* Microsoft probably haven’t ALWAYS had MSMQ. There was probably a time in the early 90’s where they didn’t have a message queue system at all.

Console Application Builds, But Will Not Run

While doing some testing recently, I created a new bog standard console application and, on pressing F5, nothing happened.

The project builds fine, but wouldn’t launch the console window.

Why (and how to fix)?

Well, I had installed the Azure Service Bus Client. Other than that, I can’t really say; however, the fix does kind of make sense:

Uncheck the “Prefer 32-bit” checkbox, and it all springs back to life!