Monthly Archives: September 2022

Sending Messages at Scale – Cannot allocate more handles

When sending many messages (thousands or tens of thousands) you may find that you run into errors that you may not see while testing with lower numbers of messages. One such error that I’ve encountered recently is this:

Azure.Messaging.ServiceBus.ServiceBusException: ‘Cannot allocate more handles. The maximum number of handles is 4999. (QuotaExceeded)’

Some code to replicate the issue:

for (int i = 1; i <= 10000; i++)
{
    var sender = serviceBusClient.CreateSender("topic-name");

    string messageText = $"Test{i}--{DateTime.Now.ToString("yy-MM-dd")}";
    var msg = new ServiceBusMessage(Encoding.UTF8.GetBytes(messageText));
    await sender.SendMessageAsync(msg);
}    

There is a limit on concurrent connections to the service bus, set at 4,999 at the time of writing; that is, you cannot have 5,000 connections at the same time. Every time a call to CreateSender is called, a new connection is made to the service bus. Because this is a tight loop, the connections cannot be cleaned up in time, and so it overwhelms the service.

There are three mechanisms for avoiding this. By far the best and easiest is to send the messages as a batch.

Sending Messages as a Batch

List<ServiceBusMessage> serviceBusMessages = new();    
for (int i = 1; i <= 10000; i++)
{
    string messageText = $"Test{i}--{DateTime.Now.ToString("yy-MM-dd")}";
    serviceBusMessages.Add(
       new ServiceBusMessage(Encoding.UTF8.GetBytes(messageText)));        
}
await sender.SendMessagesAsync(serviceBusMessages);

This is far faster, and avoids 10,000 calls to the service. The caveat here is the size of the message – if the 10,000 messages exceed a single message size, then it may be necessary to batch the messages into groups of 1000, or even 100 – depending on the size of each message.

Cleaning up the Client

This mechanism is to forcibly dispose of the message factory each loop:

for (int i = 1; i <= count; i++)
{
    var sender = serviceBusClient.CreateSender("topic-name");

    string messageText = $"Test{i}--{DateTime.Now.ToString("yy-MM-dd")}";
    var msg = new ServiceBusMessage(Encoding.UTF8.GetBytes(messageText));
    await sender.SendMessageAsync(msg);

    await sender.CloseAsync();
}    

Change the Scope of the Message Factory

The final method is to simply move the creation of the message factory outside the loop:

    var sender = serviceBusClient.CreateSender("topic-name");
    for (int i = 1; i <= count; i++)
    {
        string messageText = $"Test{i}--{DateTime.Now.ToString("yy-MM-dd")}";
        var msg = new ServiceBusMessage(Encoding.UTF8.GetBytes(messageText));
        await sender.SendMessageAsync(msg);

    }    

Summary

If you’re dealing with large quantities of messages, then the batch send is by far the best option, as it not only avoids the issue with too many handles, but speeds up the whole process.

ServiceBusAdministrationClient Update Lock Duration Not Working

When you start to experiment with very large messages, or very large quantities of messages, there are times when the default lock duration of 30 seconds for Azure Service Bus can cause issues. In this post, I’ll show how this can be changed at a subscription or queue level using the Azure.Messaging.ServiceBus client.

Caveat

As with previous articles that I’ve written on Service Bus, I’m not advocating this as a desirable way to deal with such cases; however, times do arise when it makes sense to adjust this value.

The Code

The key here is the ServiceBusAdministrationClient. It allows you to get a reference to the subscription or queue:

    var serviceBusAdministrationClient = new ServiceBusAdministrationClient(connectionString);
    var sub = await serviceBusAdministrationClient.GetSubscriptionAsync("topic", "sub1");

You can then change something on that object; for example:

sub.Value.LockDuration = TimeSpan.FromSeconds(newDuration);

But it’s not updating

Somewhat counter intuitively, the final step is to update with the new object:

await serviceBusAdministrationClient.UpdateSubscriptionAsync(sub);

The subscription / queue should then update fine.

Summary

In this post, we’ve discussed how you can change the lock duration using the Azure Service Bus SDK. We’ve also shown how easy it can be to not realise that you need to explicitly update (ask me how I know!)

Compressing a Service Bus Message Using Gzip

I’ve recently been playing around at the edges of Azure Service Bus. One of the things that occurred to me was that, for a large message, you could improve speed and save money by simply compressing the message before sending.

The obvious downside to this approach is that your consumer needs to be aware that the message is compressed. The other issue is that there is an overhead to compression and decompression, both in size and in speed, so compressing a small message is pointless (or worse – it may actually increase the size and time to transmit).

That’s the why and why not, let’s move onto the how.

How to Send a Compressed Message

Let’s start with a re-cap on how to send a message without it being compressed:

async Task SendMessage(string connectionString, string topicName, string messageText)
{
    await using var serviceBusClient = new ServiceBusClient(connectionString);
    var sender = serviceBusClient.CreateSender(topicName);
    
    var message = new ServiceBusMessage(Encoding.UTF8.GetBytes(messageText));

    await sender.SendMessageAsync(message);
}

Since this sends a message, which is just a stream of text, all we need to do now is compress that message. An easy way is to use GZipStream:

byte[] CompressBytes(byte[] bytes)
{    
    using var memoryStream = new MemoryStream();
    using var gzipStream = new GZipStream(memoryStream, CompressionLevel.SmallestSize);
    gzipStream.Write(bytes, 0, bytes.Length);
    gzipStream.Close();

    return memoryStream.ToArray();
}

This method takes an array of bytes, and runs them through the GZipStream to apply the Gzip compression algorithm on it. All we now need to do is to convert our string to a byte array – we can use something like the following, which also converts the resulting string to Base64 – meaning we can transport it easier:

string CompressMessage(string toCompress)
{
    var bytes = Encoding.UTF8.GetBytes(toCompress);

    var returnBytes = CompressBytes(bytes);

    string returnValue = Convert.ToBase64String(returnBytes);
    return returnValue;
}

We now have all the building blocks, we can simply assemble them in the correct order:

var compressed = CompressMessage(largeMessage);
await SendMessage(connectionString, "compressed-message", compressed);

All that’s left is to receive the message.

How to Receive a Compressed Message

Again, let’s start with a basic method to receive the next message on the service bus:

async Task<string> ConsumeNextMessage(string topic)
{
    await using var serviceBusClient = new ServiceBusClient(connectionString);
    var receiver = serviceBusClient.CreateReceiver(topic, "sub-1");
    var message = await receiver.ReceiveMessageAsync();
    return message.Body.ToString();
}

This will simply read the next message for the given subscription and return its contents. As with the send, we simply need the other building blocks; we’ll start with decompressing the bytes:

byte[] DecompressBytes(byte[] bytes)
{    
    using var memoryStream = new MemoryStream(bytes);
    using var outputStream = new MemoryStream();
    using var decompressStream = new GZipStream(memoryStream, CompressionMode.Decompress);
    decompressStream.CopyTo(outputStream);
    var returnBytes = outputStream.ToArray();
    return returnBytes;
}

Again, we’re using GZipStream to return a decompressed version of the data. We’ll call that in the same manner as above:

string DecompressMessage(string compressed)
{
    var bytes = Convert.FromBase64String(compressed);
    
    var returnBytes = DecompressBytes(bytes);
    return Encoding.UTF8.GetString(returnBytes);
}

It’s worth remembering that we’re transporting the compressed data using Base64, so we need to convert it from that first. Finally, we can assemble the methods:

var message = await ConsumeNextMessage("compressed-message");
var decompressed = DecompressMessage(message);

What difference does it make?

Is it really worth all this message around? The answer is probably an emphatic no. However, in a very small subset of cases, this might mean that you can continue on the standard tier, rather than upgrade to premium.

This also helps with speed – sending a 70MB message takes around 50 – 55 seconds in my tests. The same message compressed was around 5MB and took only 5 seconds to send.

References

https://docs.microsoft.com/en-us/dotnet/api/system.io.compression.gzipstream?view=net-6.0

https://www.infoworld.com/article/3660629/how-to-compress-and-decompress-strings-in-c-sharp.html

Transmitting an Image via Azure Service Bus

This post in a continuation of this earlier post around serialising an image. Once you’re able to serialise an image, then you can transmit that image. In this post, we’ll see how we could do that using Azure Service Bus.

A Recap on Image Serialisation

In the previous (linked) post, we saw how we can turn an image into a string of text, and back again. Essentially, what we did was use a BinaryWriter to write the file as a binary stream, and a BinaryReader to turn it back to an image.

The plan here is to do exactly the same thing, but to simply write the stream to an Azure Service Bus Message, and read from it at the other side.

Size Matters

One thing that you’re going to need to be aware of here is the size of the image. The free tier of Service Bus limits you to 256KB. Serialising an image to stream can be less than that, but unless you know different, you should assume that it will be bigger. Even after you sign up for the much more expensive premium tier, when you set-up the topic or queue, you’ll need to specify a maximum size. Images can be very big!

Alternatives

To be clear: the purpose of this post is to demonstrate that you can transmit an image via Service Bus – not that you should. There are other ways to do this: for example, you could upload the image to blob storage, or an S3 bucket, and then just point to that in the message. The one advantage transmitting the image with the message does give you, is that the binary data lives and dies with the message itself – so depending on what you do with the message after receipt, that may be an advantage.

Transmitting the Message

The following is a simple helper method that will send the message for us:

async Task SendMessage(string connectionString, string topicName, string messageText)
{    
    int byteCount = System.Text.ASCIIEncoding.ASCII.GetByteCount(messageText);

    await using var serviceBusClient = new ServiceBusClient(connectionString);
    var sender = serviceBusClient.CreateSender(topicName);
    
    var message = new ServiceBusMessage(Encoding.UTF8.GetBytes(messageText));

    await sender.SendMessageAsync(message);
}

byteCount tells you how big the message is going to be. Useful when you’re writing a blog post about such things, but you may find it useful for debugging.

The rest is pretty basic Azure Service Bus SDK stuff. I’ve written about this previously in more depth.

We can then combine this with the code from the last post:

        var serialisedToSend = SerialiseImageToBinary(
            @"c:\tmp\myimage.png");
        await SendLargeMessage(connectionString, "image-message", serialisedToSend);

The next step is to receive the message at the other side.

Consuming the Message

As with the send, we have a helper method here:

async Task<string> ConsumeNextMessage(string topic)
{
    var serviceBusClient = new ServiceBusClient(connectionString);
    var receiver = serviceBusClient.CreateReceiver(topic, "sub-1");
    var message = await receiver.ReceiveMessageAsync();
    return message.Body.ToString();
}

Again, I’ve written about receiving a message using the SDK before.

Here, we do the reverse of the send:

        var serialisedToReceive = await ConsumeNextMessage("image-message");

        DeserialiseImageFromBinary(
            serialisedToReceive,
            @$"c:\tmp\newimg{DateTime.Now.ToString("yy-MM-dd-HH-mm-ss")}.png");

I’ve used the date so that I could test this multiple times – other than that, we’re receiving the serialised image, and then writing it to disk.

Summary

I’ll re-iterate the sentiment that I’m not advocating this as a good or preferable way to transmit images, simply highlighting that it is possible with relatively little work.