LockedUntilUtc change in Azure Service Bus

Recently I encountered a strange problem on a project using Service Bus. We were using message sessions and monitoring the BrokeredMessage.LockedUntilUtc property to check if we needed to renew the lock (in case the message processing takes longer than the lock duration).

One day, seemingly out of the blue, LockedUntilUtc started returning DateTime.MaxValue values which crashed our message handler because our monitoring code couldn’t handle it. We did not recently make any changes to the code, update the version running in production or change anything in the production environment. Even worse, we couldn’t replicate the issue in a new Service Bus namespace. So we were quite convinced that the issue couldn’t be with our code.

As it turns out, the issue was caused by the magical combination of developer error and a slowly rolling out change in Azure Service Bus.

First of all, our code for monitoring the lock duration was using the wrong property. For session scenarios, the lock duration is applied to the session, not to the individual messages. So instead of using the BrokeredMessage.LockedUntilUtc property, we should have been using the MessageSession.LockedUntilUtc property. Sessions are mainly used for message ordering, and only one receiver per session is allowed at a time. Therefor it makes sense that the lock duration is applied at the session level.

Secondly, the Service Bus team applied an update where the BrokeredMessage.LockedUntilUtc property for a session message will always return DateTime.MaxValue. Previously, LockedUntilUtc would return the session lock duration + one day for session messages.

This change was slowly rolling out to the different scale units, which is why a newly created Service Bus namespace did not (yet) show the issue.

I do think DateTime.MaxValue is a better value to return for a session message as it’s a better indicator of ‘you shouldn’t use this property in this scenario’ than lock duration + one day.

As a summary, see this example for the session message property values before and after the update:

Before update:
EnqueuedTimeUtc: 3/29/2018 9:26:44 PM
MessageSession.LockedUntilUtc: 3/29/2018 9:27:52 PM
BrokeredMessage.LockedUntilUtc: 3/30/2018 9:26:53 PM

After update:
EnqueuedTimeUtc: 5/18/2018 9:22:15 PM
MessageSession.LockedUntilUtc: 5/18/2018 9:29:16 PM
BrokeredMessage.LockedUntilUtc: 12/31/9999 11:59:59 PM

Service Fabric announcements at Build 2017

There were a lot of great Azure Service Fabric announcements at Build this year. Most importantly, you can now run Windows Server Containers on Service Fabric. Linux remains in public preview for now, but will be GA’ing soon in the next release. You can read all about these announcements over at the Azure Service Fabric Team blog.

There were two much smaller features that I especially liked though. Both are changes in the portal experience for creating new clusters.

Continue reading “Service Fabric announcements at Build 2017”

Avoiding data loss when using the Azure Event Hubs Processor Host SDK

I’m currently working on a project where we use Azure Event Hubs to collect telemetry data from connected field devices such as gas and electricity meters. This data gets processed in a variety of ways. We use Azure Stream Analytics to transform and forward the data to various subscribers. For more complex scenario’s where the functionality provided by Stream Analytics isn’t sufficient, we use the Event Processor Host SDK (hosted in a stateless Service Fabric service) to process the events.

Continue reading “Avoiding data loss when using the Azure Event Hubs Processor Host SDK”

DEVintersection Las Vegas 2016

I recently visited the DEVintersection conference in Las Vegas.
It’s a very nice conference with interesting workshops. I attended a two-day TypeScript / Angular2 workshop by John Papa and Dan Wahlin.

While there, Edwin van Wijk and I interviewed Scott Guthrie and Scott Hanselman for the dotnetFlix podcast.
You can view these episodes at the following links: