Azure Durable Functions – stateful functions in a serverless environment

As many of you probably know I was co-organizing ApexVox, a free .NET conference that happened in Cluj, Romania. One of the many great talks there was about Azure Durable Functions. I really enjoyed it and therefore I thought on sharing my thoughts on the topic since it is a very important one.

To set the scene and make sure we all start from the same point, regular Azure Functions is a serverless compute service that enables you to run code on-demand without having to explicitly provision or manage infrastructure. That is in itself something great since as a developer you can choose to run bunches of code out of nothing or, better said, without the need take care of anything but the algorithm or action that you want to execute. This type of service is particularly useful if you need to perform certain actions on clearly defined occasions. For instance when something is uploaded to a blob, when a message enters a queue and so on. Azure Functions come with a lot of built in triggers that can be used to trigger the execution of the needed code. 

There is however a problem with Azure Functions: they are totally stateless. This means that you cannot simply take the output from one function and pass it to the other. I’m sure most of the developers who’ll read this would agree with me that a certain kind of state is useful and needed. Of course, if you wanted to take outputs from one function and feed another one with that output you could do this by putting another queue, message bus or whatever in between. The first function would write something to that media while the second will use that as trigger.

Theoretically, everything is decoupled! An most of the time when we hear “decoupling” this is usually a good thing. My experience is that extremes are never good and Azure Functions take the decoupling concept to an extreme point. Imagine you would have tens or hundreds of different functions that needs to communicate through queues, messages or whatever other mechanism. This would be a total mess!

Azure Durable Functions to the rescue!

Azure Durable Functions are an extension of Azure Functions that lets you write stateful functions in a serverless environment. The extension manages state, checkpoints, and restarts for you. The extension lets you define stateful workflows using an orchestrator function, which can provide the following benefits:

  • You can define your workflows in code. No JSON schemas or designers are needed.
  • Other functions can be called both synchronously and asynchronously. Output from called functions can be saved to local variables.
  • Progress is automatically checkpointed when the function awaits. Local state is never lost when the process recycles or the VM reboots.

Useful application patterns

I’m sure everything sounds a little bit too abstract right now so let’s look into some common patterns where Azure Durable Function would really save the day.

1. Function chaining

In the function chaining pattern, a sequence of functions executes in a specific order. In this pattern, the output of one function is applied to the input of another function.

You can use Durable Functions to implement the function chaining pattern concisely as shown in the following example:

In this example, the values F1, F2, F3, and F4 are the names of other functions in the function app. You can implement control flow by using normal imperative coding constructs. Code executes from the top down. The code can involve existing language control flow semantics, like conditionals and loops. You can include error handling logic in try/catch/finally blocks.

2. Fan out/fan in

In the fan out/fan in pattern, you execute multiple functions in parallel and then wait for all functions to finish. Often, some aggregation work is done on the results that are returned from the functions.

A diagram of the fan out/fan pattern

With normal functions, you can fan out by having the function send multiple messages to a queue. Fanning back in is much more challenging. To fan in, in a normal function, you write code to track when the queue-triggered functions end, and then store function outputs. With Azure Durable Functions things get a lot easier. The fan-out work is distributed to multiple instances of the F2 function. The work can be tracked by using a dynamic list of tasks. We can then use the .NET Task.WhenAll API, to wait for all the called functions to finish. Then, the F2 function outputs are aggregated from the dynamic task list and passed to the F3 function. Fan-in can be done in a similar manner.

3. Async HTTP APIs

The async HTTP APIs pattern addresses the problem of coordinating the state of long-running operations with external clients. A common way to implement this pattern is by having an HTTP call trigger the long-running action. Then, redirect the client to a status endpoint that the client polls to learn when the operation is finished.

A diagram of the HTTP API pattern

Durable Functions provides built-in APIs that simplify the code you write to interact with long-running function executions. After an instance starts, the extension exposes webhook HTTP APIs that query the orchestrator function status.

There surely are several other patterns where Azure Durable Functions could prove useful but this article would become too long. I am thinking to write an entire series on this to look into the internals and better understand how everything works together, because it is fairly cool. So stay tuned.

How useful was this post?

Click on a star to rate it!

Average rating / 5. Vote count:

Dan Patrascu-Baba

Dan Patrascu-Baba

Developer, consultant, trainer at Codewrinkles
,Net Developer.Focusing on both the .Net world and Microsoft Azure. Experienced speaker and trainer.
Dan Patrascu-Baba
Spread the word!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.