This article is contributed. See the original author and article here.
For the recently released Azure Logic Apps (Preview) extension for Visual Studio Code, the Logic Apps team redesigned the Azure Logic Apps runtime architecture for portability and performance. This new design lets you locally build and run logic apps in Visual Studio Code and then deploy to various hosting environments such as Logic Apps Preview in Azure and Docker containers.
The redesigned Logic Apps runtime uses Azure Functions extensibility and is hosted as an extension on Azure Functions runtime, which means you can run logic apps anywhere that Azure Functions runs. You can host the Logic Apps runtime on almost any network topology that you want, choose any available compute size to handle the necessary workload that your workflow needs. Based on the resiliency that you want your app to have, you can also choose an Azure storage account type that best supports your app’s needs. For example, if you don’t need data replication for recovery or failover scenarios, you can use an LRS or ZRS storage account. If you need failover capability, you can use GRS or RA-GRS storage accounts instead. The new runtime also utilizes Azure Function’s request processing pipeline for request and webhook triggers as well as management APIs enabling you to leverage many of App Services http capabilities when deployed to Azure such as custom domains and private endpoints.
With this new approach, the Logic Apps runtime and your workflows become part of your app that you can package together. This capability lets you deploy and run your workflows by simply copying artifacts to the hosting environment and starting your app. This approach also provides a more standardized experience for building dev-ops pipelines around the workflow projects for running the required tests and validations before you deploy changes to production environments.
Azure Logic Apps on Azure Functions runtime
The artifacts for all the workflows in a specific workflow app are in a root project folder that contains a host configuration file, connection definition file, and one or more subfolders. Each subfolder contains the definition for a separate workflow. Here is a representation that shows the folder structure:
Workflow definition file
Each workflow in your project will have a workflow.json file in its own folder that is the entry point for your logic app’s workflow definition, which describes the workflow by using a Logic Apps domain specific language (DSL) that consists of these main parts:
- The trigger definition that describes the event that starts a workflow run.
- Action definitions that describe the subsequent sequence of steps that run when the trigger event happens.
During startup, the Azure Functions runtime loads the Logic Apps extension, which then enumerates through all folders that exist in your workflow project’s root folder and attempts to load any workflow definition (workflow.json) files before starting the Logic Apps runtime. When a workflow loads, the Logic Apps runtime performs the work that’s required to prepare for handling the events that the workflow trigger specifies, such as create an HTTP endpoint that can receive inbound requests for a workflow that starts with the Request trigger, or create an Azure Service Bus listener that receives Service Bus messages. If a workflow has syntactic or semantic errors, the workflow isn’t loaded, and the corresponding errors are output as error logs.
Connection definition file
Connectors provide quick access from Logic Apps to events, data, and actions across other apps, services, systems, protocols, and platforms. By using connectors in your logic apps, you expand the capabilities for your cloud and on-premises apps to perform tasks with the data that you create and already have. Azure Logic Apps connectors are powered by the connector infrastructure that runs in Azure. A workflow running on the new runtime can use these connectors by creating a connection, an Azure resource that provides access to these connectors. The connection definition (connections.json) file is located at the root folder of your project and contains the required configurations such as endpoint details and app settings that describe the authentication details for communicating with the connector endpoints. The Logic Apps runtime uses this information when running these connector actions.
Another capability in the redesigned Logic Apps runtime introduces is the extensibility to add built-in connectors. These built-in connectors are hosted in the same process as the Logic App runtime and provides higher throughput, low latency, and local connectivity. The connections definition file also contains the required configuration information for connecting through these built-in connectors. The preview release comes with the built-in connectors for Azure Service Bus, Azure Event Hub and SQL Server. The extensibility framework that these connectors are built on can be used to build custom connectors to any other service that you need.
Host configuration file
The host configuration (host.json) file contains runtime-specific configurations and is located in the root folder of the workflow app.
Workflow event handling
The Logic Apps runtime implements each action type in a workflow definition as a job that is run by the underlying Logic Apps job orchestration engine. For example, the HTTP action type is implemented as a job that evaluates the inputs to the HTTP action, sends the outbound request, handles the returned response, and records the result from the action.
Every workflow definition contains a sequence of actions that can be mapped to a directed acyclic graph (DAG) of jobs with various complexity. When a workflow is triggered, the Logic Apps runtime looks up the workflow definition and generates the corresponding jobs that are organized as a DAG, which Logic Apps calls a job-sequencer. Each workflow definition is compiled into a job-sequencer that orchestrates the running of jobs for that workflow definition. The job orchestration engine is a horizontally scalable distributed service that can run DAGs with large numbers of jobs.
For stateful workflows, the orchestration engine schedules the jobs in the sequencers by using Azure Storage queue messages. Behind the scenes, multiple dispatcher worker instances (job dispatchers), which can run on multiple compute nodes, monitor the Logic Apps job queues. By default, the orchestration engine is set up with a single message queue and a single partition on the configured storage account. However, the orchestration engine can also be configured with multiple job queues in multiple storage partitions. For stateless workflows, the orchestration engine keeps their states completely in memory.
A job can run one or multiple times. For example, an HTTP action job can finish after a single run, or the job might have to run several times, for example, when having to retry after getting an “HTTP 500 Internal Server Error” response. After each run, the action job’s state is saved in storage as a checkpoint, which is where the workflow’s actions inherit their “at least once” execution semantic. The job’s checkpoint state includes the action’s result along with the action’s inputs and outputs. Smaller-sized input and output values are stored inline as part of the job’s state in table storage, while larger-sized input and output values are saved in blob storage.
You can change the lifetime for a sequencer up to a 1-year maximum limit. When a sequencer reaches the end of life without finishing, the orchestration engine cancels any remaining jobs from the sequencer.
The redesigned Logic Apps runtime brings new capabilities that lets you use most of your dev-ops toolbox to build and deploy logic apps, and to set up your logic apps so that they run in different compute and network configurations. The Logic Apps runtime also brings a new extensibility framework that provides more flexibility for writing custom code and building your own custom connectors.
This blog post is the first in a series that the team plans to provide so that you can get in-depth views into various aspects of the redesigned runtime. Stay tuned for more posts about logging, built-in connector extensibility, and performance bench marking.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.