This article is contributed. See the original author and article here.
Stream Analytics allows you to easily build a real-time analytics pipeline on Azure within minutes. Our customers love the zero-code integrations with other Azure services and the ability to easily write complex analytics logic using simple SQL language. We have heard it loud and clear that our customers want to secure their streaming jobs to securely connect to their resources on Azure.
Today, we are excited to announce the public preview of Stream Analytics cluster – a new single-tenant (Dedicated) SKU with support for Azure Virtual Network (VNet). This SKU is ideal for customers with medium to large streaming workloads and strong enterprise security requirements.
Stream Analytics cluster’s capacity is measured in Streaming Units (SUs) which represents the amount of CPU and memory resources allocated to your cluster. The cost and value of a Streaming Unit is the same across existing Standard and Dedicated SKUs. The minimum size of a cluster is 36 SUs and the same cluster can be shared by various Stream Analytics jobs running across your development, test and production subscriptions within your organization.
The core of this offering is the same engine that powers Stream Analytics jobs running in a multi-tenant environment. The single tenant, dedicated cluster provides the following benefits:
- Single tenant hosting with no noise from other tenants – your resources are truly “isolated” and perform better to handle any burst in traffic.
- Scale your cluster between 36 to 216 SUs as your streaming usage increases over time. Larger sizes will be offered in the future.
- VNet support that allows your Stream Analytics jobs to connect to your resources securely using private endpoints.
- Ability to author C# user-defined functions and custom deserializers in any region of your choice.
- Zero maintenance cost allowing you to focus your effort on building real time analytics solutions.
Network isolation behind the scenes
Stream Analytics provides a fully managed experience to connect ASA jobs to your resources using private link. creating a cluster, it takes 2 simple steps to securely connect streaming jobs to your resource (inputs/outputs):
- In your Stream Analytics cluster, create a private endpoint for input/output resource (e.g., Event Hubs)
- Go to the input/output resource (e.g., Event Hubs) and approve the private endpoint connection request from the cluster.
Stream Analytics clusters VNet support by taking care of all the heavy lifting behind the scenes. When you create a Stream Analytics cluster, all the necessary resources for you cluster are created inside a VNet that is dedicated to you. When you create a private endpoint in your Stream Analytics cluster, we create and manage that private endpoint resource to provide a seamless experience.
Managed Identity authentication for Event Hubs inputs and outputs
We are eager to solve the security needs of customers who may not have a large streaming workload to justify an entire Stream Analytics cluster. To address this, we will soon be rolling out support for Managed Identity authentication to Event Hub inputs/outputs. This will allow you to connect ASA jobs to Event Hubs that are behind a firewall or a VNet – as long as you have enabled “Allow trusted services” networking settings on the Event Hub side.
This does not provide blanket approval for any ASA job to access your Event Hubs as you must grant access to each of you ASA jobs individually thereby maintaining a secure link between the resources. This approach of using Managed Identity authentication along with enabling “allow trusted services” is already supported for Azure Blob stream inputs & outputs, Azure Blob reference data input, Azure Data Lake Storage Gen2 output and will be extended to support other
Stream Analytics clusters (preview) will be rolled out to West Central US by this week and quickly followed by all other regions worldwide. We hope you take full advantage of this functionality and are excited to see what you build with Stream Analytics.
The Azure Stream Analytics team is highly committed to listening to your feedback. We welcome you to join the conversation and make your voice heard via our UserVoice. You can stay up-to-date on the latest announcements by following us on Twitter @AzureStreaming. If you have any questions or run into any issues accessing the public preview, you can also reach out to us at askasa [at] microsoft [dot] com .
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.