by Contributed | May 25, 2021 | Technology
This article is contributed. See the original author and article here.
At Microsoft Build 2021, we are pleased to announce the availability of Azure Video Analyzer as an Azure Applied AI Service. Azure Video Analyzer (AVA) is an evolution of Live Video Analytics (LVA) that was launched several months ago. As an Azure Applied AI service, AVA provides a platform for solution builders to build human centric semi-autonomous business environments with video analytics. AVA enables businesses to reduce the cost of their business operations using existing video cameras that are already deployed in their business environments.
Many businesses take a manual approach to monitoring mundane business operations. This is prone to a higher degree of error and miss-rate due to human error and the fact that assessing the criticality of a visual event sometimes requires corelating multiple signals in real-time. AVA, in conjunction with other Azure services, addresses this challenge by providing a mechanism for businesses to automate mundane visual observations of operations and enable employees to focus on the critical tasks at hand.
AVA is a hybrid service spanning the edge and the cloud. The edge component of AVA is available via the Azure marketplace and is referred to as “Azure Video Analyzer Edge,” and it can be used on any X64 or ARM 64 Linux device. AVA Edge enables capturing live video from any RTSP enabled video sensor, analysis of live video using AI of choice, and publishing of results to edge or cloud applications. AVA Cloud Service is a managed cloud platform that offers REST APIs for secure management of AVA Edge module and video management functionality. This provides flexibility to build video analytics enabled IoT applications that can be operated locally from within a business environment as well as from a centralized remote corporate location.
AVA can be used with Microsoft provided AI services such as Cognitive Service Custom Vision and Spatial Analysis as well as AI services provided by companies such as Intel. It is also possible to build and integrate a custom AI service that incorporates open-source custom AI models. Business logic that corelates AI insights from AVA with signals from other IoT sensors can be integrated to drive custom business workflows.
AVA has been used by companies such as DOW Inc., Lufthansa CityLine, and Telstra to solve problems such as chemical leak detection, optimizing aircraft turn-around, and traffic analytics. AVA has enabled DOW get closer than ever to reaching their safety goal of zero safety-related incidents. In the case of Lufthansa CityLine, AVA has enabled human coordinators to make better data-driven decisions to reduce aircraft turnaround time, thus reducing costs and improving customer satisfaction significantly. Telstra has been able to unlock new 5G business opportunities with the combination of Azure Video Analyzer, Azure Percept, and Azure Stack Edge.
AVA offers all capabilities that were available in LVA and more. A short summary of some of the prominent capabilities are as follows:
Process live video at the Edge
AVA Edge can be deployed on any Azure IoT Edge enabled X64 or AMD64 Linux device. Live video from existing cameras can be directed to AVA and processed on the edge device within the business environment’s network. In turn, this provides organizations with a solution that considers limited bandwidth and potential internet connectivity issues, all while maintaining the privacy of their environments.
Analyze video with AI of choice
As mentioned earlier, AVA can be integrated with Microsoft provided AI or with AI that is custom built to solve a niche problem, such as Intel’s OpenVINO™ DL Streamer that provides capabilities such as detecting objects (a person, a vehicle, or a bike), object classification (vehicle attributions) and object tracking (person, vehicle and bike).
Azure AI is based on Microsoft’s Responsible AI Principles and the Transparency Note provides additional guidance on designing response AI integrations.
Flexible video workflows
In addition to the existing building blocks provided by LVA, the current release of AVA enables a variety of live video workflows via new building blocks such as Object Tracker, Line Crossing, Cognitive Services extension processor and Video Sink.
Simplify app development with easy-to-use widgets
AVA provides a video playback widget which simplifies app development with a secure video player and enables visualization of AI metadata overlaid on video.
Azure Video Analyzer offers all these capabilities and more. Take a look at Azure Video Analyzer product page and documentation page to learn more.
by Contributed | May 25, 2021 | Technology
This article is contributed. See the original author and article here.
Starting today, the Apache Spark 3.0 runtime is now available in Azure Synapse. This version builds on top of existing open source and Microsoft specific enhancements to include additional unique improvements listed below. The combination of these enhancements results in a significantly faster processing capability than the open-source Spark 3.0.2 and 2.4.
The public preview announced today starts with the foundation based on the open-source Apache Spark 3.0 branch with subsequent updates leading up to a Generally Available version derived from the latest 3.1 branch.
Performance Improvements
In large-scale distributed systems, performance is never far from the top of mind, “to do more with the same” or “to do the same with less” are always key measures. In addition to the Azure Synapse performance improvements announced recently, Spark 3 brings new enhancements and the opportunity for the performance engineering team to do even more great work.
Predicate Pushdown and more efficient Shuffle Management build on the common performance patterns/optimizations that are often included in releases. The Azure Synapse specific optimizations in these areas have been ported over to augment the enhancements that come with Spark 3.
Adaptive Query Execution (AQE)
There is an attribute of data processing jobs run by data-intensive platforms like Apache Spark that differentiates them from more traditional data processing systems like relational databases. It is the volume of data and subsequently the length of the job to process it. It’s not uncommon for queries/data processing steps to take hours or even days to run in Spark. This presents unique challenges and opportunities to take a different approach to optimize and access the data. Over several days the query plan shape can change as estimates of data volume, skew, cardinality, etc., are replaced with actual measurements.
Adaptive Query Execution (AQE) in Azure Synapse provides a framework for dynamic optimization that brings significant performance improvement to Spark workloads and gives valuable time back to data and performance engineering teams by automating manual tasks.
AQE assists with:
- Shuffle partition tuning: This is a major source of manual work data teams deal with today.
- Join strategy optimization: This requires human review today and deep knowledge of query optimization to tune the types of joins used based on actual rather than estimated data.
Dynamic Partition Pruning
One of the common optimizations in high-scale query processors is eliminating the reading of certain partitions, with the adage that the less you read, the faster you go. However, not all partition elimination can be done as part of query optimization; some require execution time optimization. This feature is so critical to the performance that we added a version of this to the Apache Spark 2.4 codebase used in Azure Synapse. This is also built into the Spark 3.0 runtime now available in Azure Synapse.
ANSI SQL
Over the last 25+ years, SQL has become and continues to be one of the de-facto languages for data processing; even when using languages such as Python, C#, R, Scala, these frequently just expose a SQL call interface or generate SQL code.
One of SQL’s challenges as a language, going back to its earliest days, has been the different implementations by different vendors being incompatible with each other (including Spark SQL). ANSI SQL is generally seen as the common definition across all implementations. Using ANSI SQL leads to supporting the least amount of rework and relearning; as part of Apache Spark 3, there has been a big push to improve the ANSI compatibility within Spark SQL.
With these changes in place in Azure Synapse, the majority of folks who are familiar with some variant of SQL will feel very comfortable and productive in the Spark 3 environment.
Pandas
While we tend to focus on high-scale algorithms and APIs when working on a platform like Apache Spark, it does not diminish the value of highly popular and heavily used local-only APIs like pandas. In fact, for some time, Spark has included support for User Defined Functions (UDF’s) which make it easier and more scalable to run these local only libraries rather than just running them in the driver process.
Given that ~70% of all API calls on Spark are Python, supporting the language APIs is critical to maximize existing skills. In Spark 3, the UDF capability has been upgraded to include a capability only available in newer versions of Python, type hints. When combined with a new UDF implementation, with support for new Pandas UDF APIs and types, this release supports existing skills in a more performant environment.
Accelerator aware scheduling
The sheer volume of data and the richness of required analysis have made ML a core workload for systems such as Apache Spark. While it has been possible to use GPUs together with Spark for some time, Spark 3 includes optimization in the scheduler, a core part of the system, brought in from the Hydrogen project to support more efficient use of (hardware) accelerators. For hardware-accelerated Spark workloads running in Azure Synapse, there has been deep collaboration with Nvidia to deliver specific optimizations on top of their hardware and some of their dedicated APIs for running GPUs in Spark.
Delta Lake
Delta Lake is one of the most popular projects that can be used to augment Apache Spark. Azure Synapse uses the Linux Foundation open-source implementation of Delta Lake. Unfortunately, when running on Spark 2.4, the highest version of Delta Lake that is supported is Delta Lake 0.6.1. By adding support for Spark 3, it means that newer versions of Delta Lake can be used with Azure Synapse. Currently, Azure Synapse is shipping with support for Linux Foundation Delta Lake 0.8.
The biggest enhancements in 0.8 versus 0.6.1 are primarily around the SQL language and some of the APIs. It is now possible to perform most DDL and DML operations without leaving the Spark SQL language/environment. In addition, there have been significant enhancements to the MERGE statement/API (one of the most powerful capabilities of Delta Lake) expanding scope and capability.
Get Started Today
Customers with *qualifying subscription types can now try the Apache Spark pool resources in Azure Synapse using free quantities until July 31st, 2021 (up to 120 free vCore-hours per month).
by Contributed | May 25, 2021 | Technology
This article is contributed. See the original author and article here.
Red Hat and Microsoft have collaborated to bring enterprise solutions to Java Enterprise Edition (EE) / Jakarta EE developers with solution templates on Azure Marketplace. Deploy Red Hat JBoss Enterprise Application Platform (EAP) on Azure Red Hat Enterprise Linux (RHEL) Virtual Machines (VM) and Virtual Machine Scale Sets (VMSS) if you are migrating away from proprietary application servers to a production supported open source application server or from on-premises to the cloud.
Red Hat and Microsoft
The Azure Marketplace offerings for JBoss EAP on RHEL is a joint solution from Red Hat and Microsoft. Red Hat is the world’s leading provider of enterprise open source solutions and a contributor for the Java standards, OpenJDK, MicroProfile, Jakarta EE, and Quarkus. JBoss EAP is a leading open source Java application server platform that is Java EE Certified and Jakarta EE Compliant in both Web Profile and Full Platform. Every JBoss EAP release is tested and supported on a variety of market-leading operating systems, Java Virtual Machines (JVMs), and database combinations. Microsoft Azure is a globally trusted cloud platform with a range of services from VMs on infrastructure as a service (IaaS) to platform as a service (PaaS). This joint solution by Red Hat and Microsoft includes integrated support and software licensing flexibility. Read the press release from Red Hat to learn more about the collaboration and JBoss EAP on Azure.
Why JBoss EAP and RHEL?
Customers heavily invested in Java EE / Jakarta EE who want to migrate to the cloud while preserving their investments with open-source solutions can utilize JBoss EAP on Azure RHEL VM/VMSS solutions. This reduces the time, complexity, and cost of migrating Java applications to Azure as it is fully supported and offers flexible subscription choices with Pay-As-You-Go (PAYG) and Bring-Your-Own-Subscription (BYOS) options. With the Red Hat Enterprise Linux (RHEL) PAYG option, your operating system can be more secure and up to date with Red Hat Update Infrastructure (RHUI) on Azure and can benefit from running older versions with the Extended Lifecycle Support (ELS) option.
Azure Marketplace Offerings
The Azure Marketplace solutions use the latest versions for RHEL, JBoss EAP, and OpenJDK for production deployments. JBoss EAP is offered only as BYOS, and you can select either BYOS or PAYG for RHEL. Once deployed, you can perform an upgrade by running the *yum update* command. These Marketplace solutions create the Azure compute resources to run JBoss EAP on RHEL. Solution configuration includes stand-alone and clustered mode on Azure VM and VMSS.
Support and subscriptions
Red Hat Enterprise Linux is available as on-demand PAYG or BYOS via the Red Hat Gold Image model using Red Hat Cloud Access. To use RHEL in the PAYG model, you will need an Azure Subscription. Red Hat JBoss EAP is available through BYOS only for now. Customers will need to supply their Red Hat Subscription Manager (RHSM) credentials along with RHSM Pool ID showing valid JBoss EAP entitlements when deploying this solution.
If you are a new JBoss EAP customer and don’t have a Red Hat subscription, create an account on the Red Hat Customer Portal and you can work directly with Red Hat to get set up. Red Hat provides a variety of flexible billing options.
Benefits of using Azure VMs and VMSS
With Azure VMs and VMSS, you get built-in identity with AAD, Role-Based Access Controls (RBAC), networking, data, storage, and security management. You can troubleshoot with Serial Console or enterprise support and have cloud spend transparency with Azure Cost Management.
In addition, JBoss EAP on VMSS allows automatic scaling of resources, up to 600 VMs. VMSS supports integration with a load balancer or Application Gateway. High availability and resiliency are available across single or multiple Data Centers. VM instance scaling can automatically increase or decrease in response to demand or a defined schedule that you can set after template deployment.
Customers will receive integrated support from Microsoft and Red Hat for any production issues with JBoss EAP on RHEL VM and VMSS solutions.
Migrating to JBoss EAP on Azure
The Red Hat Migration Toolkit for Applications (MTA) is a collection of tools that support large-scale Java application modernization and migration projects across a broad range of transformations and use cases. It is recommended to use MTA for planning and executing any JBoss EAP-related migration projects. It accelerates application code analysis, supports effort estimation, accelerates code migration, and helps you move applications to the cloud and containers. MTA allows you to migrate applications from other application servers to Red Hat JBoss EAP.
Image 1 – Red Hat Migration Toolkit for Applications Dashboard
Interested in Other Azure Hosting Options for Red Hat JBoss EAP?
JBoss EAP is also available on Azure Red Hat OpenShift (ARO) and Red Hat OpenShift Container Platform (for multi-cloud strategy) if you are looking for a container-based solution. For a managed hosting option, try JBoss EAP on Azure App Service (in preview). These services include integrated support where you can start your ticket with either Microsoft or Red Hat. So, the real question should be “How much control do you want or need?” Check out the flow chart and technology stack images below to help you identify the best-suited service for your JBoss EAP apps on Azure.
Image 2 – Migration Paths to Red Hat JBoss EAP on Azure
Image 3 – Comparison of Customer vs. Cloud Provider Responsibilities for JBoss EAP Hosting Options on Azure
Try it!
Here are great resources to help you get started.
by Contributed | May 25, 2021 | Technology
This article is contributed. See the original author and article here.
Corey Sanders, CVP of Microsoft Solutions, will be on camera in the session, “ATEBRK233 – Ask the Experts: Build consistent hybrid and multicloud applications with Azure Arc,” at Build 2021 on May 26th, at 10:30AM – 11:00AM PST. Don’t forget to sign up for Build and add this session to your schedule!
He will also be the featured speaker alongside several other leaders in the upcoming Hybrid Digital Event, held on June 29th, at 9:00AM – 11:00AM PST. Register for our upcoming Hybrid and Multicloud Digital Event on June 29th!
by Contributed | May 25, 2021 | Technology
This article is contributed. See the original author and article here.
We are thrilled to announce that Azure Kubernetes Service on Azure Stack HCI is now generally available. Over the last 8 months we have made 5 public preview releases as we worked closely with customers, and responded to the early feedback that they provided.
You can evaluate AKS-HCI by registering here: https://aka.ms/AKS-HCI-Evaluate.
AKS-HCI makes our popular Azure Kubernetes Service (AKS) available on-premises. It fulfills the key need for an on-premises App Platform in the Azure hybrid cloud stack that goes from bare metal all the way into Azure-connected experiences in the cloud.
AKS-HCI is a turn-key solution for Administrators to easily deploy, manage lifecycle of, and secure Kubernetes clusters in datacenters and edge locations, and developers to run and manage modern applications – all in an Azure-consistent manner. Complete end-to-end support and servicing from Microsoft – as a single vendor – makes this is a robust Kubernetes application platform that customers can trust with their production workloads.
AKS-HCI is an Azure service that is hybrid by design. It leverages our experience with AKS, follows the AKS design patterns and best-practices, and uses code directly from AKS. This means that you can use AKS-HCI to develop applications on AKS and deploy them unchanged on-premises. It also means that any skills that you learn with AKS on Azure Stack HCI are transferable to AKS as well. With Azure Arc capability built-in, you can manage your fleet of clusters centrally from Azure, deploy applications and apply configuration using GitOps-based configuration management, view and monitor your clusters using Azure Monitor for containers, enforce threat protection using Azure Defender for Kubernetes, apply policies using Azure Policy for Kubernetes, and run Azure services like Arc-enabled Data Services on premises.
No matter how you choose to deploy AKS-HCI – wizard-driven workflow in Windows Admin Center (WAC) or PowerShell – your cluster is ready to host workloads in less than an hour. Under the hood, the deployment takes care of everything that’s required to bring up Kubernetes and run applications. This includes core Kubernetes, container runtime, networking, storage, and security, and operators to manage underlying infrastructure. Scaling the cluster up or down by adding/removing nodes and cluster-updates/upgrades are equally quick and easy. So is ongoing local management through WAC or PowerShell.
AKS-HCI is the best platform for running .Net Core and Framework applications – whether your applications are based on Linux or Windows. The infrastructure required to run containers is included and fully supported. For Windows, AKS-HCI offers an industry-leading solution with advanced features like GMSA non-domain joined hosts, Active Directory integration, and WAC based application deployment, migration, and management. We want to ensure that AKS-HCI remains the best destination for Windows containers.
With Microsoft, security is not an afterthought. AKS-HCI is kept up to date just like any Azure service. Security updates for the entire solution including core Kubernetes platform, network and storge drivers, Linux and Windows container VM images, and other binaries are delivered by Microsoft. Hardened VM images, single sign on with Active Directory integration, encryption, certificate management, and integration with Azure Security Center are just a few features – from a long list – that demonstrate leadership in this space.
Customers are using AKS-HCI to run cloud-native workloads, modernize legacy Windows workloads, and/or Arc-enabled Data Services on-premises. As more and more Azure services become available to be run on-premises, AKS-HCI will continue to be the industry-leading and preferred destination.
Here’s how AKS-HCI is enabling digital transformation at one of our customers – SKF.
“At SKF we continue to execute our vision of digitally transforming the company’s backbone through harnessing the power of technology, interconnecting processes, streamlining operations, and delivering industry-leading digital products and services for our customers. Our focus is on digitalizing all segments of the chain and interconnecting them to unlock the full potential of digital ways of working for our business and customers. Our challenge in manufacturing area is that we have more than 100 factories, and we need to be able to provide them with an IT/OT platform that comes with speed, reliability, and low cost, while providing for critical production systems. SKF plan to standardize on Kubernetes as the primary hosting platform for modern workloads. AKS running both on Azure Cloud and Azure Stack HCI, and the fact that Microsoft has also chosen Kubernetes strategy running their other products, allow us to deploy for example Azure Arc enabled Data Services virtually on any of our new or existing environments. This gives us a tremendous jump start into Lean Digital Transformation of our factories worldwide.” — Sven Vollbehr, Head of Digital Manufacturing at SKF
We are honored and thrilled by the trust you have placed in us and are excited to deliver innovation that empowers you – starting with this GA milestone.
Finally, we would also like to take a moment to thank the entire AKS on Azure Stack HCI team, who have worked tirelessly on this project amid the very trying and tough circumstances of the last year. It has been great to partner with all of you and to get to know you all better.
Thanks!
Ben Armstrong, AKS on Azure Stack HCI Group Program Manager
Dinesh Kumar Govindasamy, AKS on Azure Stack HCI Engineering GC
Register for our upcoming Hybrid and Multicloud Digital Event on June 29th!
Learn more about Azure Kubernetes Service on Azure Stack HCI
- Watch our On-demand Build session: Azure Kubernetes Service on Azure Stack HCI (AKS-HCI): A turnkey App Platform for modern .NET apps – with demos featuring e2e deployment, Azure arc, Azure Monitor, Azure Policy, App modernization using WAC, GMSA for Windows Containers, Vision on Edge IoT app on AKS-HCI, and a customer story by SKF.
- Watch Skilling videos
Recent Comments