by Contributed | Nov 25, 2023 | Technology
This article is contributed. See the original author and article here.
Introduction
A Practical Guide for Beginners: Azure OpenAI with JavaScript and TypeScript is an essential starting point for exploring Artificial Intelligence in the Azure cloud. This guide will be divided into 3 parts, covering: ‘How to create the Azure OpenAI Service resource,’ How to implement the model created in Azure OpenAI Studio, and finally, how to consume this resource in a Node.js/TypeScript application. This series will help you learn the fundamentals so that you can start developing your applications with Azure OpenAI Service. Whether you are a beginner or an experienced developer, discover how to create intelligent applications and unlock the potential of AI with ease.
Responsible AI
Before we start discussing Azure OpenAI Service, it’s crucial to talk about Microsoft’s strong commitment to the entire field of Artificial Intelligence. Microsoft is deeply dedicated to this topic. Therefore, Microsoft is committed to ensuring that AI is used in a responsible and ethical manner. Furthermore, Microsoft is working with the AI community to develop and share best practices and tools to help ensure that AI is used in a responsible and ethical way, thereby incorporating the six core principles, which are:
- Fairness
- Inclusivity
- Reliability and Safety
- Transparency
- Security and Privacy
- Accountability
If you want to learn more about Microsoft’s commitment to Responsible AI, you can access the link Microsoft AI Principles.
Now, we can proceed with the article!
Understand Azure OpenAI Service
Azure OpenAI Service provides access to advanced OpenAI language models such as GPT-4, GPT-3.5-Turbo, and Embeddings via a REST API. The GPT-4 and GPT-3.5-Turbo models are now available for general use, allowing adaptation for tasks such as content generation, summarization, semantic search, and natural language translation to code. Users can access the service through REST APIs, Python SDK, or Azure OpenAI Studio.
To learn more about the models available in Azure OpenAI Service, you can access them through the link Azure OpenAI Service models.
Create the Azure OpenAI Service Resource
The use of Azure OpenAI Service is limited. Therefore, it is necessary to request access to the service at Azure OpenAI Service. Once you have approval, you can start using and testing the service!
Once your access is approved, go to the Azure Portal and let’s create the Azure OpenAI resource. To do this, follow the steps below:
- Step 01: Click on the Create a resource button.
- Step 02: In the search box, type Azure OpenAI and then click Create.
- Step 03: On the resource creation screen, fill in the fields as follows:
Note that in the Pricing tier field, you can test Azure OpenAI Service for free but with some limitations. To access all features, you should choose a paid plan. For more pricing information, access the link Azure OpenAI Service pricing.
Step 04: Under the Network tab, choose the option: All networks, including the internet, can access this resource. and then click Next.
Step 05: After completing all the steps, click the Create button to create the resource.
- Step 06: Wait a few minutes for the resource to be created.
Next steps
In the next article, we will learn how to deploy a model on the Azure OpenAI Service. This model will allow us to consume the Azure OpenAI Service directly in our code.
Oh, I almost forgot to mention! Don’t forget to subscribe to my YouTube Channel! In 2023/2024, there will be many exciting new things on the channel!
Some of the upcoming content includes:
- Microsoft Learn Live Sessions
- Weekly Tutorials on Node.js, TypeScript, & JavaScript
- And much more!
If you enjoy this kind of content, be sure to subscribe and hit the notification bell to be notified when new videos are released. We already have an amazing new series coming up on the YouTube channel this week.
See you in the next article!
by Contributed | Nov 24, 2023 | Technology
This article is contributed. See the original author and article here.
In this blog series dedicated to Microsoft’s technical articles, we’ll highlight our MVPs’ favorite article along with their personal insights.
Jungsun Kim, Data Platform MVP, Korea
SQL Server technical documentation – SQL Server | Microsoft Learn
“This article introduces various DB technologies ranging from the latest version of SQL Server, Azure SQL, Business Intelligence, to Machine Learning.”
(In Korean: SQL Server 최신 버전, Azure SQL, Business Intelligence, Machine Learning에 이르기까지 다양한 DB기술을 소개합니다.)
*Relevant Activity: I have been writing a series of articles about new features of SQL Server 2022 on my website: 김정선의 Data 이야기 (visualdb.net)
Sergio Govoni, Data Platform MVP, Italy
SQL Server encryption – SQL Server | Microsoft Learn
“Databases in a company are the place where information is stored to drive the company’s production processes. Tera of data, dozens of databases, millions of rows, the entire activity depends on this, and information security can no longer be an option, we have to think about security by design and security by default.”
*Relevant Blog:
– Encryption in your SQL Server backup strategy! | by Sergio Govoni | CodeX | Medium
– Database encryption becomes transparent with SQL Server TDE! | by Sergio Govoni | CodeX | Medium
– Advanced database encryption with SQL Server Always Encrypted! | by Sergio Govoni | CodeX | Medium
– How to manage Always Encrypted columns from a Delphi application! | by Sergio Govoni | CodeX | Medium
James Hickey, Developer Technologies MVP, Canada
SQL Server and Azure SQL index architecture and design guide – SQL Server | Microsoft Learn
“If you work with SQL Server, it’s really helpful to understand advanced information about how indexing works, memory management, etc. You’ll find tips about creating better indexes & queries and how to debug issues better.”
Inhee LEE, Business Applications MVP, Korea
Create a machine ordering app with Power Apps – Online Workshop – Training | Microsoft Learn
“This is a practical course that enables you to understand the power platform comprehensively and organically. Through the practical course, you can learn each element systematically while covering various practical aspects, making it a very suitable course for beginners and intermediate learners. I recommend this course.”
(In Korean: 파워플랫폼에 대한 전체적인 이해를 종합적이고 유기적으로 가능하게 하는 실습과정입니다. 실습과정을 통해서 실무적인 요소를 두루 다루면서도 체계적으로 하나하나 익힐 수 있어서 초보자와 중급자 모두에게 매우 적절한 실습과정이어서 추천했습니다.)
*Relevant Activity: I shared this content with a group of Facebook: Power Platform | Facebook
by Contributed | Nov 23, 2023 | Technology
This article is contributed. See the original author and article here.
Description
Report CreateFile operations during Event Grid ingestion from Adlsv2 as errors
When uploading a file with Azure Data Lake SDK, there is an initial CreateFile event with size of 0.
Another FlushAndClose event is sent to Azure Data Explorer when setting the close parameter to ‘true’ on the SDK’s file upload option.
This event indicates that the final update has been made and the file stream is ready for ingestion.
Current Behavior:
Azure Data Explorer ignores the CreateFile event during data ingestion and no monitoring of these errors is reflected to users.
Expected behavior:
Azure Data Explorer will treat CreateFile events as an error of empty blob.
As a result, users are expected to get the following permanent errors for these events:
- Failure ingestion metrics:
- “IngestionResult” of permanent bad request
- “BlobsDropped” from the relevant event grid data connection.
- “EventsDropped” from the relevant event grid data connection.
- Failed ingestion logs in case of turning on the failed ingestion diagnostic logs
Required Change:
Filter out CreateFile events from the event grid subscription.
This filtering reduces the traffic coming from Event Grid and optimizes the ingestion of events into Azure Data Explorer.
You can read more about how to use the SDK correctly and avoid empty file errors here.
Schedule & plan
Step 1: Existing clusters which do not use the functionality today will get the change immediately.
Step 2: Clusters created after end of December 2023 will get the change.
Step 3: Current flow users as well as new clusters created until end of December 2023 will receive the changes after end of February 2024.
- Deprecating the metric “Events Processed (for Event/IoT Hubs)”
This metric represents the total number of events read from Event Hubs/ IoT hub and processed by the cluster. These events can be split by the status: Received, Rejected, Processed.
Required Change
Users can use the metrics “Event received”, “Events processed” and “Event dropped” to get the number of events that were received, processed, or dropped from each data connection respectively.
You can read more about these metrics here.
To ensure continuity of monitoring, please move any automated process that uses the old metric to use the new metrics instead.
Schedule & plan
The changes above are planned to take place at the end of February 2024.
by Contributed | Nov 22, 2023 | Technology
This article is contributed. See the original author and article here.
Author(s): Arun Sethia is a Program manager in Azure HDInsight Customer Success Engineering (CSE) team.
Co-Author: Sairam is a Product manager for Azure HDInsight on AKS.
Introduction
Azure Logic Apps allows you to create and run automated workflows with little to no code. These workflows can be stateful or stateless. Each workflow starts with a single trigger, after which you must add one or more actions. An Action specifies a task to perform. Trigger specifies the condition for running any further steps in that workflow, for example when a blob is added or updated, when http request is received, checks for new data in an SQL database table, etc. These workflows can be stateful or stateless, based on your Azure Logic App plan (Standard and Consumption).
Using workflows, you can orchestrate complex workflow with multiple processing steps, triggers, and interdependencies. These steps can involve certain Apache Spark and Apache Flink jobs, and integration with Azure services.
The blog is focused on how you can add an action to trigger Apache Spark or Apache Flink job on HDInsight on AKS from a workflow.
Azure Logic App – Orchestrate Apache Spark Job on HDInsight on AKS
In our previous blog, we discussed about different options to submit Apache Spark jobs to HDInsight on AKS cluster. The Azure Logic Apps workflow will make use of Livy Batch Job API to submit Apache Spark job.
The following diagram shows interaction between Azure Logic Apps, Apache Spark cluster on HDInsight on AKS, Azure Active Directory and Azure Key Vault. You can always use the other cluster shapes like Apache Flink or Trino for the same, with the Azure management endpoints.
HDInsight on AKS allows you to access Apache Spark Livy REST APIs using OAuth token. It would require a Microsoft Entra service principal and Grant access to the cluster for the same service principal to the HDInsight on AKS cluster (RBAC support is coming soon). The client id (appId) and secret (password) of this principal can be stored in Azure Key Vault (you can use various design pattern’s to rotate secrets).
Based on your business scenario, you can start (trigger) your workflow; in this example we are using “Http request is received.” The workflow connects to Key Vault using System managed (or you can use User Managed identities) to retrieve secrets and client id for a service principal created to access HDInsight on AKS cluster. The workflow retrieves OAuth token using client credential (secret, client id, and scope as https://hilo.azurehdinsight.net/.default).
The invocation to the Apache Spark Livy REST APIs on HDInsight is done with Bearer token and Livy Batch (POST /batches) payload.
The final workflow is as follows, the source code and sample payload are available on this GitHub
Azure Logic App – Orchestrate Apache Flink Job on HDInsight on AKS
HDInsight on AKS provides user friendly ARM Rest APIs to submit and manage Flink jobs. Users can submit Apache Flink jobs from any Azure service using these Rest APIs. Using ARM REST API, you can orchestrate the data pipeline with Azure Data Factory Managed Airflow. Similarly, you can use Azure Logic Apps workflow to manage complex business workflow.
The following diagram shows interaction between Azure Logic Apps, Apache Flink cluster on HDInsight on AKS, Azure Active Directory and Azure Key Vault.
To invoke ARM REST APIs, we would require a Microsoft Entra service principal and configure its access to specific Apache Flink cluster on HDInsight on AKS with Contributor role. (resource id can be retrieved from the portal, go to cluster page, click on JSON view, value for “id” is resource id).
az ad sp create-for-rbac -n --role Contributor --scopes
The client id (appId) and secret (password) of this principal can be stored in Azure Key Vault (you can use various design pattern’s to rotate secrets).
The workflow connects to Key Vault using System managed (or you can use User Managed identities) to retrieve secrets and client id for a service principal created to access HDInsight on AKS cluster. The workflow retrieves OAuth token using client credential (secret, client id, and scope as https://management.azure.com/.default).
The final workflow is as follows, the source code and sample payload is available on GitHub
Summary
HDInsight on AKS REST APIs lets you automate, orchestrate, schedule and allows you to monitor workflows with your choice of framework. Such automation reduces complexity, reduces development cycles and completes tasks with fewer errors.
You can choose what works best for your organization, let us know your feedback or any other integration from Azure services to automate and orchestrate your workload on HDInsight on AKS.
References
We are super excited to get you started:
by Contributed | Nov 21, 2023 | Technology
This article is contributed. See the original author and article here.
The Viva Engage Festival, hosted by Swoop Analytics, is an interactive virtual event that brings together Viva Engage thought leaders, communication innovators, and community enthusiasts from around the globe. This is not just another webinar; it’s an opportunity to dive deep into the future of employee engagement, learn about new tech, explore the latest Viva Engage experiences, and connect with a community passionate about driving change in their businesses.
Hear from leading customers and directly from Microsoft
Viva Engage Festival includes customer speakers and industry experts who will share knowledge and expertise on a wide range of topics around Viva Engage, from Comcast, NSW Government, Johnson and Johnson, Vestas and more. Join us for an exclusive look into Microsoft’s journey with Viva Engage and communities as we share our own experiences.
We hope you join us to connect with like-minded individuals who share a passion for driving meaningful engagement. Whether you’re a business leader, a professional, or an enthusiast, you’ll leave the festival with the inspiration and knowledge needed to take your Viva Engage investments to the next level.
Nominate Viva Engage Community Champion!
As part of our 2023 Viva Engage Festival, Microsoft and SWOOP Analytics will announce this year’s regional winners of the Community Champion Award. The Viva Engage Community Champion Award is an opportunity to recognize passionate community managers around the world who are committed to employee engagement, knowledge sharing, and collaboration in their Viva Engage networks. Can you think of anyone who deserves this title? Let us know who it might be! The 2023 Viva Engage Community Champion will be announced for each region during the festival. Nominations close November 30, 2023.
Hope to see you there!
Don’t miss this opportunity to be part of a global community that is shaping the way we connect and collaborate. Register now, mark your calendar, and get ready to unlock the doors to a new era of engagement!
Recent Comments