Azure Archives - Page 106 of 121 - Dr. Ware Technology Services

Data Science Video Series to get started with Machine Learning on Azure

by Scott Muniz | Jul 14, 2020 | Azure, Microsoft, Technology, Uncategorized

This article is contributed. See the original author and article here.

Data science is about extracting knowledge from data. Data science is an important area of study because it is a tool that data scientists leverage to gain insights from data and prepare it for the machine learning modeling phase. By “doing data science”, data scientists actually apply techniques, such as data pre-processing and cleaning, feature engineering and descriptive statistics, to their data in order to understand it and start building AI solutions.

In this sense, data science has become an area of study that universities and companies should look at as a first step to start their machine learning journey:

To learn more about Machine Learning versus AI and Deep Learning, please visit: https://www.aka.ms/DLvsML

Sarah Guthals, PhD (@sarahguthals) and Francesca Lazzeri, PhD (@frlazzeri) have released “A Developer’s Introduction to Data Science”, a 28-part video series that focuses on how to use data science to build machine learning solutions: this series is now live on both Channel 9 and YouTube.

The content of this series is structured as follow:

Video Title	Video Description
Introduction to the Developer’s Intro to Data Science Video Series	In this 28-video series, you will learn important concepts and technologies to build your end-to-end machine learning applications on Azure. To learn more, check out: http://www.aka.ms/DevIntroDS_GitHub and http://www.aka.ms/DevIntroDS_Learn
What is the Data Science Lifecycle?	In this video you will learn what the Data Science Lifecycle is and how you can use it to design your data science solutions.
How do you define your business goal and scope your data science solution?	In this video, Sarah describes the problem that she is facing where she thinks data science methods might be able to help her improve her business goals.
What is Machine Learning?	What is Machine Learning? In this video you will learn what machine learning, supervised learning and unsupervised learning are and how you can use the model development cycle to build, train, test and deploy your machine learning models.
Which Machine Learning Algorithm Should You Use?	Which machine learning algorithm should I use? In this video you will learn how to select the right machine learning algorithm for your data science scenario and how to answer different questions with different machine learning approaches.
What is AutoML?	In this video, you will learn how you can use Automated machine learning (Automated ML) to accelerate the data science life cycle.
How do you create a machine learning resource in Azure?	In this video, you will learn how to create a Machine Learning resource inside of Azure. By using Azure for your machine learning toolset, you’re able to create the storage account, application insights, key vault and container registry (all resources that will support your machine learning work) in just a matter of minutes.
How do you setup your local environment for data exploration?	In this video, you will learn how to setup your local environment for data exploration. Specifically, you will setup Visual Studio Code to be able to run Python Jupyter Notebooks and connect to your Azure Machine Learning resource.
How do Jupyter notebooks work in Visual Studio Code?	In this video, you will get an introduction to how Jupyter Notebooks work inside of Visual Studio Code and install the Python packages useful for this data science project, and make sure you have access to the AzureML SDK.
How do you connect your Azure Machine Learning resources to your local Visual Studio Code environment?	In this video, you will learn how to connect the Machine Learning resource that you created in Azure to your local Visual Studio Code environment. This allows you to run your machine learning experiments on the cloud instead of locally.
How do you prepare your data for a time series forecast?	In this video you will learn how to prepare your data to be effectively run through machine learning algorithms. Then, you will learn how to upload your data from your local computer into your Azure Machine Learning resource (specifically the datastore resource) and how to
Why do you split data into testing and training data in data science?	In this video, you will learn why you split your data into training and testing data. Then you will learn how to actually split your data using a date into two different Pandas DataFrames using Python in Visual Studio Code.
What is an AutoML Config file?	In this video you will learn how to run a machine learning experiment with Automate ML and how to create your AutoMLConfig file to submit an automated ML experiment in Azure Machine Learning.
What should your parameters be when creating an AutoML Config file?	See how Sarah and Francesca configure and run an AutoMLConfig file to submit an automated ML experiment in Azure Machine Learning.
How do you create an AutoML Config file and run your data science experiments on the cloud?	In this video, you will actually put your data through AutoML in Azure to train and test with a number of machine learning algorithms that Azure supports.
What is Azure Machine Learning?	Azure Machine Learning is a cloud-based environment that you can use to train, deploy, automate, manage, and track your machine learning models.
How can you collaborate on Jupyter Notebooks using Azure Machine Learning studio?	In this video, you will see the Azure Machine Learning Studio and learn how create a Jupyter Notebook in the cloud. By doing this, you ensure you have access to your code anywhere.
How do you choose the best model and perform feature engineering?	In this video you will learn how to use Automated ML to select your best model and perform features engineering: Automated ML is the process of automating the time consuming, iterative tasks of machine learning model development and can help you optimize when developing end to end applications on Azure.
How do you use Azure ML for best model selection and featurization?	During training, the Azure Machine Learning service creates a number of in parallel pipelines that try different algorithms and parameters. When configuring your experiments, you can enable the advanced setting featurization, that can help you with automatic data cleansing, preparing, and transformation to generate synthetic features.
How do you evaluate and retrieve a time series forecast from Azure Machine Learning?	In this video, you will learn how to use an external python function to run your data through a forecast evaluation. Using Python files uploaded to the cloud environment within the Azure Machine Learning Studio, you can call functions within those files from the Jupyter Notebooks within the same cloud environment.
How do you score your machine learning model on accuracy?	In this video, you will use the root mean squared error, mean absolute error, and mean absolute percentage error to score the accuracy of your model. You will then learn how to visualize the productions of your model within the Jupyter Notebook within the Azure Machine Learning studio cloud environment using scatter plots.
How do you deploy a machine learning model as a web service within Azure?	In this video, you will gather all of the important pieces of your model to be able to deploy it as a web service on Azure so that your other applications can call it on the fly.
What have you learned from deploying a machine learning model as a web service?	In this video, Sarah summarizes all of the learnings from measuring the accuracy of the machine learning model used in this series. Sarah also revisits the business goal to determine whether the effort would actually provide valuable information for her business.
What is the importance of model deployment in machine learning?	In this video, Francesca summarizes the most important steps to deploy your machine learning models with Azure Machine Learning. Model deployment is the method by which you integrate a machine learning model into an existing production environment.
How do you select the right machine learning algorithm?	A common question in data science is “Which machine learning algorithm should I use?”. In this video you will learn how the algorithm you select depends primarily on two different aspects of your data science scenario: 1) What you want to do with your data? Specifically, what is the business question you want to answer by learning from your past data? 2) What are the requirements of your data science scenario? Specifically, what is the accuracy, training time, linearity, number of parameters, and number of features your solution supports?
How does ethics play a role in data science?	In this video, Sarah challenges you to think about where ethics plays a role in all data science problems. Regardless of the type of data analysis or machine learning model you are using, your questions, data, and parameters to your algorithms might introduce bias and actually cause harm.
What is model interpretability and how can you incorporate it into your data science solutions?	Interpretability is critical for data scientists, auditors, and business decision makers alike to ensure compliance with company policies, industry standards, and government regulations. In this video you will learn how to use the Model Interpretability toolkit to explain your models
Concluding the Developer’s Intro to Data Science Video Series	In this 28-video series, you learnt important concepts and technologies to build your end-to-end machine learning applications on Azure: Sarah and Francesca guided you through the data science process, from understanding your data, to applying machine learning algorithms and deploying your models on Azure.

Additional resources:

Video series GitHub repo: https://www.aka.ms/DevIntroDS_GitHub
The Data Science Lifecycle: https://www.aka.ms/DataScienceLifecycle
Algorithm Cheat Sheet: https://www.aka.ms/AlgorithmCheatSheet
Automated Machine Learning: https://www.aka.ms/AutomatedML
Azure Machine Learning service: https://www.aka.ms/AzureMLservice
Auto ML Featurization : https://www.aka.ms/AutoMLfeaturization
How to Select Machine Learning algorithms: https://www.aka.ms/SelectAlgos
Model Interpretability: https://www.aka.ms/AzureMLModelInterpretability

More videos coming:

The month of July 2020 is Data month for the Microsoft Reactors global live streams on Twitch! For the middle two weeks of July, you will get to dive even deeper on these and similar concepts with Sarah, Francesca, and other Cloud Advocates and Microsoft employees!

Azure Sentinel Machine Learning Behavior Analytics: Anomalous RDP Login Detection

by Scott Muniz | Jul 14, 2020 | Azure, Microsoft, Technology, Uncategorized

This article is contributed. See the original author and article here.

We are delighted to introduce the Public Preview for the Anomalous RDP Login Detection in Azure Sentinel’s latest machine learning (ML) Behavior Analytics offering. Azure Sentinel can apply machine learning to Windows Security Events data to identify anomalous Remote Desktop Protocol (RDP) login activity. Scenarios include:

Unusual IP – the IP address has rarely or never been seen in the last 30 days.
Unusual geolocation – the IP address, city, country, and ASN have rarely or never been seen in the last 30 days.
New user – a new user logs in from an IP address and geolocation, both or either of which were not expected to be seen based on data from the last 30 days.

Configure anomalous RDP login detection

You must be collecting RDP login data (Event ID 4624) through the Security events data connector. Make sure that in the connector’s configuration you have selected an event set besides “None” to stream into Azure Sentinel.

From the Azure Sentinel portal, click Analytics, and then click the Rule templates tab. Choose the (Preview) Anomalous RDP Login Detection rule, and move the Status slider to Enabled.

As the machine learning algorithm requires 30 days’ worth of data to build a baseline profile of user behavior, you must allow 30 days of Security events data to be collected before any incidents can be detected.

What’s New: Azure Sentinel Machine Learning Behavior Analytics: Anomalous RDP Login Detection

by Scott Muniz | Jul 14, 2020 | Azure, Microsoft, Technology, Uncategorized

This article is contributed. See the original author and article here.

Unusual IP – the IP address has rarely or never been seen in the last 30 days.
Unusual geolocation – the IP address, city, country, and ASN have rarely or never been seen in the last 30 days.
New user – a new user logs in from an IP address and geolocation, both or either of which were not expected to be seen based on data from the last 30 days.

Configure anomalous RDP login detection

You must be collecting RDP login data (Event ID 4624) through the Security events data connector. Make sure that in the connector’s configuration you have selected an event set besides “None” to stream into Azure Sentinel.

From the Azure Sentinel portal, click Analytics, and then click the Rule templates tab. Choose the (Preview) Anomalous RDP Login Detection rule, and move the Status slider to Enabled.

Azure Lab Services Enable remote hands-on learning – story from DeVry Univeristy

by Scott Muniz | Jul 14, 2020 | Azure, Microsoft, Technology, Uncategorized

This article is contributed. See the original author and article here.

Azure Lab Services allow you to Invite users to access lab resources right away. When they sign in, they’ll see a full list of VMs they can access across multiple labs. With one click they can connect and start working—no Azure subscription needed.

Immediate access to VMs for invited users, with no need to share your Azure subscription
Custom templates to quickly provision lab VMs and use repeatedly across labs
Scheduling feature to automatically shut down and start VMs and limit usage hours
Provisioning and scaling to hundreds of VMs with a single click—with the service managing all underlying infrastructure

Cost optimization and tracking

Manage your lab budget with usage control features. Schedule designated usage times or set up recurring auto-shutdowns and start times. Track individuals’ hourly usage or limit usage by setting up quotas.

Automatic management and scaling

As a managed service, Lab Services gives you automatic provisioning and management of your lab’s underlying infrastructure. Just prepare the right lab experience for your users and the service will handle the rest—rolling out and scaling your lab to hundreds of VMs with a single click.

Using Azure Database Migration Service hybrid mode (Preview)- Online migration

by Scott Muniz | Jul 14, 2020 | Azure, Microsoft, Technology, Uncategorized

This article is contributed. See the original author and article here.

Azure DMS Hybrid (Preview) is helpful when you want to migrate from On-Premise SQL server to Azure and don’t have site-to-site connectivity between the both, also, if there is limited site-to-site connectivity bandwidth. Azure DMS Hybrid (Preview) uses a migration worker which is hosted On-Premise together with an instance of Azure DMS running in the cloud. You can use Azure Database Migration Service hybrid mode to migrate data from an on-premises instance of SQL Server to Azure SQL Database

Note:- The Azure Database Migration Service hybrid installer runs on Microsoft Windows Server 2012 R2, Window Server 2016, Windows Server 2019, and Windows 10.

Note:- The Azure Database Migration Service hybrid installer requires .NET 4.7.2 or later. To find the latest versions of .NET, see the Download .NET Framework page.

In this article we will be using Azure DMS Hybrid (preview) for online database migration from On-Premise to Azure Managed Instance. Below are the steps to perform the mentioned operation:

Sign into Azure portal and go to the subscription where you want to deploy Database Migration Service.

Look for Resource providers, search for migration, and then register Microsoft.DataMigration.

In the global search, look for Azure Database Migration Services.

Give the required information and select service mode as Hybrid (Preview)

Review networking and tags. Once done, create Database Migration Services.

Go to the properties of Azure DMS and copy the resource ID. This will be needed later.

Now, we need to register the application in the Azure Active Directory. Registering the application means that your developers can use Azure AD to authenticate users and request access to user resources such as email, calendar, and documents.

Any member of your directory (not guests) can register an application, otherwise known as creating an application object.

Registering an application allows any user to do the following:

Get an identity for their application that Azure AD recognizes
Get one or more secrets/keys that the application can use to authenticate itself to AD
Brand the application in the Azure portal with a custom name, logo, etc.
Apply Azure AD authorization features to their app, including:
- Role-Based Access Control (RBAC)
- Azure Active Directory as oAuth authorization server (secure an API exposed by the application)
Declare required permissions necessary for the application to function as expected, including:
- App permissions (global administrators only). For example: Role membership in another Azure AD application or role membership relative to an Azure Resource, Resource Group, or Subscription
- Delegated permissions (any user). For example: Azure AD, Sign-in, and Read Profile

For knowing who can add application on to Azure AD instance please refer https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance

For registering app in Azure Active directory:- Go to Azure active directory> App registration > New Registration

In new registration provide display name (This can be changes later). Click register.

Go to DMS > Access control (IAM) > Add > Add role assignment > assign contributor role to your application ID (you can also create custom roles as well by following MSDN).
Note:- To add role assignments, you must have Microsoft.Authorization/roleAssignments/write and Microsoft.Authorization/roleAssignments/delete permissions, such as User Access Administrator or Owner

Once role is assigned, go to Hybrid blade under settings of Azure Database Migration Service. Download the installer folder by clicking on Installer download.

Unzip the downloaded folder and open dmsSettings.Json file in notepad.

In the dmsSettings file provide the Application ID of your registered application and resource ID of your DMS and leave rest of the settings as it is. Save the file.

Now we would need to generate certificate which Azure Database Migration Service will use to authenticate the communication from the hybrid worker. For generating certificate go the location of your recently downloaded folder and run the below command in admin cmd.

<drive>:<folder>Install>DMSWorkerBootstrap.exe -a GenerateCert

A certificate would get generated in the mentioned location.

14. Now upload the recently created certificate to Certificates & secrets of your application.

After uploading certificates create a new client secret. Please copy and save the value of Client Secret, we will need it later.

Now we need to install Azure Database Migration Service hybrid worker on your on-premise machine. For this go the location of your folder which we have recently downloaded and unzipped, and run below command in admin Command Prompt.

<drive>:<folder>Install>DMSWorkerBootstrap.exe -a Install -IAcceptDMSLicenseTerms -d

Once above command is successful you will see Database Migration Service status as Online.

In your subscription, assign contributor role to your application ID.

For this go to your subscription > Access Control (IAM) > Add
Select “Contributor” role from the drop down and Select “search for your Application ID”

Once the above operation is successful, we will start online Migration of the database. For this,
go to Azure Database Migration Service, on the overview blade, select New Migration Project.

Enter Project Name > choose type of activity > Online Data Migration. (Note:- This article is about online Migration)

Under select source > enter on-premises SQL server`s FQDN and credentials.

Under select target > provide the application ID and in the key column enter Client secret of your Application. Also, provide the target MI credentials.

Under Select Databases > select Source database which you wish to migrate to MI.

Before starting the migration, please make sure that On-premise SQL server service has required permission on the backup folder ( the folder where you have kept required backup of your database).

Go to your backup folder > right click – > Properties > Security > Edit > Add your On-Premise SQL Service and give read and write permission to it.

Note: – We also need give read/write permission to the windows account which will impersonate the DMS.

Under Configure migration settings > please provide the network share location of the backup folder and Azure storage location, where DMS will upload the file.

Note:- To get network share location go to your backup folder location > Right click- > Sharing > Share

Provide migration activity name and click on run migration.

Once migration has started check the status of the migration.

On the migration activity page click on the database name to see the status of the migration.
Once backup status shows restored, you can start cutover based upon your business requirements.