by Contributed | May 24, 2022 | Technology
This article is contributed. See the original author and article here.
Azure Event Hubs enables you to stream millions of events per second from any source using Kafka, AMQP or HTTPS protocols. Using Event Hubs capture feature, you can load real-time streaming data to data lakes, warehouses, and other storage services, so that they can be processed or analyzed by analytics services.
Today we are excited to announce the preview of Apache Parquet capturing support in Azure Event Hubs.
Why Apache Parquet for big data analytics?
Apache Parquet is column-oriented storage format that is designed for efficient data storage and retrieval. It’s open source and is not tied to any processing framework, data model or programming language. Parquet is ideal for storing any kind of big data and is built to support efficient compression and encoding schemes.
Capture streaming data in Parquet format using Event Hubs
Using Azure Event Hubs, no code editor for event processing, you can automatically capture streaming data in an Azure Data Lake Storage Gen2 account in Parquet format. The no code editor allows you to easily develop an Azure Stream Analytics job without writing a single line of code.
Once data is captured, any analytics service of your choice can process or analyze Parquet data.
Get Started Today
by Contributed | May 23, 2022 | Technology
This article is contributed. See the original author and article here.
Data loss prevention is a major concern with all customers who would like to have granular control over their data and how it gets exported from their databases on Azure.
The steps below guide on how Outbound Firewall Rules can be leveraged to improve the security posture and ensure data gets exported only to approved Azure Storage accounts. All other Azure Storage accounts are treated as unapproved unless explicitly whitelisted.
The steps below use Azure APIs and Powershell cmdlets to implement the lockdown and enable OFRs.
Pre-requisites:
1. Valid Service Principal based Azure Active Directory (AAD) token for authentication of requests.
2. Latest version of Azure Powershell cmdlets
To enable Restrict Outbound Network Access and add/create OFRs using APIs
1. Check the current OFR configuration of the SQL Server using a GET request on
https://management.azure.com/subscriptions/{{subId}}/resourceGroups/{{sqlRg}}/providers/Microsoft.Sql/servers/{{sqlServer}}?api-version=2021-02-01-preview
where
{{subId}} = Subscription ID
{{sqlRg}} = Resource Group hosting the SQL server
{{sqlServer}} = name of the SQL server
It should show that the restrictOutboundNetworkAccess is disabled.
2. Create two storage accounts on Azure Storage. Example:
– auditallowstorage
– auditdenystorage
3. Export database to both storage accounts. The export should be successful for both accounts.
4. Issue a PUT request to Enable RestrictOutboundNetworkAccess on the SQL server using SQL API
https://management.azure.com/subscriptions/{{subId}}/resourceGroups/{{sqlRg}}/providers/Microsoft.Sql/servers/{{sqlServer}}?api-version=2021-02-01-preview
with JSON body as
{ "properties" :
{"restrictOutboundNetworkAccess": "Enabled"},
"location": "<sql_server_region>"
}
where
{{subId}} = Subscription ID
{{sqlRg}} = Resource Group hosting the SQL server
{{sqlServer}} = name of the SQL server
<server_region> = region where the SQL server is hosted
5. Verify that the restrictOutboundNetworkAccess property is now set to Enabled by issuing a GET request on
https://management.azure.com/subscriptions/{{subId}}/resourceGroups/{{sqlRg}}/providers/Microsoft.Sql/servers/{{sqlServer}}/outboundfirewallrules?api-version=2021-02-01-preview
where
{{subId}} = Subscription ID
{{sqlRg}} = Resource Group hosting the SQL server
{{sqlServer}} = name of the SQL server
It should show that the provisioned state is “ready” for restrictOutboundNetworkAccess
6. Ensure that there is no existing Outbound Firewall Rule in place using this Powershell command:
Get-AzSqlServerOutboundFirewallRule -ServerName <sql_server_name> -ResourceGroupName <resource_group_name>
where
<resource_group_name> = Resource Group hosting the SQL server
<sql_server_name> = name of the SQL server
7. Export database again to both of the storage accounts (auditallowstorage and auditdenystorage). This should fail.
8. Create OFR only for storage account auditallowstorage using this PUT request:
https://management.azure.com/subscriptions/{{subId}}/resourceGroups/{{sqlRg}}/providers/Microsoft.Sql/servers/{{sqlServer}}/outboundfirewallrules/{{saName}}.blob.core.windows.net?api-version=2021-02-01-preview
where
{{subId}} = Subscription ID
{{sqlRg}} = Resource Group hosting the SQL server
{{sqlServer}} = name of the SQL server
{{saName}} = Storage Account name for which OFR is created. In this case, its auditallowstorage
9. Verify that OFR was successfully created for storage account using this Powershell command:
Get-AzSqlServerOutboundFirewallRule -ServerName <sql_server_name> -ResourceGroupName <resource_group_name>
where
<resource_group_name> = Resource Group hosting the SQL server
<sql_server_name> = name of the SQL server
It should show the list of the allowed FQDN (Fully Qualified Domain Name). In this case, its auditallowstorage.
10. Export database to storage account auditallowstorage. This should be successful.
11. Export database to storage account auditdenystorage should still fail.
To disable Restrict Outbound Network Access and remove OFRs
1. Remove all Outbound Firewall Rules:
Remove-AzSqlServerOutboundFirewallRule -ServerName <sql_server_name> -ResourceGroupName <resource_group_name> -AllowedFQDN <sa_name>.blob.core.windows.net
where
<resource_group_name> = Resource Group hosting the SQL server
<sql_server_name> = name of the SQL server
<sa_name> = Storage Account Name
2. Issue a PUT request to disable RestrictOutboundNetworkAccess on the SQL server using SQL API
https://management.azure.com/subscriptions/{{subId}}/resourceGroups/{{sqlRg}}/providers/Microsoft.Sql/servers/{{sqlServer}}?api-version=2021-02-01-preview
with JSON body as
{ "properties" :
{"restrictOutboundNetworkAccess": "Disabled"},
"location": "<sql_server_region>"
}
where
{{subId}} = Subscription ID
{{sqlRg}} = Resource Group hosting the SQL server
{{sqlServer}} = name of the SQL server
<server_region> = region where the SQL server is hosted
3. Verify that the restrictOutboundNetworkAccess property is disabled on the SQL server by issuing the following GET request on
https://management.azure.com/subscriptions/{{subId}}/resourceGroups/{{sqlRg}}/providers/Microsoft.Sql/servers/{{sqlServer}}?api-version=2021-02-01-preview
where
{{subId}} = Subscription ID
{{sqlRg}} = Resource Group hosting the SQL server
{{sqlServer}} = name of the SQL server
To enable Restrict Outbound Network Access and add/create OFRs using Powershell
1. Execute the following command to enable restrictOutboundNetworkAccess property on the SQL server:
Set-AzSqlServer -ServerName <server_name> -ResourceGroupName <resource_group> -RestrictOutboundNetworkAccess "Enabled"
where
<server_name> = name of the SQL server
<resource_group> = name of the resource group
2. Check the current list of Outbound Firewall Rules on the SQL server:
Get-AzSqlServerOutboundFirewallRule -ServerName <server_name> -ResourceGroupName <resource_group>
where
<server_name> = name of the SQL server
<resource_group> = name of the resource group
3. Export database again to both of the storage accounts (auditallowstorage and auditdenystorage). This should fail.
4. Add a new Outbound Firewall Rule on the server using the command:
New-AzSqlServerOutboundFirewallRule -ServerName <server_name> -ResourceGroupName <resource_group> -AllowedFQDN <sa_name>.blob.core.windows.net
where
<resource_group> = Resource Group hosting the SQL server
<server_name> = name of the SQL server
<sa_name> = Storage Account Name
5. List the OFRs on the server using the following command:
Get-AzSqlServerOutboundFirewallRule -ServerName <server_name> -ResourceGroupName <resource_group>
6. Export database to storage account auditallowstorage. This should be successful
7. Export database to storage auditdenystorage should still fail.
Hope this was useful folks! Feel free to get in touch :)
by Contributed | May 20, 2022 | Technology
This article is contributed. See the original author and article here.
If you’re not already familiar with Mixed Reality Toolkit (MRTK), it’s an open-source project led by Microsoft that provides UX building blocks for MR and VR applications. The experiences you build with MRTK can run on any device that supports the OpenXR runtime such as HoloLens and Meta Quest. We’ve heard from the community that they love the richness of the MRTK UI controls and that it reduces development time, especially for apps that need to run on multiple platforms. Components for hand and eye tracking, inputs, solvers, diagnostic tools, scene management, and more can help you to build experiences that look great with less effort.
We’re excited to share the next release of this powerful toolkit, MRTK3 Public Preview, at Mixed Reality Dev Days on June 8-9. With MRTK3, you’ll have the option of a lighter-weight solution which allows you to select only the components of the toolkit you need. The release also includes a new interaction system, new theming and databinding features, Unity canvas support, and an updated design language that can help you refresh your app’s look and add polish. Additionally, native OpenXR support makes it even easier to target multiple devices such as HoloLens, Meta Quest, Windows Mixed Reality, and future OpenXR-supported devices.
Be the first to learn about MRTK3 at a free event, online or in-person
Join us June 8th and 9th via livestream or at the Microsoft Campus in Redmond, WA. Either way, you’ll learn about MRTK3 directly from the engineers who are building the latest features. Catch deep technical sessions, provide feedback to the team, and ask your questions live.
By attending in-person, you’ll have access to even more goodness.
- Network with the Microsoft team and other developers.
- Catch a fireside chat or panel discussion
- Get expanded session content covering:
- How to build applications with C# and OpenXR using StereoKit, a code-first, open-source library for cross-platform development.
- Introduction to Babylon.js and how easy it is to bring mixed reality to the web.
- Recently released HoloLens features like Moving Platform Mode
Participate in the online hackathon
Mixed Reality Dev Days also marks the kickoff of a month-long online hackathon where you can compete for prizes while getting hands on with MRTK3 public preview or StereoKit. Join a team or build a solo project with access to expert support.
Learn more about Mixed Reality Dev Days and sign up now.
We look forward to connecting with you soon!
Mixed Reality Dev Days Team
Recent Comments