Name update for Agent and associated resources in Azure Monitor – Container Insights

by Contributed | Jul 19, 2022 | Technology

This article is contributed. See the original author and article here.

What is changing?  

When customers enable Container Insights, Azure Monitor deploys a containerized collection agent. This agent is being renamed from OMSAgent to Azure Monitor Agent. There are no feature updates or functional changes to the agent in this release.  

Why are the changes being made? 

The current OMSAgent name is a legacy name from the OMS product and does not reflect the branding for Azure Monitor and Container Insights products. The Azure Monitor Agent is being standardized as the single collection agent for Azure Monitor. The name change brings the agent’s name in line with these updates.  

When are the changes going live? 

The changes will roll out region by region starting from early September with AKS service update, expected to be these changes rolled out to all regions by the end of October. Exact dates and a version number of Container Insights image version with changes will be shared closer to the release date with an update to this blog post. 

AKS customers will receive the changes as part of an AKS service update. Arc and Hybrid customers will have to update the Extension and HELM chart versions manually. See detailed instructions below.  

What are the exact changes? 

The agent used by Container Insights has been renamed. So, customers will see a new pod name in their clusters. Alongside that, several other related resources have also been renamed. See the “List of renamed resources” and “List of renamed labels ” tables below for complete details.  

What is the impact? What do I need to do?  

Log analytics queries  

All log queries with string the omsagent in them will not work and results will only be returned for dates before the change. To mitigate, update the queries to have query clause to cover “omsagent” or “ama-logs” in existing queries. See examples below.  

Alert Rules 

Alert Rules with the string the omsagent will not work.  

1. Query alerts with the string “omsagent” 

1. Metric alerts which reference omsagent in a dimension   

You can find alert rules using  

Option 1 – In the Alerts section of Azure Monitor in the Azure portal

Option 2 – Programmatically using Azure Resource Graph queries  

Go to the “Azure Resource Graph Explorer” in the portal and run the below query:  
 
alertsmanagementresources 

| where properties contains “omsagent” 

The above steps should help you find the affected rules. To update the rules, simply edit them in the portal (as below).  

User scripts  

Any user scripts explicitly referencing the pod names will not work.  

Resource name in all UIs  

Customers will see the updated controller names in the Container Insights UI as well as the command line. 

Screenshot 1 – Container Insights Portal UI   

Screenshot 2 – Command Line 

Screenshot 3 – Workbooks    

Instructions for consuming the update 

Instructions for AKS customers  

No action needed from customers. Customers will receive this as part of an AKS service update.  

Instructions for Arc K8s customers  

Note: The extension version with the above changes is version 3.0.0 

1. Get the Azure Log Analytics Workspace Resource Id which configured on the existing Azure Monitor for containers extension  

az k8s-extension show –name azuremonitor-containers –cluster-name <cluster-name> –resource-group <resource-group> –cluster-type connectedClusters -n azuremonitor-containers 

1. Delete the existing Azure Monitor for Containers extension either through Azure Portal or Azure CLI 

1. Create Azure Monitor for containers extension either through Azure Portal or Azure CLI. Make sure to use the same Azure Log Analytics Workspace ResourceId which is obtained in Step #1, to ensure the data being ingested to the same Log analytics workspace as before  

Instructions for Hybrid customers  

Note: The Helm chart version with the above changes is version 3.0.0 

1. Get the Azure Log Analytics workspace which is being used for the Azure Monitor for containers HELM chart.  You can obtain through   

helm get values <azure-monitor-for-containers-release-name> -A 

1. Delete the Azure Monitor for containers HELM chart – https://docs.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-optout-hybrid#how-to-stop-monitoring-using-helm 

1. Install the HELM chart as per the Instructions –  https://docs.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-hybrid-setup#install-the-helm-chart . Make sure to same Azure Log Analytics Workspace to ensure the data being ingested to the same Log Analytics workspace as before. 

List of renamed resources 

Note: The xxxx below is a placeholder for random hash values appended to the resource name. In practice, a name like “ama-logs-xxxxx” will appear as “ama-logs-cfkst” or similar.   

List of renamed labels 

Microsoft Purview Information Protection now includes enhanced security for detection of credentials

by Contributed | Jul 18, 2022 | Technology

This article is contributed. See the original author and article here.

Hybrid work environments have introduced new vulnerable access points to organizations’ data and credentials, requiring improvements in credential security to help prevent the risk of cyber-attacks. In addition, the associated costs of security incidents that involve remote work are over $1 million more expensive on average than incidents that don’t involve remote work.¹ Sixty-one percent of data breaches involve credentials, making them the most compromised data type in breaches.² Cyber attackers often leverage compromised credentials to access personal data like medical history and banking information, which they can later sell on the “dark web.”

At Microsoft, our goal is to provide a built-in, intelligent, unified, and extensible solution to protect sensitive data across your digital estate – in Microsoft 365 cloud services, on-premises, third-party SaaS applications, and more. With Microsoft Purview Information Protection, we are building a unified set of capabilities for data classification, labeling, and protection not only in Office Apps, but also in other popular productivity services where information resides (e.g., SharePoint Online, Exchange Online, and Microsoft Teams), as well as endpoint devices.  

There are currently over 250 pre-built Sensitive Information Types available (e.g., PII identifiers, social security, credit card, bank account numbers, etc.). We are pleased to announce that we are now starting public preview* of 42 new Sensitive Information Types (SITs) enabling organizations to identify, classify, and protect credentials found in documents across OneDrive, SharePoint, Teams, Office Web Apps, Outlook, Exchange Online, Defender for Cloud Apps, and Windows devices. These credential SITs can be included in information protection auto-labeling and data loss prevention policies to help organizations discover a wide range of digital authentication credential types (aka “secrets”), such as user credentials (username and passwords), default passwords, and Azure cloud resources (e.g., Storage Account Keys, SQL Server Connection Strings, and SAS). Also included are new SITs for Amazon S3 Client Secret Access Key, X.509 Certificate Private Key, GitHub Personal Access Token, ASP.NET Machine Key, Slack Access Token, Google API, Ansible Vault, and more. Note that many of these SITs are credentials that provide access to cloud development and other resources, which have been the target of sophisticated attacks on DevOps pipelines within organizations.

List of all 42 new SITs:

New credential SITs key capabilities

• Within the Microsoft Purview compliance portal, these new credential SITs can be added to auto-labeling and DLP policies to quickly and accurately detect and classify complex digital authentication credentials


• System administrators can test the accuracy of individual SITs against sample data


• These new credential SITs will be visible in Content Explorer and Activity Explorer, enabling users to:
  
  
  
  • Locate documents that contain sensitive credentials within their environment
  
  
  • Identify activity involving the use of credential data within their environment
  
  
  
  

Public preview also includes:

*Note: Rollout has begun as of July 18th and is expected to be fully completed within a 24-hour window (July 19th).

**Note: Office client-side labeling is currently not supported, but it will be available sometime in CY22H2. Please stay tuned for additional updates on this capability. 

In-Product Screenshot(s)

Figure 1: Detection of general passwords using Credential SIT. Note that an E5 or A5 license is required for accessing Credential SITs, which will be in public preview within the next few weeks for commercial cloud customers and government clouds (GCC, GCC-High, Department of Defense).

Learn more about Microsoft Purview Information Protection and Credential SITs here. We are constantly extending our product capabilities to help organizations more easily classify and protect sensitive data.

Get Started 

We are happy to share that there is now an easier way for you to try Microsoft Purview solutions directly in the Microsoft Purview compliance portal with a trial. By enabling the trial in the Purview compliance portal, you can quickly access the new Credential SITs and Easy Trials, and start using all capabilities of Microsoft Purview, including Insider Risk Management, Records Management, Audit, eDiscovery, Communication Compliance, Information Protection, Data Lifecycle Management, Data Loss Prevention, and Compliance Manager. Visit your Microsoft Purview compliance portal for more details or check out the Microsoft Purview solutions trial (an active M365 E3 subscription is required as a prerequisite).

With Information Protection Easy Trials, users can apply default labels and get label recommendations on items containing sensitive data such as credit card numbers and activate features with a single click. System admins can review items containing credit card numbers and decide whether to automatically apply a label to them. Also, get further information on how to set up recommended information protection features and how to create auto-labeling policies.

We look forward to hearing your feedback! 

¹ IBM Security and Ponemon Institute, “Cost of a Data Breach Report 2021,” July 2021

² Verizon “2021 Data Breach Investigations Report”, May 2021

Lesson Learned #224:Hands-On-Labs: Checking the performance with Perf Collector Analyzer

by Contributed | Jul 16, 2022 | Technology

This article is contributed. See the original author and article here.

One of the main problem that we have when we are working with multiple databases in an Elastic Databases Pool or Azure SQL Managed Instace (that are running on same SQL Instance) is how to consolidate all the information in a single place. 

For example, if we use Query Data Store we need to review one by one the database finding the performance problem. Trying to resolve these insights we would like to suggest to use PerfCollector and PerfCollector Analyzer to allows you to capture and query the information collected for all the databases that you have in an Elastic Database Pool and SQL Managed Instance and even, all the databases associated in an Azure SQL Server or On-Premises. 

How to collect the data?

• PerfCollector PowerShell

Videos how to use? 

• Lesson Learned #220:Hands-On-Labs: Activity Monitor in my Elastic Database Pool – Microsoft Tech Community


• Lesson Learned #221:Hands-On-Labs: Activity Monitor in my Azure SQL Managed Instance – Microsoft Tech Community

Where download the PerfCollector Analyzer

• JMNetwalker/PerfCollector: Performance Collector Checker. (github.com)

Advantages to use PerfCollector Analyzer:

• Aggregate Query Data Store collected information for all databases that you have in Elastic Database Pool, Managed Instance,  Azure SQL or On-Premises


• Review best practices in performance:
  
  
  
  • Missing Indexes
  
  
  • Out-Of-Date Statistics
  
  
  
  


• Other topics.
  
  
  
  • Based on: Tune performance with the Query Store – SQL Server | Microsoft Docs
  
  
  
  

Some Example queries to obtain the information from Query Data Store for all the databases:

• Highest wait durations at instance level

SELECT TOP 10
    sum(total_query_wait_time_ms) AS sum_total_wait_ms, 
	 ws.[wait_category_desc]
FROM [_xTotalxAcummulatedx_xQDSx_query_store_wait_stats] ws
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_plan] p ON ws.plan_id = p.plan_id and ws.dbname = p.dbname
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_query] q ON p.query_id = q.query_id and p.dbname = q.dbname
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_query_text] qt ON q.query_text_id = qt.query_text_id and q.dbname = qt.dbname
GROUP BY ws.[wait_category_desc]
ORDER BY sum_total_wait_ms DESC;

• Highest wait durations per database at instance level/elastic database pool level

SELECT TOP 100
    sum(total_query_wait_time_ms) AS sum_total_wait_ms, 
	 ws.[wait_category_desc], 
	 q.dbname
FROM [_xTotalxAcummulatedx_xQDSx_query_store_wait_stats] ws
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_plan] p ON ws.plan_id = p.plan_id and ws.dbname = p.dbname
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_query] q ON p.query_id = q.query_id and p.dbname = q.dbname
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_query_text] qt ON q.query_text_id = qt.query_text_id and q.dbname = qt.dbname
GROUP BY q.dbname , ws.[wait_category_desc]
ORDER BY q.dbname, sum_total_wait_ms DESC;

• Highest wait durations per instance level/elastic database pool level and query

SELECT TOP 10 rs.avg_duration, qt.query_sql_text, q.query_id,
    qt.query_text_id, p.plan_id, GETUTCDATE() AS CurrentUTCTime,
    rs.last_execution_time, p.dbname
FROM [_xTotalxAcummulatedx_xQDSx_query_store_query_text] AS qt
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_query] AS q
    ON qt.query_text_id = q.query_text_id and qt.dbname = q.dbname
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_plan] AS p
    ON q.query_id = p.query_id and q.dbname = p.dbname
JOIN [_xTotalxAcummulatedx_xQDSx_query_store_runtime_stats] AS rs
    ON p.plan_id = rs.plan_id and p.dbname = rs.dbname
WHERE rs.last_execution_time > DATEADD(DAY, -10, GETUTCDATE())
ORDER BY rs.avg_duration DESC;

Video how to use it

Enjoy!

Introducing Azure Well-Architected Framework Assessments for Azure Stack Hub (Preview)

by Contributed | Jul 15, 2022 | Technology

This article is contributed. See the original author and article here.

Azure Stack Hub extends the capabilities of Azure to on-premises and edge locations, enabling you to deploy, manage, and operate a subset of Azure services from your own datacenter or remote locations. The local control plane and Azure consistent developer experience provided by Azure Stack Hub can be used to address challenging technical or regulatory requirements, such as low latency, data sovereignty and security or compliance requirements. For more information, please see Why use Azure Stack Hub?

The Azure Well-Architected Framework (WAF) provides prescriptive guidance and recommendations for architects to use when creating or reviewing cloud solutions. The WAF guidance is organized into five pillars, Reliability, Security, Cost Optimization, Operational Excellence, and Performance Efficiency. Incorporating the recommendations into workload designs helps to ensure reliable, scalable, and performant architecture patterns are implemented for cloud solutions.

Today we are announcing two pillars of the Well-Architected Framework are available in Preview for Azure Stack Hub on the Microsoft Assessment Platform. These are the Reliability and Operational Excellence pillars. If you are using Azure Stack Hub to deploy and operate workloads for key business systems, it is now possible to answers questions for these pillars within the assessments platform. After completing the assessments, you will be provided with a maturity or risk score, together with prescriptive guidance and knowledge links that suggest possible improvements you could make to your architecture design and score.

Reliability

The Reliability pillar of the WAF for Azure Stack Hub focuses on User workload solutions that have been deployed on an Azure Stack Hub scale unit, with questions probing architecture design areas such as high availability, disaster recovery, app and data resiliency capabilities, error handling, performance, and monitoring.

When designing applications using the guidance in the Well-Architected Framework Reliability pillar, the objective is not to avoid any and all failures – it is to respond to failure in a way that avoids downtime and data loss. This is achieved by increasing the resiliency of your applications using patterns that include data replication and application failover capabilities. These design principles are used to provide high availability and increase system uptime, which is a key success-criteria for business-critical systems.

Operational Excellence

The questions in the Excellence pillar for Azure Stack Hub, focus on the Cloud Operator Persona. As a Cloud Operator, you use the administrator portal and PowerShell for day-to-day management and operations of Azure Stack Hub, such as populating the Marketplace, defining quotas, plans and offers, monitoring infrastructure health, and applying platform updates.

The questions in the Operational Excellence review focus on identifying what processes and procedures Cloud Operators are using to effectively manage and operate the Azure Stack Hub scale unit. Question topics include, identity and access management, network integration, monitoring, and alerting, offering services, capacity management, updates, business continuity and disaster recovery (BC/DR) and effective support and administration.

Cloud Operators can use the Azure Well-Architected Review to complete a set of easy-to- questions in order to obtain their maturity score, together with a list of curated / individual recommendations and guidance for how they can improve the processes and procedures they use for operating and managing Azure Stack Hub. An example of the assessment in action is shown below:

We are working to bring the three remaining pillars of the Azure Well-Architected Framework (WAF) to the Assessments Platform for Azure Stack Hub in the coming months. If you have any feedback or comments in relation to the Azure Stack Hub WAF Assessments, please send an email to AzS-WAF-Feedback@microsoft.com.

To get started with the Azure Stack Hub Well-Architected Review Assessments today, please visit this link: https://aka.ms/architecture/review

About the Author 

Neil Bird is a Senior Program Manager in the Azure Edge & Platform Engineering team at Microsoft. His background is in Azure and Hybrid Cloud infrastructure, operational excellence, and automation. He is passionate about helping customers deploy and manage cloud solutions effectively using Azure and Azure Edge technologies.