Azure Service bus || Peek/Delete scheduled message which has future enqueue time

by Scott Muniz | Sep 9, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

Use Case:

To peek/delete the scheduled messages from Azure service bus.

Pre-Requisites:

• Azure Service bus Namespace
• Azure Service bus SAS connection string
• Console Application to peek/delete the scheduled message

Scenarios:

• Query scheduled messages before the enqueue time.
• You can process the scheduled message with future enqueue time and can delete it post processing them.

Steps to follow:

You can schedule messages either by setting the ScheduledEnqueueTimeUtc property when sending a message through the regular send path, or explicitly with the ScheduleMessageAsync API. The latter immediately returns the scheduled message’s SequenceNumber, which you can later use to cancel the scheduled message.

Scheduled messages and their sequence numbers can also be discovered using message browsing.

Message browsing, or peeking, enables a Service Bus client to enumerate all messages that reside in a queue or subscription, typically for diagnostic and debugging purposes. The peek operations return all messages that exist in the queue or subscription message log, not only those available for immediate acquisition with Receive() or the OnMessage() loop. The State property of each message tells you whether the message is active (available to be received), deferred, or scheduled.

Pull messages from the service bus entity using the ‘Peek’ method, this method fetches all the active (available to be received) and scheduled messages (future Enqueue date) and if fetched message ScheduledEnqueueTimeUtc property is not null then that message is scheduled message.

Post processing the scheduled message if you want to delete it then please use the CancelScheduledMessageAsync method by passing the sequenceNumber. 

Refer the sample code to Peek the messages from the queue and then delete the Scheduled messages.

using System.Threading.Tasks;

using Microsoft.Azure.ServiceBus;

using Microsoft.Azure.ServiceBus.Core;

using System.Collections;

using System.Collections.Generic;

namespace ReceiverQueueScheduledMessage

{

    class Program

    {

        // Connection String for the namespace can be obtained from the Azure portal under the

        // ‘Shared Access policies’ section.

        const string ServiceBusConnectionString = “[Service bus connection string]”;

        const string QueueName = “[Queue Name]”;

        static IQueueClient queueClient;

        static IMessageReceiver messageReceiver;

        static void Main(string[] args)

        {

            MainAsync().GetAwaiter().GetResult();

        }

        static async Task MainAsync()

        {

            queueClient = new QueueClient(ServiceBusConnectionString, QueueName);

            Console.WriteLine(“======================================================”);

            Console.WriteLine(“Press ENTER key to exit after receiving all the messages.”);

            Console.WriteLine(“======================================================”);

                messageReceiver = new MessageReceiver(ServiceBusConnectionString, QueueName);

                Message message = await messageReceiver.PeekAsync();

                // Message with property ScheduledEnqueueTimeUtc not null means scheduled message     

                if (message != null && message.ScheduledEnqueueTimeUtc != null)

                {

                    await queueClient.CancelScheduledMessageAsync(message.SystemProperties.SequenceNumber);

                }

                 Console.ReadKey();

            await queueClient.CloseAsync();

        }

}

}

Pushed schedule messages to the queue, as shown in below screenshot (Queue name ‘samplequeue2’) has 2 scheduled messages:

Run the above code and Peek the message and check schedule message ‘ScheduledEnqueueTimeUtc’ property:

Once CancelScheduledMessageAsync method is called for the schedule message, schedule message will be deleted from the queue:

Running the above sample code, you should be able to peek/delete all the scheduled messages from Azure service bus.

Hope this helps!

Troubleshooting Windows 10 Update for Business by Using Azure Update Compliance and Log Analytics

by Scott Muniz | Sep 9, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

TROUBLESHOOTING WINDOWS 10 UPDATE for BUSINESS 

                                                           With AZURE UPDATE COMPLIANCE 

                                                                           & AZURE LOG ANALYTICS 

____________________________________________________________________________________________Cory Roberts and Tan Tran 

Dear IT Pros,  

Recently I and Cory Roberts, Microsoft Sr. CE, worked together on a Customer’s Project, We were upgrading roughly eight thousands Windows 10 devices from multiple versions of Windows 10, (1803, 1809, 1903, 1909) to the current branch 2004. The upgrade deployment has been proceeded with Microsoft Endpoint Manager. 

In Endpoint Manager, besides Device Status and End User Update Status, there was not much data provided by Endpoint Monitor or Log. It was hard to troubleshoot the windows 10 feature update process… We decided to go with Azure Update Compliance and Azure Log Analytics Query for monitoring and troubleshooting the Windows Feature Update deployment to match our Customer’s need. 

The steps to use Log Analytics for troubleshooting of Endpoint Manager Deployment on Windows 10 Feature Update as follow: 

1. In Endpoint Manager, create Windows 10 Feature Update Deployment and assign to the related Device Group. 
2. Create Log Analytics Workspace (if you do not have one). 
3. Install Update Compliance from Azure Market Place 
4. Onboarding Update Compliance for Windows 10 devices 
5. Set Windows 10 Clients to forward telemetry data to Log Analytics Workspace. 
6. Using Kusto Queries to Monitor and Troubleshoot the Upgrade Process. 

____________________________________________________________________________ 

1. In Endpoint Manager, create Windows 10 Feature Update Deployment and assign to the related Device Group 

– In Endpoint ManagerDevices,  

– Windows 10 Feature Update, Create Profile 

              – Choose the update to deploy   

             – Assign to Device Group and create the deployment.    

II. Create Log Analytics Workspace (if you do not have one). 

• In Azure Portal, search for log analytic workspace 

• Creating the Log Analytic Workspace: 

• Configure Resource Group and location for Log Analytics Workspace

• Click Create  

III. Install Azure Update Compliance from Market Place: 

Update Compliance uses Windows 10 diagnostic data for all of its reporting. It collects system data including update deployment progress, Windows Update for Business configuration data, and Delivery Optimization usage data, and then sends this data to a customer-owned Azure Log Analytics workspace to power the experience. 

• Update Compliance works only with desktops of Windows 10 Professional, Education, and Enterprise editions. It is not support for Windows Server, Surface Hub, IoT. 
• Update Compliance required windows 10 device telemetry at minimum basic level and a Commercial ID, a globally-unique identifier assigned to a specific Solution of Log Analytics workspace.  

• After Update Compliance is configured, it could take 48-72 hours before they first appear and continue refreshing its data every 12 hour 
• Update Compliance also provide Windows Update Delivery Optimization Status (WUDOAggregratedStatus, WUDOStatus), and Windows Defender Antivirus Threat and Update status (WDAV Threat, WDAVStatus) 

To Install Azure Update Compliance 

• Go to Azure search and type Update Compliance,  

• Choose MarketplaceUpdate Compliance 

• Choose the same LogAnalytics Workspace 

• Create 

Now, the Update Compliance Log will be available for Query search in Log Analytics Workspace as shown here: 

 To Configure GPO for Update Compliance Clients: 

• Go to Computer Configuration>Administrative Templates>Windows ComponentsData Collection and Preview Build 

• Choose “Allow Telemetry” and set level of diagnostic to at least basic level 
• Choose “Configure the Commercial ID” and copy and paste ID from WaaSUpdateInsight to the GPO setting box 

You could view the Commercial ID from the WaaSUpdateInsight as shown: 

• Choose “Allow device name to be sent in Windows diagnostic data” and Enabled 

IV. Onboarding Update Compliance for Windows 10 Devices. 

           The Update Compliance Configuration Script is the recommended method of configuring devices to send Telemetry data to Azure Log Analytics Workspace for use with Update Compliance. The script configures device policies via Group Policy, ensures that required services are running, and more. 

You can download the script here. 

The script is organized into two folders Pilot and Deployment. Both folders have the same key files: ConfigScript.ps1 and RunConfig.bat.  

You configure RunConfig.bat according to the directions in the .bat itself, which will then execute ConfigScript.ps1 with the parameters entered to RunConfig.bat. 

• The Pilot folder is more verbose and is intended to be use on an initial set of devices and for troubleshooting. Pilot script will collect and output detailed logs  
• The Deployment folder is intended to be deployed across an entire device population in a specific environment once devices in that environment have been validated with the Pilot script. 

• Configure commercialIDValue in RunConfig.bat to your CommercialID. 
• Use a management tool like Configuration Manager or Intune to broadly deploy the script to your entire target population. 

Steps to Deploy Update Compliance to Clients: 

• Edit PilotRunConfig.bat with Commercial ID of your WaaSInsight Solution and Location for log folder. 

• Run the PilotRunConfig.bat and generate errors in report from log folder 

• Review log files and correct the problems. 

• Edit DeploymentRunConfig.bat with Commercial ID of your WaaSInsight Solution and Location for log folder. 

• Run the DeploymentRunConfig.bat. Process may take more than 48hrs for collected data to show up in the Update Compliance Dashboard 

V. Set Windows 10 Clients Agent to forward data to Log Analytics Workspace. 

         Deploy Microsoft Monitoring Agents (MMA) as installation application to all Windows 10 Clients using SCCM. 

• Download MMASetup-AMD64.exe and use 7-zip to extract MOMAgent.msi from MMASetup-AMD64.exe 

• Create SCCM MMA application using the following command: 

msiexec /i MOMAgent.msi ADD_OPINSIGHTS_WORKSPACE=1 OPINSIGHTS_WORKSPACE_ID=Your-WorkspaceID OPINSIGHTS_WORKSPACE_KEY=Your-PrimaryKEY AcceptEndUserLicenseAgreement=1 /q 

• Deploy the MOMAgent Application to all Windows 10 SCCM Clients 

Configure Log Analytic to collect Windows 10 upgrade logs. 

Configure Log Analytics to collect the system event log and application event log together with the Windows update client event logs.  

To collect Event Viewer Log for Log Analytics Workspace: 

• Log Analytics WorkspaceAdvanced settings 

• Choose Data,  

• Choose Windows Event Logs 
• Type “Application” and click the + button 
• Type “System” and click the + button 
• Type “Microsoft-Windows-DeviceSetupManager/Admin” and click the + button 
• Type “Microsoft-WindowsUpdateClient/Operation” and click the + button 

• Save 

To collect Windows Upgrade logs for Log Analytics:   

There are 4 Windows 10 upgrade phases,  

• Downlevel phase: prepare upgrade installer source and destination in Windows 10 OS current version. 
• SafeOS phase, WinPE running phase, copying file for setup, prepare disk and file system table if needed, … 
• Firstboot phase, Windows system driver installation and reboot. 
• Secondboot phase, New version of Windows 10 OS is running, continue installing software applications and drivers. 

Depend on the Windows 10 upgrade phases the same upgrade log name could be in different Windows directory locations as shown here, The $WINDOWS~BT path is not working in Log Analytics service: 

•  Go to DataCustom log 
   

  

•  Click Add, and “Choose File” button to browse to the log directories specified in the above table. 

• Continue add all the logs and path as shown: 

• Enter Name of log collection CL, no space allowed in Name. 

• Done 

 You may get permission error, and you would need to “enable inheritance” permission as shown: 

VI. Using Kusto Queries to Monitor and Troubleshoot the Upgrade Process.  

All the search for upgrade status, update compliance status, Windows update delivery optimizaton information could be done by one tool, the Analytics Workspace Log Query as shown: 

To Review Update Log and search for errors: 

Run Log Analytics Query to search for update error in windows logs of devices: 

• In Azure Portal, Log Analytics Workspace 
• Logs, click on + to create new query 

• Choose the Custom Logs, double click to insert the related log to Query Windows 
• Run Query 

Query Custom Logs for All Upgrade errors:  

CompleteWindowsSetupLog_CL  

| where TimeGenerated <= ago(24m) 

| where RawData contains “error“ 

Query Custom Logs for Upgrade Device Driver Error 

PNPDeviceError_CL 

| where RawData contains “failure“ 

Query Custom Logs for Upgrade OOBE and other setup error: 

WindowsUpdatePhaseGC_CL  

| where RawData contains “error“ 

| where RawData contains “WimBoot“ or RawData contains “OOBE” or RawData contains “storage“ 

| project TimeGenerated, Computer, RawData, Type 

To Search Event Log for Update Errors: 

Query System Event for Update information: 

Event 

|where TimeGenerated > ago(1d) 

| where EventLog contains “system“ 

| where RenderedDescription contains “Update“ 

| project TimeGenerated, EventLog, Computer, EventID 

To Search Update Compliance Log for Ugrade Errors: 

Update Compliance Log provide the pre-built Desktop Analytics queries for all status of update included the following: 

• Deployment failures, 
• Reboot pending, 
• Feature or quality update deferral/pause, 
• Update automatic hold by Windows 10 Safeguard (to prevent hardware or software incompatibilities) 

In Log Analytics Workspace, there is very useful Update Compliance Logs’ tables related to WaaS and Windows Update Delivery Optimization as shown: 

The Update Compliance Logs Query might become our best option in troubleshooting Endpoint Manager Update Feature Deployment. 

• Query WaaS for Windows 10 Upgrade with “Not up-to-date” Status:  

WaaSUpdateStatus 

| where OSFeatureUpdateStatus contains “Not Up-to-date“ 

| project Computer, LastScan, OSName, OSVersion, FeatureDeferralDays, FeaturePauseState, NeedAttentionStatus 

• Query WaaS for Upgrade Deployment with error:  

WaaSDeploymentStatus 

| where DeploymentErrorCode != “0” 

• Query WaaS for Feature Update Deployment not successful, listed by Computer name, Last Scan time, Deployment Status, DetailedStatus… 

The DetailedStatus column may show recent feature of Windows 10 2004 with “Safeguard Hold” in the column, Safeguard hold was used to prevent imcompatible device hardware from being upgraded. 

       WaaSDeploymentStatus 

| where DetailedStatus != “UpdateSuccessful“ 

| where UpdateCategory == “Feature” 

| project Computer, LastScan, DeploymentStatus, DeploymentErrorCode, DetailedStatus  

• List Feature Update and Quality Update Status of a specific Computer:  

WaaSUpdateStatus  

| where Computer == “YourComputerName“ and TimeGenerated > ago(30d) 

| summarize arg_max(TimeGenerated,OSFeatureUpdateStatus, OSQualityUpdateStatus, NeedAttentionStatus, OSVersion) by Computer 

• Query WaaS for Upgrade Deployment with Failed Status and not contain a specific error code: 

       WaaSDeploymentStatus 

| where TimeGenerated > ago(7d) 

| where UpdateCategory == ‘Feature’ 

| where UpdateClassification == ‘Upgrade’ 

| where DeploymentStatus == ‘Failed‘ 

| where DeploymentErrorCode notcontains “8007001F” 

| where DeploymentError == “N/A” 

| where PauseState != “” 

Export result to csv file for later investigation of update failure’s root cause. 

After we get update error code from query result, then we will need to translate code error to meaningful root cause by using the error reference table from the following link: 

 https://docs.microsoft.com/en-us/windows/deployment/update/windows-update-error-reference 

Continue troubleshooting and testing the deployment of feature updates until we get no error in Log Analytics Query. 

You could generate an Intune’s update report-workbook as suggested by Jeff Gilbert blog 

References: 

Windows update logs file:  

https://docs.microsoft.com/en-us/windows/deployment/update/windows-update-logs 

Installing Log Analytics Agent for Windows Computers: 

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/agent-windows 

Update Compliance Setup and Log Analytics Queries: 

https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-get-started 

https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-configuration-manual 

https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-configuration-script 

https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-configuration-manual 

https://www.configjon.com/update-compliance-log-analytics-queries/ 

https://www.jeffgilb.com/update-compliance-with-intune/ 

Kusto Query tips and examples: 

https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/datetime-operations?toc=%2Fazure%2Fazure-monitor%2Ftoc.json#date-time-basics 

https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/get-started-portal 

https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/examples 

https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/samples 

I hope the information is useful for Windows Feature Update Troubleshooting.  

On my next blogpost, we will re-visit and discuss Update Compliance again. 

Cheers! 

____________________________________________________________________________________________________ 

Disclaimer 

The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages. 

What’s new: Azure DDoS Protection connector in Public Preview for Azure Sentinel

by Scott Muniz | Sep 8, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

This installment is part of a broader series to keep you up to date with the latest features in Azure Sentinel. The installments will be bite-sized to enable you to easily digest the new content.

Even more Azure Sentinel connector news for you! If you are using Azure DDoS Standard Protection, you can now ingest this via our connector into your Azure Sentinel workspace.

In addition to the core DDoS protection in the Azure platform, Azure DDoS Protection Standard provides advanced DDoS mitigation capabilities against network attacks. It’s automatically tuned to protect your specific Azure resources. Protection is simple to enable during the creation of new virtual networks. It can also be done after creation and requires no application or resource changes.

Connecting Azure DDoS Protection Standard logs to Azure Sentinel enables you to view and analyze this data in your workbooks, query it to create custom alerts, and incorporate it to improve your investigation process, giving you more insight into your platform security.

How to enable Azure DDoS Protection log ingestion in Azure Sentinel

Prerequisite – You must have a configured Azure DDoS Standard protection plan.

1. From the Azure Sentinel navigation menu, select Data connectors.

2. Select Azure DDoS Protection from the data connectors gallery, and then select Open Connector Page on the preview pane.

3. Enable Diagnostic logs on all the firewalls whose logs you wish to connect:

a. Select the Open Diagnostics settings > link and choose a Public IP Address resource from the list.

b. Select + Add diagnostic setting.

c. In the Diagnostics settings screen

• Enter a name in the Diagnostic setting name field.
• Mark the Send to Log Analytics check box. Two new fields will be displayed below it. Choose the relevant Subscription and Log Analytics Workspace (where Azure Sentinel resides).
• Mark the check boxes of the rule types whose logs you want to ingest. We recommend DDoSProtectionNotifications, DDoSMitigationFlowLogs, and DDoSMitigationReports.

d. Click Save at the top of the screen. Repeat this process for any additional firewalls (public IP addresses) for which you have enabled DDoS protection.

4. To use the relevant schema in Log Analytics for Azure DDoS Protection alerts, search for AzureDiagnostics. Here’s an example query below:

AzureDiagnostics 
| where ResourceType == "PUBLICIPADDRESSES"
| sort by TimeGenerated

And that’s it! You will now have Azure DDoS Standard logs connected to your Sentinel workspace.

Get Started Today!

Try out the new connector and let us know your feedback using any of the channels listed in the Resources.

You can also contribute new connectors, workbooks, analytics and more in Azure Sentinel. Get started now by joining the Azure Sentinel Threat Hunters GitHub community and follow the guidance.