This article is contributed. See the original author and article here.
While extensive, the Ninja training has to follow a script and cannot expand on every topic. Like any training, you may have questions after the session. This live blog post tries to address that by providing answers to common questions ordered by the Ninja training modules.
Let go!
Module 1: Get started with Azure Sentinel
Q: How do I do a free of charge trial for Azure Sentinel?
There is no straight forward free trial for Sentinel:
- Every new workspace is not billed for *Azure Sentinel* for a month.
- However, the Azure Sentinel cost is made of the Azure Sentinel cost and the Log Analytics cost, and there is *no free trial for Log Analytics*.
There is, however, some usage that is always free, and you try to limit yourself to those to have a free POC:
- Log Analytics is free for the first 5GB for each month, across an *account*
- Both Log Analytics and Sentinel are free when Sentinel is deployed for selected sources such as Office 365.
So, how do I run a free PoC? Either of those:
- Using free sources only.
- On top of an existing, already paid for Log Analytics data. Giving 30 days of free Sentinel ingestion.
- A dedicated Azure tenant unrelated to the EA gives 30 days of free Sentinel ingestion and 5GB/m free Log Analytics ingestion. The 30 days can be restarted by creating a new workspace.
Q. How can I send sample data?
For CEF (CommonEventLog) events stored in a file, you can use Logstash to read data from your CEF sample log file and send it directly into the Log Forwarder.
{
input {
file {
path => “/home/stefan/samplelogs/cef.log”
start_position => “beginning”
sincedb_path => “/dev/null”
}
output {
# change to your log forwarder host and port
tcp {
host => “127.0.0.1”
port => 514
}
}
Q: How can I have a direct link to the Azure Sentinel overview page? Any other page?
You don’t need to get to Azure Sentinel through the Azure Portal every time. Just bookmark any page (or copy the URL) and use it to access your favorite starting point. The URL will have the following format, with the blade number changing based on the specific page you wanted to start with:
Pricing and billing
Use the Azure portal cost management screen. Filter by the scope relevant to you (the workspace or resource group, for example).
The usage information is available in the workspace, and you can use these queries to report or as a starting point for your reporting. The usage reporting workbooks for Azure Sentinel uses this information to provide a comprehensive view of usage.
Our official pricing is to charge for retention beyond 90 days for sources ingested for free. However, you may find that in some cases, we do not actually charge. While we may start charging for such retention in the future, we will not charge for past charges not collected.
It does for logs collected from VMs using the agent or the Log Forwarder, but not for service to service connectors or Azure diagnostics logs.
Q: When I enable Azure Sentinel on an existing Log Analytics workspace, how does pricing change?
- There will be an additional cost for Sentinel applied to all data in the workspace.
- All data will be retained for 90 days with no additional charge. Additional retention remains at the current Log Analytics rate.
- The sources free for Sentinel ingestion will not be charged for ingestion (Log Analytics or Sentinel tier). Retaining this data beyond 90 days costs at the Log Analytics retention price.
Q: Can Azure Sentinel capacity reservations be reserved for 1 year, 3 years?
No. Azure Sentinel capacity reservations are different from Azure reserved instances and behave like standard Azure meters, billed daily. They differ from pay-as-you-go pricing as they offer a lower per-unit price for reserving a larger amount of units.
Q: Why is the pricing calculator using different capacity reservations for Log Analytics and Azure Sentinel?
Module 2: How is Azure Sentinel used?
Azure Sentinel as part of the Microsoft Security stack
Q: On a Windows system with Defender for endpoints already installed, would you install the Log Analytics agent to report Security Events to Azure Sentinel as well?
In general, the answer is yes, but it would depend on the use cases. Windows events are wide in scope but broadly fall into two groups:
- Activity (such as process, file, and network activity) that overlap with MDATP.
- Management audit (for example, user management) is not in the MDATP domain.
Other event sources such as SQL use the Windows Event Log and are not covered by MDATP.
Q: How does Azure Sentinel compare with the Graph Security API?
The main difference is that the Graph Security API does not support raw telemetry, which is the bread and butter of Azure Sentinel. The Sentinel connectors focus on getting raw telemetry. There are exceptions in areas we need to improve the cross-utilization, and we are working on that.
Side by side with your existing SIEM
Q: How do I forward alerts from Azure Sentinel to another system?
See the Ninja training side-by-side section.
Q: How do I forward data, alerts, or events from my current SIEM to Azure Sentinel?
The most common way would be to use Syslog or CEF, which most SIEM products support. Note that you would like to forward from the 3rd party SIEM collector layer in many cases, which is more efficient than overloading the 3rd party SIEM processing layer.
The following links can get you started:
- Splunk:
- Using the app: Send data and notable events from Splunk to Azure Sentinel using the Azure Sentinel Splunk App.
- Using CEF.
- ArcSight (CEF),
- Kiwi Syslog Server (Syslog),
- SolarWinds (Syslog)
Q: Ticket System Integration? Is it ServiceNow only?
While ServiceNow is the most popular ticketing system and many of our examples are focused on it, Logic Apps, on which the integration is based, has connectors with other ticketing systems:
If not available, you can still connect to your ticketing systems using a custom Logic App connector, the HTTP connector that supports most APIs, or an Azure function from Logic Apps.
Q: How do I forward events from Azure Sentinel to another SIEM?
We do not recommend forwarding all events from Azure Sentinel to your on-prem SIEM. It may imply you are not getting enough value from Azure Sentinel and worth looking into.
In case you want to forward events (all of some), export from Azure Sentinel / Log Analytics to Azure Storage and Event Hub or move Logs to Long-Term Storage using Logic Apps.
Module 3: Workspace and tenant architecture
Q: Best practice is to minimize the number of workspaces, but I want to split the bill. How do I do that?
Read how to report on the ingestion volume per computer, resource, resource group of subscription.
Q: Are the best practices for Log Analytics and Azure Sentinel concerning workspace architecture the same?
Not always. Log Analytics and Azure Sentinel have different use cases and users, which sometimes require a different approach. If Azure Sentinel uses a workspace, use the Azure Sentinel best practices. Also, try to minimize the amount of data not relevant to Azure Sentinel in the workspace to avoid unnecessary costs.
As a reference, you can find the Log Analytics multi-workspace best practices here:
Module 4: Collecting events
General
Log Forwarder
Note that the Log Forwarder is based on the Linux based Log Analytics Agent (MMA), so the questions in the next section, as far as they pertain to the Linux MMA, are relevant for the Log Forwarder as well.
Q: How do I set the Log Forwarder to listen to encrypted Syslog
Configure the Syslog server part of the Log Forwarder (rsyslog or Syslog-NG) to listen to TLS based Syslog:
Q: Can I filter Syslog of CEF events?
Yes, See the Log Forwarder webinar: YouTube, MP4, Deck.
Q: Should I filter firewall events?
Unlike windows events, Firewall events are simple and of only a handful of types. The most common event types (using Palo Alto’s terminology) are:
- Traffic events – any connection through the Firewall.
- Threat events – any URL accessed through the Firewall (the name is misleading here)
Both have significant value for your security but have a large volume and therefore cost. Preferably, all should be collected. Inbound failures are candidates for filtering out, as they include a huge volume of low quality attack attempts.
Q: What size VM should I use for the Log Forwarder?
The Log Forwarder does little itself as parsing is done in the cloud. Therefore, comparatively, smaller and cheaper systems can be used.
Recent reports from customers have suggested:
- 500 GB/d of CEF data using a three VM scale set of Standard_D4s_v3 (4 CPU, 16GB) VMs.
- 6000 EPS of CEF data using a single physical VM: 8 vCPUs, 16 GB memory, Intel Xeon Platinum 8171M CPU @ 2.60GHz.
Use a VM scale set with an Azure load balancer or an on-prem load balanced to go beyond.
Log Analytics Agent and Azure Monitor Agent
Q: Is the workspace key stored on the agent machine?
We don’t store the workspace key. It’s only used during onboarding to generate the certs used for on-going communications by the Agent. The Workspace ID is stored in a config file per workspace here: /etc/opt/microsoft/omsanget/ws-id.
Q: Can Azure Sentinel filter Windows Events?
The Log Analytics agent (MMA) offers limited control over the Windows events forwarded. You can set a collection tier for all agents. However, the common tier is often not enough for Azure Sentinel customers, especially as it has to be set for all agents.
The new Azure Monitoring Agent (AMA) can granularly filter Windows events using WEF like XPath expressions.
Q: Does the Agent compress data from on-prem to the cloud?
Yes, the Log Analytics agent (MMA) compresses data when sending it to the cloud. This is used for Syslog, CEF, and local Windows or Linux telemetry. For Linux, the agent uses Zlib compression. The lib compression ratio is typically between 2:1 to 5:1 and maxes out theoretically at 1032:1
Specific connectors
Q: Which API does Azure Sentinel to collect CloudTrail events
The AWS CLoudTrail API LookupEvents end point.
- Start an Administrative Command Prompt and run ‘Net Stop HealthService’
- Start File Explorer and navigate to C:Program Files or C:Program Files(x86)
- Go to this location: Microsoft Monitoring AgentAgent
- Rename the folder Health Service State to Old Health Service State
- In the Administrative Command Prompt, run Net Start Health Service
Module 5: Log Management
Q: The log search is limited to 10K results; what can I do?
Indeed, there is a 10K cap on the result set size in the UI. There is usually not meaningful need to review so many results in the UK. The API, and hence PowerShell, can return up to 500,000 results. Use the PowerShell script to run a query and get the results in a CSV file.
If you still need more than 10K results in the portal:
- You can transform your results into an array, which can hold much more than 10K values. See this example, where over 40K values are put into a single array that you can later export to excel. That would mean you need to use Excel formulas if you want to return to a tabular structure.
- Reduce the size of your results – you can use “distinct Computer,” “summarize by Computer,” or “summarize make_set” to remove duplicate values from your results (Also, if all you need is that computer’s name, “project” only that column)
Q: Which columns are displayed in a search result if not specifically projected?
Multiple heuristics determine which fields to display. Some common ones are:
- Hiding system columns that typically pollute the visual space and are not commonly used (_ResourceId, for example)
- Hiding any columns that do not contain any data for the entire result set
Hiding by default predefined columns for specific tables.
Q: Can I delete unused custom log tables from a workspace?
The tables will disappear once empty. Use the purge API or wait for the retention period ends.
Standard fields include event time fields, record type, and billing information fields. See Standard properties in Azure Monitor Logs for more details.
Module 6: Enrichment: TI, Watchlists, and more
Q: How often does Azure Sentinel Poll TAXII for new IOCs, and can this be configured?
This depends on the TAXII server. Generally speaking, if a well-formed TAXII server adheres to the standards, the TAXII data connector will pull the entire collection on the first connection and then pull only incremental changes every minute.
Q: What information from the TAXII server does Azure Sentinel pull
Currently, Azure Sentinel requests from the TAXII server and ingests only indicator STIX objects. We are planning the support of other STIX Domain Objects in the future. We perform a mapping from STIX to the ThreatIntelligenceIndicator table schema when we import the data.
Q: Is pagination supported in TAXII?
Yes, we support pagination. The TAXII server determines the size of the page. The TAXII server that you are connected to decides the number of IOC’s to be returned in a request.
Q: Do we have specific IP addresses that we would use to pull this data into Sentinel?
While there are no specific IP addresses, they will be Azure IP addresses within the relevant workspace region.
Q: Since the Graph Security API is a tenant level, can one control what threat indicators each workspace receives?
Module 8: Analytics
Q: Are there any restrictions to queries used in Azure Sentinel rules?
Azure Sentinel supports Log Analytics KQL queries; those may somewhat differ from Azure Data Explorer KQL queries.
Also, queries used in alert rules have the following limitations:
- The query max length is 10000
- Cannot contain “search *” and “union *”.
Q: The field I need is not available for entity mapping. Why?
If the field you want to map to an entity in the alert rule configuration screen is not available, the chances are that the value is not a string.
You can check that by trying to manually map as part of the query by adding to the query an “extend” operation:
| extend AccountCustomEntity = your_value
To solve typecast to string the value using the “tostring” function:
| extend AccountCustomEntity = tostring(your_value)
Module 10: Workbooks, reporting, and visualization
Q: Can I add custom Images to a workbook?
You can insert images in a markdown (text) steps in a workbook using the markdown image syntax. The text’s content can also use workbook parameters if you want the paths to change based on the values of parameters.
Q: Can I embed videos in a workbook?
Not at this time, though animated images will work.
Module 12: A day in a SOC analyst’s life, incident management, and investigation
Q: How do I get a notification when a resource is updated?
- When rule templates are updated, the template is flagged as “new” in the UI.
- When a workbook is updated, you are notified in the UI to update it.
- For other resources subscribing to notifications on GitHub
Q: How are incidents updates when Microsoft alerts are updated?
When using Microsoft rules which create incidents directly from an alert from Microsoft products, Azure Sentinel handles updates for those alerts automatically:
- For a new alert arrives, a new incident is created. If the alert is sent as resolved, the incident will be created as resolved.
- If an incident for the alert (meaning, SystemAlertId) already exists, Azure Sentinel updates the incident but will not change its status.
- However, when presented with an alert, Azure Sentinel looks only 1 month back for existing incidents. This means that if an alert is resolved at the providers’ after, say, 50 days, a new resolved incident will be created for that alert update.
Module 13: Hunting
Q: Is there a reason to choose the MITRE attacks tactic in Sentinel for Hunting?
A hunting campaign has to start with a strategy – where do I hunt? This translates to filtering the hunting queries in Azure Sentinel and running the relevant queries to your starting point. A strategy that takes a specific MITRE tactic as a starting point is a popular one.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments