Cross Service Query – Azure Monitor (LA/AI) and Azure Data Explorer (ADX)

by Contributed | Nov 24, 2020 | Azure, Microsoft, Technology

This article is contributed. See the original author and article here.

Azure Monitor<->Azure Data Explorer cross-service querying (join between LA/AI and ADX!)

This experience enables you to query Azure Data Explorer in Azure Log Analytics/Application Insights tools (See more info here),

and the ability to query Log Analytics/Application Insights from Azure Data Explorer tools to make cross resource queries. (See more info here.),

For example (querying Azure Data Explorer from Log Analytics):

Where the outer query is querying a table in the workspace, and then joining with another table in an Azure Data Explorer cluster (in this case, clustername=help, databasename=samples) by using a new “adx()” function, like how you can do the same to query another workspace from inside query text.

Both experiences are in Private Preview.

The ability to query Azure Monitor from Azure Data Explorer is open for everyone to use – no need to be allowlisted,

The ability to query Azure Data Explorer from Log Analytics/Application Insights requires to be allowlisted – We need the following to get you enrolled (you can send the info to me):

1. Tenant ID


2. List of the Azure Data Explorer clusters (the list is required to enable the team to modify the callout policy of that cluster, that will allow them to communicate with the proxy)


3. Email address

We started a private preview program, and we are happy to add early adopters to experience the new functionality.

Please note that the product is new with limited SLA, and we estimate that we will be able to move to pubic preview with production level SLA within ~2-4 months.

Cross Service Query – Azure Monitor and Azure Data Explorer

by Contributed | Nov 24, 2020 | Azure, Microsoft, Technology

This article is contributed. See the original author and article here.

Azure Monitor<->Azure Data Explorer cross-service querying

This experience enables you to query Azure Data Explorer in Azure Log Analytics/Application Insights tools (See more info here),

and the ability to query Log Analytics/Application Insights from Azure Data Explorer tools to make cross resource queries. (See more info here.),

For example (querying Azure Data Explorer from Log Analytics):

Where the outer query is querying a table in the workspace, and then joining with another table in an Azure Data Explorer cluster (in this case, clustername=help, databasename=samples) by using a new “adx()” function, like how you can do the same to query another workspace from inside query text.

Both experiences are in Private Preview.

The ability to query Azure Monitor from Azure Data Explorer is open for everyone to use – no need to be allowlisted,

The ability to query Azure Data Explorer from Log Analytics/Application Insights requires to be allowlisted – We need the following to get you enrolled (you can send the info to me):

1. Tenant ID


2. List of the Azure Data Explorer clusters (the list is required to enable the team to modify the callout policy of that cluster, that will allow them to communicate with the proxy)


3. Email address

We started a private preview program, and we are happy to add early adopters to experience the new functionality.

Please note that the product is new with limited SLA, and we estimate that we will be able to move to pubic preview with production level SLA within ~2-4 months.

Key Features of SQL Analytics in Azure Databricks

by Contributed | Nov 23, 2020 | Azure, Microsoft, Technology

This article is contributed. See the original author and article here.

As customers continue to standardize on data lakes and the Lakehouse architecture, users expect to be able to query the data in their data lake using SQL.  In fact, approximately 41% of all code executed on Azure Databricks is SQL.  The SQL Analytics service in Azure Databricks was created to provide SQL users with a familiar SQL-editor experience as well as provide optimized BI connections for querying and analyzing data in the data lake.

SQL Analytics Key Features

Below are some of the key features in the SQL Analytics service in Azure Databricks:

• The first key feature to highlight is the Query Editor. This editor provides a familiar experience (vs. the traditional notebook experience in Azure Databricks) where users can explore their databases, write SQL queries with intelligent auto-complete, and view query output in either a tabular display or in a rich set of visualizations. 

• Users can turn the queries and visualizations they create in the query editor into Dashboards.  Dashboards can include content from several different queries, and also allow for basic text/markdown cells.  Once created, dashboard data can be manually refreshed or refreshed on a schedule.


• In addition to dashboards, queries can be scheduled, and alerts can be created to notify users when a field in the scheduled query meets a certain threshold.  Alerts can even be created for multiple columns.


• SQL Analytics also has a catalog for dashboards and queries.  This catalog allows users to save their queries and dashboards and share them with other users.  Users can also add tags to their saved queries and dashboards to make them easier to search, and they can even use Favorites to mark frequently used queries and dashboards.


• The SQL Endpoint in the SQL Analytics service also provides easy connectivity to other BI and SQL tools via ODBC/JDBC connections.  Tools such as Power BI can connect using the native Azure Databricks connector and take advantage of faster, more efficient ODBC/JDBC drivers.


• Another exciting feature in the SQL Analytics service is the ability to see Query History details.  On the History page, users and admins can see details about all the queries that have been run.  This includes specifics such as the query itself, who ran the query, what endpoint did it run on, was it successful, the duration, how much time was spent compiling-executing-result-fetching, rows read/returned, files scanned, bytes read from cache, and many other details.

• The final key feature to look at in the SQL Analytics service is the compute engine.  SQL Analytics uses the same Delta Engine found in the rest of Azure Databricks.  This means a single, consistent set of APIs and functions across the entire workspace.  The SQL Analytics service goes one step further by also making use of the Photon-powered Delta Engine. This vectorized engine was purpose built for doing SQL and data frame operations while maintaining the same single, consistent set of APIs and functions currently used across the entire workspace.

For more information about the SQL Analytics service in Azure Databricks, check out the docs page and the Databricks launch blog. The SQL Analytics service is currently in Public Preview. Contact your Azure Databricks representative to request access. Get started with Azure Databricks by joining an Azure Databricks workshop.