This article is contributed. See the original author and article here.
Azure Data Factory and Azure Synapse Analytics have a new update for the monitoring UI to make it easier to view your data flow ETL job executions and quickly identify areas for performance tuning.
Large data flows are now much easier to visualize and traverse inside the monitoring view in ADF and Synapse. The visual indicators of data flow streams from source to sink is now easily viewable by selecting the related sink in either the graph or the details pane at the bottom.
To switch back into large icon mode with visible labels, increase your zoom in ratio with the right-hand zoom slider:
Note that every data point collected from your data flow ETL job execution can be sorted in that results table. This feature makes it easy to find transformations that had partitioning issues via skew or kurtosis as well as quickly identifying which stages of your transformation had the longest execution times by sorting processing time. Sort by rows written to find which stages wrote the most data.
At any zoom level, you can always search for your transformations using the search icon and search bar:
Detailed rows and lineage analysis of each stage of your transformations is available from the Stages and Lineage links on the “Sinks” detail pane.
Selecting a transformation icon from the graph will present details about partitions and timings:
We’ve also added more details in the sink transformation monitoring. When you select the sink transformation in your monitoring graph, you can view properties that now include pre-processing and post-processing times for SQL sinks as well as the temp table name utilized during staging and the time taken for each SQL operation. This will provide more details around the amount of time each data flow ETL job takes to complete.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments