This article is contributed. See the original author and article here.

Welcome to Azure Data Factory’s November monthly update! Here we’ll share the latest updates on What’s New in Azure Data Factory. You can find all our updates here.


 


We’ll also be hosting our monthly livestream next week on December 15th at 9:00am PST/ 12:00pm EST! Join us to see some live demos and to ask us your ADF questions! 


 


Join the livestream here.


 


 


Table of Contents


Continuous Integration and continuous deployment (CI/CD)



Data flow



Developer Productivity



 


Continuous integration and continuous delivery (CI/CD)


 


Disable publish button in ADF studio now available


We have added the ability to disable the publish button from the ADF studio when source control is enabled. Once this is enabled, the Publish button will be grayed out in the ADF Studio. This will help to avoid overwriting the last automated publish deployment.


 


Noelle_Li_0-1670017635045.png


 


To learn more about this update, read Source control – Azure Data Factory | Microsoft Learn.


 


Data flow


 


New improvements to SAP Change Data Capture (CDC)


We added some new updates to the SAP CDC connector in mapping data flows, adding new capabilities:


Incremental only is now available


You can get changes only from SAP system without initial full load. With this feature, you have the flexibility to focus on incremental data process without worrying about big data volume movement on an initial full data load.


 

 

sap-change-data-capture-mapping-data-flow-run-mode.png


 


Performance improvements have been made – source partitions in initial full data load


With a simple radio button, you can now connect your SAP system to multiple sessions in parallel to extract data, which will greatly reduce the time to perform an initial full data load from your SAP system.


 


sap-change-data-capture-mapping-data-flow-optimize-partition.png


 


Pipeline template added to Template Gallery


There is a new pipeline template in the Template Gallery to help speed up your journey on loading big amounts of SAP objects by a parameterized pipeline with SAP CDC enabled in ADF. You can get more information on the template here.


 


To learn more about this, read Transform data from an SAP ODP source with the SAP CDC connector in Azure Data Factory or Azure Synapse Analytics – Azure Data Factory & Azure Synapse | Microsoft Learn.


 


Developer Productivity


 


Pipeline designer enhancements added to ADF Studio preview experience


We have added three new UI updates to the ADF Studio preview experience:


Dynamic Content Flyout


A new dynamic content flyout has been added to make it easier to set dynamic content in your pipeline activities without having to use the expression builder.


 


Noelle_Li_1-1670017635052.png


 


Error Messaging


Error messages have been relocated to the Status column in Debug and Pipeline monitoring. This will make it easier to view errors whenever a pipeline run fails.


 


Noelle_Li_2-1670017635057.png


 


Container view for monitoring


A new monitoring view is available, adding the option to enable the container view. This view provides a more consolidated monitoring view of activities that were run. A big change that you will see is that nested activities are now grouped under the parent activity. To be able to view more activities on one page, you can choose to hide the list of nested activities under the parent activity.  


 


Noelle_Li_3-1670017635064.png


 


To learn more about these enhancements, read Managing Azure Data Factory studio preview experience – Azure Data Factory | Microsoft Learn.


 


We hope that you found this helpful! Let us know in the comments if there’s anything else you’d like to see in our blogs or livestreams. We love hearing your feedback!


 

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.