This article is contributed. See the original author and article here.
Welcome to Azure Data Factory’s November monthly update! Here we’ll share the latest updates on What’s New in Azure Data Factory. You can find all our updates here.
We’ll also be hosting our monthly livestream next week on December 15th at 9:00am PST/ 12:00pm EST! Join us to see some live demos and to ask us your ADF questions!
Join the livestream here.
Table of Contents
Continuous Integration and continuous deployment (CI/CD)
Continuous integration and continuous delivery (CI/CD)
Disable publish button in ADF studio now available
We have added the ability to disable the publish button from the ADF studio when source control is enabled. Once this is enabled, the Publish button will be grayed out in the ADF Studio. This will help to avoid overwriting the last automated publish deployment.
To learn more about this update, read Source control – Azure Data Factory | Microsoft Learn.
New improvements to SAP Change Data Capture (CDC)
We added some new updates to the SAP CDC connector in mapping data flows, adding new capabilities:
Incremental only is now available
You can get changes only from SAP system without initial full load. With this feature, you have the flexibility to focus on incremental data process without worrying about big data volume movement on an initial full data load.
Performance improvements have been made – source partitions in initial full data load
With a simple radio button, you can now connect your SAP system to multiple sessions in parallel to extract data, which will greatly reduce the time to perform an initial full data load from your SAP system.
Pipeline template added to Template Gallery
There is a new pipeline template in the Template Gallery to help speed up your journey on loading big amounts of SAP objects by a parameterized pipeline with SAP CDC enabled in ADF. You can get more information on the template here.
To learn more about this, read Transform data from an SAP ODP source with the SAP CDC connector in Azure Data Factory or Azure Synapse Analytics – Azure Data Factory & Azure Synapse | Microsoft Learn.
Pipeline designer enhancements added to ADF Studio preview experience
We have added three new UI updates to the ADF Studio preview experience:
Dynamic Content Flyout
A new dynamic content flyout has been added to make it easier to set dynamic content in your pipeline activities without having to use the expression builder.
Error messages have been relocated to the Status column in Debug and Pipeline monitoring. This will make it easier to view errors whenever a pipeline run fails.
Container view for monitoring
A new monitoring view is available, adding the option to enable the container view. This view provides a more consolidated monitoring view of activities that were run. A big change that you will see is that nested activities are now grouped under the parent activity. To be able to view more activities on one page, you can choose to hide the list of nested activities under the parent activity.
To learn more about these enhancements, read Managing Azure Data Factory studio preview experience – Azure Data Factory | Microsoft Learn.
We hope that you found this helpful! Let us know in the comments if there’s anything else you’d like to see in our blogs or livestreams. We love hearing your feedback!
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.