This article is contributed. See the original author and article here.
We are happy to announce that REST APIs for scanning data plane are now released. Software engineers or developers in your organization can now call these APIs to register data sources, set up scans and classifications programmatically to integrate with other systems or products in your company.
Purview Scaning Data Plane Endpoints
You need to have the purview account name to call scanning APIs. Below is how the endpoint will look:
https://{your-purview-account-name}.scan.purview.azure.com
Set up authentication using service principal.
To call the scanning APIs, the first thing you need to do is to register an application and create a client secret for that application in Azure Active Directory. When you register an application a service principal is automatically created in your tenant. For more information on how to create a service principal (application) and client secret, please refer here.
Once service principal is created, you need to assign ‘Data source Admin’ role of your purview account to the service principal created above. The below steps need to be followed to assign role to establish trust between the service principal and purview account.
- Navigate to your Purview account.
- On the Purview account page, select the tab Access control (IAM)
- Click + Add.
- Select Add role assignment.
- For the Role select Purview Data Source Administrator from the drop down.
- For Assign access to leave the default, User, group, or service principal.
- For Select enter the name of the previously created service principal you wish to assign and then click on their name in the results pane.
- Click on Save.
You’ve now configured the service principal as an application administrator, which enables it to send content to the scanning APIs. Learn about roles here.
Get Token
You can send a POST request to the following URL to get access token.
https://login.microsoftonline.com/{your-tenant-id}/oauth2/token
The following parameters needs to be passed to the above URL.
- client_id: client id of the application registered in Azure Active directory and is assigned ‘Data Source Admin’ role for the Purview account.
- client_secret: client secret created for the above application.
- grant_type: This should be ‘client_credentials’.
- resource: This should be ‘https://purview.azure.net’
Figure 1: Screenshot showing a sample response in Postman.
Scanning Data Plane REST APIs
Once you have followed all the above steps and have received access token you can now call various scanning APIs programmatically. The different types of entities you can interact with are listed below:
- Classification Rules
- Data Sources
- Key Vault Connections
- Scans and scan related functionality like triggers and scan rule sets.
The below examples explains the APIs you need to call to configure a data source , set up and run a scan for the data source but for complete information on all the REST APIs supported by scanning data plane refer here –
1. To create or update a data source using APIs the following REST API can be leveraged:
PUT {Endpoint}/datasources/{dataSourceName}?api-version=2018-12-01-preview
You can register an Azure storage data source with name ‘myStorage’ by sending a PUT request to the following URL
{Endpoint}/datasources/myStorage?api-version=2018-12-01-preview with the below request body:
{
“name”: “myStorage”,
“kind”: “AzureStorage”,
“properties”: {
“endpoint”: “https://azurestorage.core.windows.net/“
}
}
2. To create a scan for a data source already registered in Purview the following REST API can be leveraged:
PUT {Endpoint}/datasources/{dataSourceName}/scans/{scanName}?api-version=2018-12-01-preview
You can schedule a scan ‘myStorageScan’ using a credential ‘CredentialAKV’ and system scan rule set ‘AzureStorage’ for the already registered data source ‘myStorage’ by sending a PUT request to the following URL with the below request body:
{Endpoint}/datasources/myStorage/scans/myStorageScan?api-version=2018-12-01-preview
{
“kind”: “AzureStorageCredential”,
“properties”: {
“credential”: {
“referenceName”: “CredentialAKV”,
“credentialType”: “AccountKey”
},
“connectedVia”: null,
“scanRulesetName”: “AzureStorage”,
“scanRulesetType”: “System”
}
}
The above call with return the following response:
{
“name”: “myStorageScan”,
“id”: “datasources/myDataSource/scans/myScanName”,
“kind”: “AzureStorageCredential”,
“properties”: {
“credential”: {
“referenceName”: “CredentialAKV”,
“credentialType”: “AccountKey”
},
“connectedVia”: null,
“scanRulesetName”: “AzureStorage”,
“scanRulesetType”: “System”,
“workers”: null
},
“scanResults”: null
}
3. Once the scan is created you need to add filters to the scan which is basically scoping your scan or determining what objects should be included as part of scan. To create a filter, you can leverage the following REST API
PUT {Endpoint}/datasources/{dataSourceName}/scans/{scanName}/filters/custom?api-version=2018-12-01-preview
You can create a filter for the above scan ‘myStorageScan’ by sending a PUT request to the following URL with the below request body. This will create a scope to include folders /share1/user and /share1/aggregated and exclude folder /share1/user/temp/ as part of the scan.
{Endpoint}/datasources/myStorage/scans/myStorageScan/filters/custom?api-version=2018-12-01-preview
{
“properties”: {
“includeUriPrefixes”: [
“https://myStorage.file.core.windows.net/share1/user“,
“https://myStorage.file.core.windows.net/share1/aggregated“
],
“excludeUriPrefixes”: [
“https://myStorage.file.core.windows.net/share1/user/temp“
]
}
}
The above call will return the following response:
{
“name”: “custom”,
“id”: “datasources/myStorage/scans/myStorageScan/filters/custom”,
“properties”: {
“includeUriPrefixes”: [
“https://myStorage.file.core.windows.net/share1/user“,
“https://myStorage.file.core.windows.net/share1/aggregated“
],
“excludeUriPrefixes”: [
“https://myStorage.file.core.windows.net/share1/user/temp“
]
}
}
4.To run a scan, you need to use the following REST API
PUT {Endpoint}/datasources/{dataSourceName}/scans/{scanName}/runs/{runId}?api-version=2018-12-01-preview
You can now trigger the above scan ‘myStorageScan’ by sending a PUT request to the below URL. The runId is a guid.
{Endpoint}/datasources/myStorage/scans/myStorageScan/runs/138301e4-f4f9-4ab5-b734-bac446b236e7?api-version=2018-12-01-preview
The above call will return the following response:
{
“scanResultId”: “138301e4-f4f9-4ab5-b734-bac446b236e7”,
“startTime”: “2019-05-16T17:01:37.3089193Z”,
“endTime”: null,
“status”: “Accepted”,
“error”: null
}
To learn more about Azure Purview, check out our full documentation today.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments