by Contributed | Jun 1, 2021 | Technology
This article is contributed. See the original author and article here.
Synapse Serverless SQL Pool is a serverless query engine platform that allows you to run SQL queries on files or folders placed in Azure storage without duplicating or physically storing the data.
There are broadly three ways to access ADLS using synapse serverless.
- Azure Active Directory
- Managed Identity
- SAS Token
The user would need to be assigned to one of the RBAC role : Azure storage blob data ownercontributorreader role. However, there might be scenario that you would or could not provide access to the ADLS account or container and provide access to granular level directories and folder levels and not complete storage container or blob.
Scenario
You have a data lake that contains employee and social feed data. You have data residing in an employee folder that is used by HR team members and twitter for live social feeds that is usually used by marketing folks. If you use SAS token or RBAC, you cannot control to the folder level.
How do you allow users to perform data exploration using synapse serverless with fine grain control on underlying storage.
Fig1. A storage account with container demo contains two folder employee and twitter.
Solution
To solve this challenge, you can use directory scoped SAS token along with database scope credentials in synapse serverless.
Directory scoped SAS provides constrained access to a single directory when using ADLS Gen2. This can be used to provide access to a directory and the files it contains. Previously a SAS could be used to provide access to a filesystem or a file but not a directory. This added flexibility allows more granular and easier access privilege assignment.
Directory scoped shared access signatures (SAS) generally available | Azure updates | Microsoft Azure
|
Storage Account
|
Container
|
Folder
|
File
|
AAD
|
YES (RBAC on Account)
|
YES (RBAC on Container)
|
YES (via POSIX ACLs)
|
YES (via POSIX ACLs)
|
Managed Identity (same as AAD)
|
YES (RBAC on Account)
|
YES (RBAC on Container)
|
YES (via POSIX ACLs)
|
YES (via POSIX ACLs)
|
SAS Token
|
YES (Scope – Account)
|
YES(Scope – Container)
|
YES (Scope – Directory and Files)
|
YES (Scope – Files)
|
For
How to create Directory based SAS token
You can do via SDK or portal. To create a SAS token via portal.
a. Navigate to the folder that you would like to provide access and right click on the folder and select generate SAS token.
Fig 2 : Directory scope selection for employee folder
b. Select permissions Read, list and execute to read and load all the files in the folder. Provide the expiration date and click generate SAS token and URL. Copy blob SAS token.
Fig 3 : Generate SAS token.
The step b. can be similar to create storage SAS token . Earlier, it used to apply to storage account, now you can reduce the surface area to directory and files as well.
Use Serverless with directory SAS token
Once the storage account access has been configured using SAS token, the next to access the data using synapse serverless engine.
On Azure synapse Studio, go to develop and SQL Script.
a. Create a master key, if it is not there.
— create master key that will protect the credentials:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = <enter very strong password here>
b. Create a database scope credential using the sas token. You would like to access HR data. So use the blog storage sas token generated for Employee directory.
CREATE DATABASE SCOPED CREDENTIAL mysastokenemployee
WITH IDENTITY = ‘SHARED ACCESS SIGNATURE‘,
SECRET = ‘<blob sas token>‘
c. Create external data source till the container path demo1 and use credential mysastokenemployee
CREATE EXTERNAL DATA SOURCE myemployee
WITH ( LOCATION = ‘https:// <storageaccountname>.dfs.core.windows.net/<filesystemname>‘,
CREDENTIAL = mysastokenemployee
)
d. Once the data is created, lets read the data using OPENROWSET BULK in serverless
SELECT * FROM OPENROWSET(
BULK ‘/employee/*.csv’,
DATA_SOURCE = ‘myemployee’,
FORMAT =’CSV’,
parser_version = ‘2.0’,
HEADER_ROW = TRUE
) AS Data
Fig 4 : Openrowset bulk output
e. Now to confirm whether the scope of the SAS token is only restricted to employee folder, lets use the same data source and database credential to access file in twitter folder.
SELECT * FROM OPENROWSET(
BULK ‘/twitter/StarterKitTerms.csv’,
DATA_SOURCE = ‘myemployee’,
FORMAT =’CSV’,
parser_version = ‘2.0’,
HEADER_ROW = TRUE
) AS Data
Fig 5: Bulk openrowset access failure
f. You will encounter an error because the scope of the SAS token was restricted to employee folder.
Now, to access twitter folder for the marketing representative, create a database scoped credential using a sas token for twitter folder. Repeat the steps “How to create Directory based SAS token” for twitter folder.
CREATE DATABASE SCOPED CREDENTIAL mysastokentwitter
WITH IDENTITY = ‘SHARED ACCESS SIGNATURE‘,
SECRET = ‘<blob sas token>‘
g. Create an external data source using the scope credential created for twitter directory.
CREATE EXTERNAL DATA SOURCE mytwitter
WITH ( LOCATION = ‘https://<storageaccountname>.dfs.core.windows.net/<filesystemname>/‘,
CREDENTIAL = mysastokentwitter
)
h. Once the data source is created , you can query the twitter data using newly created data source.
SELECT * FROM OPENROWSET(
BULK ‘/twitter/StarterKitTerms.csv’,
DATA_SOURCE = ‘mytwitter’,
FORMAT =’CSV’,
parser_version = ‘2.0’,
HEADER_ROW = TRUE
) AS Data
Fig 6 : Twitter data accessed using the directory sas token
Summary
- In a central data lake environment or any file store , directory sas token is a great way of reducing the access surface area without providing access at storage root or account level.
- Separation of duties and roles can be easily achieved as data access is controlled at storage level and synapse serverless
- Create sas token with read, list and execute to minimize the impact of accidental deletion etc. sharing the sas token should be done in a secured manner.
- Expire sas token, regenerate new token and recreate the scope credentials frequently.
- Serverless is great way of data exploration without spinning any additional SQL resources. You would be charged based on data processed by each query.
- Managing too many sas tokens will be challenge. So, use a hybrid approach of breaking the large data lake to smaller pools or mesh and grant RBAC access control and blend with SAS token for regulated users is best way of scaling the serverless capability.
by Contributed | Jun 1, 2021 | Technology
This article is contributed. See the original author and article here.
The future of IoT is inherently visual, so why isn’t our tooling? We’ve seen from very early on that Azure Digital Twins solutions are built on visual mental models, so we’ve set out to offer visual tooling with developers in mind. Instead of creating visual tooling from scratch, developers can fork and customize the digital-twins-explorer GitHub sample for their solutions. We’ve had such resounding support for this tool that we’ve taken the next step of hosting it in a web application for developers to take advantage of on Day 1. Learning and exploring Azure Digital Twins just got way easier – and even more collaborative.
Build and validate queries
It’s paramount that the queries you’re leveraging in your solution reflect the logic that you’re intending. The Query Explorer lets you intuitively validate these critical queries through both visual and JSON-structured responses, and even lets you save the queries you use often. Leverage features like Query Overlay to view query results in context of a superset graph and Filtering and Highlighting to investigate those results even further.
Prototype and test your digital twins
Your digital twin graph is never static – there are new machines to model, model versions to upgrade and properties being updated all the time to reflect your environment’s state. The Explorer lets you both create twins and relationships from inside the Twin Graph viewer and click into them to modify properties of all types. From creating missing relationships to testing a property-change-triggered event handler, managing properties in the Explorer makes your ad-hoc operations simpler.
Explore your model ontology
Model graphs can be large and complex, especially with more real-world domains being modeled – like parking infrastructure and weather data in a Smart City. Whether you’re authoring your own ontology or leveraging the industry-standard ontologies, the Model Graph viewer helps you understand your ontology at a high level and drill into the models with filtering and highlighting.
Now, more collaborative!
Importing and exporting your digital twin graph has already made it far easier to share the current state of your modeled environment with teammates, but we’ve extended the collaborative nature of the tool. With the hosted Explorer, we’ve enabled linking to environments which lets you share a specific query of your twin graph with teammates that have access to your Azure Digital Twins instance.
Get started
Resources
by Contributed | Jun 1, 2021 | Technology
This article is contributed. See the original author and article here.
By John Mighell, Sr. Product Marketing Manager, Viva Learning Marketing Lead
Along with so many aspects of our work and personal lives, enterprise learning has undergone a massive shift in the last 18 months. As we look towards the future of work and the future of learning, one thing is certain – we’ll never go back to the way it was.
With so much accelerated forward progress, we’re more excited than ever to share our latest insights and learning product news at the upcoming Microsoft Learning Transformation Briefings.
You’ll not only hear the latest from Microsoft & LinkedIn, but also from CLOs at companies right at the forefront of the learning transformation, and from leading industry analyst Josh Bersin, as he provides insights on his new research: “Developing Capabilities for the Future: the Whole Story.”
If you haven’t signed up yet, there’s still time. The Learning Transformation Briefings start on June 2 and run through June 9 across 10 geography specific deliveries. You can register here and choose the session that works best for you.
As a critical part of our next generation employee experience approach, we’ll be discussing our journey with Microsoft Viva – toward enabling an employee experience that empowers people to be their best at work. Make sure to attend Kirk Koenigsbauer’s (Microsoft COO & CVP, Experiences + Devices) session for his deep dive on Viva Learning, including a product demo with an exciting set of new features.
By bringing learning in the flow of work with Microsoft 365, we see Viva Learning as a transformational tool for learning leaders to help build a robust learning culture. And we’re not the only ones excited. Our outstanding partner ecosystem sees a similar opportunity, and we’re thrilled to share the comments below from a few names you’ll recognize.
“The world of work is evolving at speed. The workforce, workplace and workspace are all changing the nature of work, and learning is evolving with it. We see the entire Viva suite as being a next generation solution that will transform the employee experience. With Viva Learning in particular, we’re excited to work with Microsoft on cloud-based KPMG Learning Solutions to help transform learning experiences for clients and their employees by bringing learning into the flow of their work.”
– Jens Rassloff, Global Head of Strategic Relations & Investments, KPMG
“We see Viva as a progressive EX platform that can accelerate the Workplace Experience journey for our clients – putting people at the center of the organization. With Viva Learning in particular, we’re excited to partner with Microsoft on a product that brings learning seamlessly into the flow of work, making it more accessible for employees, and easier than ever for leaders to build a learning culture.”
– Veit Siegenheim, Global Modern Workplace Solution Area Lead, Avanade & Accenture Microsoft Business Group
“Viva Learning opens new opportunities for learning in the flow of work, which is a foundational strategy of our sales L&D transformation. We look forward to the ongoing partnership with Microsoft as we explore the value to our organization.”
– Bruce Sánchez, Global Lead, Sales Learning and Development Technology, Dell Technologies Global Sales L&D
We hope to see you at the Learning Transformation Briefings. And if you’re not convinced yet, we’ll let Josh Bersin – one of our featured speakers – have the last word.
“I’m really excited about the 2021 Microsoft Learning Transformation briefings – we’ve brought together the latest research, case studies, and breakthrough new ideas to help you develop your company’s skills for the future!”
– John Bersin, Founder and Dean, Josh Bersin Academy
We’re looking forward to sharing the latest on Viva Learning, and so much more. See you there!
by Contributed | Jun 1, 2021 | Technology
This article is contributed. See the original author and article here.
The launch of Microsoft Lists and Tasks in Microsoft Teams last year added new options to an already robust catalog of Microsoft work management tools. They seemed to overlap with Microsoft To Do, Microsoft Planner, and Microsoft Project for the web, causing a lot of (understandable) confusion and questions, all of which boiled down to, “Which tool should I use?”
Today, we’re answering that question with three aptly named when-to-use guides. These one-page documents, which are linked below, focus on different work management scenarios and the Microsoft tools that enable them:
The goal of these guides is to help you determine the best tool for managing your work and its associated tasks and information; they are not meant as comprehensive fact sheets. Those details are available on the associated support pages, which are linked in the guides. Instead, the when-to-use guides focus on the best use for each tool and its distinguishing features. All in all, the guides are broken up into four main sections:
- General tool description
- How should I use it? Overall purpose of the tool
- What’s it best for? Scenarios where the tool excels
- How’s it different? Features that distinguish the tool from others
There’s also a pair of sections about where each tool is available (How do I get them?) and how you can find more information (Where can I learn more?).
It’s important to note that the four main sections describe each tool in the context of the others. For example, you’ll see Planner is for “visually managing simple, task-based efforts” in the guide focused on team-based work. Both Lists and Project for the web can support simple, task-based efforts too—but compared to Planner, it’s not where they excel. Lists is for tracking information and Project for the web is for managing more complex work initiatives—scenarios where Planner is not the best fit.
This approach is worth remembering as you’re reading through these guides. If you find a tool is missing a feature or obvious use case, it’s because there’s another one that’s better suited for that scenario. Again, our goal is to help you decide which tool is best for managing your work, not providing a comprehensive run-down of those tools.
The when-to-use guides are part of our ongoing journey/effort for task management. Work is more disorienting than ever these days, but Microsoft 365 helps streamline all the competing to-dos, resources, and collaboration requirements. For all the latest task management in Microsoft 365 news , continue visiting our Planner Tech Community.
by Contributed | Jun 1, 2021 | Technology
This article is contributed. See the original author and article here.
Everyone likes to get reliable answers from a trustworthy expert, right? Stoneridge Software is the expert that other organizations call when they need help tackling their most complex business systems. But what does it take these days to be an expert in a world that’s constantly changing—and to cultivate trust in that expertise? According to Stoneridge, it takes a love of technology, more than 200 Microsoft Certifications, and a company-wide commitment to learning.
The ever-curious team at Microsoft Learn wanted to find out more about what learning, training, and certification mean to Stoneridge Software and how these contribute to the company’s success. We talked to Principal Developer and Team Lead Nicole Gentz and her colleague, Team Manager Jessica Dunlap, and asked them about the company’s learning culture and how the experts at Stoneridge got so many Microsoft Certifications.
Staying Gold means staying on top of change
Stoneridge is a Microsoft Gold partner and an expert in Microsoft Dynamics 365—a suite of cloud-based products that bring together customer relationship management (CRM) and enterprise resource planning (ERP) functions for large and small organizations. It’s the software that companies need to run their mission-critical systems, including sales, finance, operations, service, human resources, and more.
“In tech especially, you’re only as good as what you keep up with,” Dunlap points out. “In order for us to be the best in the industry, we have to make sure we’re allowing people to invest in themselves.”
Stoneridge was founded in 2012 by former Microsoft employees who proudly call themselves “technology nerds.” It’s even on the Stoneridge website! Even longtime employees like Gentz invest in training classes. She has been with the company since its earliest days and has more than 20 years of ERP experience. “You can’t be stagnant,” she notes. “You have to constantly keep up and grow.”
Today she and Dunlap lead teams of expert consultants, solution architects, and developers who maintain their technology nerd status in a variety of ways—instructor-led programs taught by Microsoft Learning Partners, self-paced online courses, in-house lunch-and-learn events, monthly get-togethers, and tactical internet browsing.
That investment has helped Stoneridge grow into an award-winning global business that employs more than 200 people and provides guidance for a long client list that represents manufacturing, agriculture, construction, and other industries.
Certifications drive a culture that drives the certifications
To maintain Gold partner status, an organization needs to employ a certain number of employees who have current Microsoft Certifications. That’s another reason why Stoneridge promotes a culture of learning where employees are rewarded for achieving those certifications.
“It’s a big reason why we want our employees to be certified,” explains Dunlap, “but at the same time, we realize how important it is to have the knowledge and skill set to support the type of clients we have asking for our services.”
Gentz agrees, adding, “It says a lot to our clients to know, ‘Hey, these guys say they know what they’re doing!’ Certification is an extra layer.” Gentz has personally accumulated an impressive number of Microsoft Certifications in Dynamics 365 during her time at Stoneridge. “I worked in ERP for years, but the certifications helped me ensure that I knew what I knew.”
Gentz’s team supports enterprises that use Dynamics 365 Finance and other products in the suite that meet the needs of Stoneridge’s largest clients. Dunlap’s team works with small-to-medium-sized organizations with implementations of all Dynamics products, including Dynamics 365 Business Central and earlier versions of Dynamics. “Our training programs help us keep up with all the changes,” Dunlap notes. “It’s been great having the experts to show us what’s new.”
Regardless of the size of the client, the Stoneridge teams take their training seriously. This year, the company set a goal to achieve 60 new certifications by the end of the year. “We’re more than halfway there already,” Dunlap reports.
The training and certification process typically starts with a simple request, such as, “Hey, Jessica, I’d really like to get certified.” And she says, “OK! When?”
Even Stoneridge developers who’ve been at it for decades work toward Microsoft Certifications in their areas of interest. “I have guys on my team who worked at Microsoft for 25 years,” Gentz notes. “They still want to make sure their skills are up to par. The exams test you on another level.”
Gentz and Dunlap agree. When it comes to gaining the skills and knowledge for the associate and expert certifications, there’s no substitute for real-world experience. And that day-to-day, hands-on work with the technology augments the more structured training in the classroom. The result is good for Stoneridge clients, because as Gentz explains, “You really need to have a deep understanding of the products.”
Finding time for training
In addition to the Dynamics 365 certifications, such as Microsoft Certified: Dynamics 365 Finance and Operations Apps Developer Associate (candidates need to pass Exam MB-300 and Exam MB-500) and Microsoft Certified: Dynamics 365 Field Service Functional Consultant Associate (candidates need to pass Exam PL-200 and Exam MB-240), this year, the Stoneridge team is adding Microsoft Power Platform training to the list. The team is skilling up for the Microsoft Certified: Power Platform Functional Consultant Associate certification (candidates need to pass Exam PL-200) and for the Microsoft Certified: Power Platform Developer Associate certification (candidates need to pass Exam PL-400).
According to Dunlap, “The timing of these courses was great for us, since we just rolled out our five-year vision.” The company’s focus is Dynamics 365, but it recognizes the value of Microsoft Power Platform and the growing trend of low-code and no-code projects. As noted in Microsoft named a Leader in the 2021 Gartner Magic Quadrant for Analytics and BI Platforms on the Microsoft Power BI Blog, 97 percent of Fortune 500 companies are using Microsoft Power Platform. Its popular approach to app making is creating a new category of users called “citizen developers.” Other developers also appreciate how quickly they can build apps using Microsoft Power Platform tools. For Stoneridge, that means added value for their Dynamics 365 clients.
However, even the most enthusiastic learners at Stoneridge have a busy client schedule. “Developers always think they’re too busy,” Dunlap laughs. However, every developer sets certification goals, and the team managers give them the time they need. It’s a key part of the company’s core values.
Dunlap’s strategy for her busy team is to find downtime between client engagements. “Before the next project starts, I like to give my team members a change of pace, a week to wind down and invest in some learning and development.” Even in a small project that might take only half a day of client work, “I tell them to spend the other half of your day investing in yourself. Study for an exam. Write a blog.”
Not only does the company maintain a budget for its learning culture, but also it encourages team members to share what they know. Team members write blog posts with practical tips from their Dynamics 365 client engagements. Internal experts lead “confabs” where they talk about the latest industry trends.
“Last week they talked about data lakes,” Dunlap recalls. “It was really interesting, and they put it in a way that people who aren’t technical can understand. It’s super important that we’re not only investing in ourselves, but we share all the info we can to let others know, hey, this is great stuff!”
Expertise builds trust
Like any smart business, Stoneridge Software knows that fostering customer trust creates return engagements. Each individual Microsoft Certification is a building block toward creating that trust, and each team member’s commitment to training helps the company uphold its proud technology nerd status.
“Customers come to us for advice, asking, ‘How should I design my system?’” Gentz explains. “They come to us for expert knowledge. They come to us with big, long-term plans, and we have to be able to provide that. When we do, we gain their trust. And then they come back years later because of that trust.”
Learn more:
Microsoft Certifications
Microsoft Learning Partners
Dynamics 365 on Microsoft Learn
Microsoft Power Platform on Microsoft Learn
Recent Comments