Realtime analytics from SQL Server to Power BI with Debezium

by Contributed | May 5, 2021 | Technology

This article is contributed. See the original author and article here.

Almost every modern data warehouse project touches the topic of real-time data analytics. In many cases, the source systems use a traditional database, just like SQL Server, and they do not support event-based interfaces.

Common solutions for this problem often require a lot of coding, but I will present an alternative that can integrate the data form SQL Server Change Data Capture to a Power BI Streaming dataset with a good help of an Open-Source tool named Debezium.

The Problem

SQL Server is a batch-oriented service, just like any DBMS. This means that one program must query it in order to get the result – so to have real time analytics we would have to change this batch behavior to a streaming /event/push behavior.

On the other side, we have Azure Event Hubs, Stream Analytics and Streaming datasets on Power BI. They work pretty well together if the data source is a stream producing events (something we can have with a custom code application or some Open Source solution like Kafka).

The challenge here was to find something to make the bridge between SQL Server and Event Hubs.

After some time looking for solutions, I found this Docs page (Integrate Apache Kafka Connect on Azure Event Hubs with Debezium for Change Data Capture – Azure Event Hubs | Microsoft Docs) with an approach to bring CDC data from Postgres to Event Hubs.

The solution presented on Docs was more complex than I needed, so I simplified it by using a container deployment and by removing unnecessary dependencies, then I wrote this post where I hope I can present it is a simpler way.

The solution looks like this: On one side, we have SQL Server with CDC enabled in a few tables, on the other we have Azure ready to process events that came to Event Hub. To make the bridge, we have Debezium that will create one event per row found on the CDC tables.

Have you ever used Docker?

For my development environment, I decided to go for Docker Desktop. The new WSL2 backend makes it easy to run Linux containers (such as those needed by Debezium). It works well on Windows 10 and on recent builds of Windows Server (Semi-Annual Channel). If you still never tried WSL2, I highly recommend it. (See: Install Docker Desktop on Windows | Docker Documentation)

After a few steps, I have installed the WSL2 feature, chose the Ubuntu 20.04 distro (there other distros available on Windows Store) and finished the setup of Docker Desktop on my Windows 10 Surface Laptop :smiling_face_with_smiling_eyes:.

How to install a Debezium container?

Debezium has a Docker Image available on hub.docker.com, named “debeziumserver”.

Debezium Server is a lightweight version that do NOT have Kafka installed. The image has already the connector you need for SQL Server and can output the events directly to Event Hubs.

To install and configure the container, I ran only this single line on PowerShell.

docker run -d -it --name SampleDebezium -v $PWD/conf:/debezium/conf -v $PWD/data:/debezium/data debezium/server

This will download the docker image “debezium/server” and start a container named “SampleDebezium”

We are mounting 2 folders from the host machine to the container:

/conf                     – Holds the configuration file.

/data                     – Will store the status of Debezium. This avoids missing or duplicate data once the container is recreated or restarted.

On my lab, I used the configuration file bellow (place it on the /conf folder named as application.properties). (Don’t worry about the keys here, I changed them already)

You will have to change the SQL Server and Event Hubs connections to match to match your enviorement.

Sample application.properties file:

debezium.sink.type=eventhubs
debezium.sink.eventhubs.connectionstring=Endpoint=sb://er-testforthisblogpost.servicebus.windows.net/;SharedAccessKeyName=TestPolicy;SharedAccessKey=O*&HBi8gbBO7NHn7N&In7ih/KgONHN=
debezium.sink.eventhubs.hubname=hubtestcdc
debezium.sink.eventhubs.maxbatchsize=1048576

debezium.source.connector.class=io.debezium.connector.sqlserver.SqlServerConnector
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.offset.flush.interval.ms=0
debezium.source.database.hostname=sqlserverhostname.westeurope.cloudapp.azure.com
debezium.source.database.port=1433
debezium.source.database.user=UserDemoCDC
debezium.source.database.password=demo@123
debezium.source.database.dbname=TestCDC
debezium.source.database.server.name=SQL2019
debezium.source.table.include.list=dbo.SampleCDC

debezium.source.snapshot.mode=schema_only
debezium.source.max.queue.size=8192
debezium.source.max.batch.size=2048
debezium.source.snapshot.fetch.size=2000
debezium.source.query.fetch.size=1000

debezium.source.poll.interval.ms=1000

debezium.source.database.history=io.debezium.relational.history.FileDatabaseHistory
debezium.source.database.history.file.filename=data/FileDatabaseHistory.dat

Setting up the Change Data Capture

My SQL Server is hosted on Azure (but this is not a requirement) and to create a lab enviorement, I created a single table and enabled CDC on it by using this script:

-- Create sample database
CREATE DATABASE TestCDC
GO
USE TestCDC  
GO  


-- Enable the database for CDC
EXEC sys.sp_cdc_enable_db  
GO  

-- Create a sample table
CREATE TABLE SampleCDC (
ID int identity (1,1) PRIMARY KEY ,
SampleName nvarchar(255)
)

-- Role with privileges to read CDC data
CREATE ROLE CDC_Reader

-- =========  
-- Enable a Table 
-- =========  
EXEC sys.sp_cdc_enable_table  
@source_schema = N'dbo',  
@source_name   = N'SampleCDC',  
@role_name     = N'CDC_Reader',     -- The user must be part of this role to access the CDC data
--@filegroup_name = N'MyDB_CT',     -- A filegroup can be specified to store the CDC data
@supports_net_changes = 0           -- Debezium do not use net changes, so it is not relevant

-- List the tables with CDC enabled
EXEC sys.sp_cdc_help_change_data_capture
GO

-- Insert some sample data
INSERT INTO dbo.SampleCDC VALUES ('Insert you value here')

-- The table is empty
select * from SampleCDC

-- But it recorded CDC data
select * from [cdc].[dbo_SampleCDC_CT]


/*
-- Disable CDC on the table
--EXEC sys.sp_cdc_disable_table  
--@source_schema = N'dbo',  
--@source_name   = N'SampleCDC',  
--@capture_instance = N'dbo_SampleCDC'  
*/

Debezium will query the latest changed rows on CDC based on its configuration file and create the events on Event Hub.

Event Hub and Stream Analytics

I created a Event Hub Namespace with a single Event Hub to hold this experiment. There is no special requirement for the event hub. The size will depend only on the volume of events your application will send to it.

Once it is done, we have to create a Shared Access Policy. The connection string is what you need to add to the Debezium application.properties file.

To consume the events and create the Power BI streaming dataset, I used Azure Stream Analytics.

Once the Stream Analytics job is created, we have to configure 3 things: Inputs, Outputs and Query.

Inputs

Here is where you say what the stream analytics will listen to. Just create a Inputs for your Event Hub with the default options. Debezium will generate uncompressed JSON files encoded on UTF-8.

Outputs

Here is where we will configure the Power BI streaming dataset. But you first need to know on which Power BI Workspace it will stay.

On the left menu, click on Outputs and then Add -> Power BI.

The options “Dataset name” and “Table name” are what will be visible to Power BI.

The a test, the “Authentication mode” as “User Token” is a good one, but for production, better use “Managed Identity”

Query

Stream used a query language very similar to T-SQL to handle the data that comes in a stream input.

Check this link to find more about it Stream Analytics Query Language Reference – Stream Analytics Query | Microsoft Docs

On the example, I’m just counting row many rows (events) were generated on the last 10 minutes FROM an input INTO an output. The names on the query must match the ones you defined on prior steps.

Here it is in the text version:

SELECT
    count(*) over( LIMIT DURATION (minute, 10)) as RowsLast10Minutes
INTO
    [TestCDC]
FROM
    [InputCDC]

Make it Run

If everything is correctly configured, we will be able to start our Stream Analytics and our Container.

Stream Analytics:

Docker Desktop:

Power BI

Once the Stream Analytics is started, we will go to the Power BI workspace and create a tile based on the streaming dataset.

If you don’t have a Power BI Dashboard on your workspace, just create a new one.

On the Dashboard, add a Tile.

Click on Real-time data and Next.

The dataset with the name you chose on Stream Analytics should be visible here.

Select the Card visual and the column RowsLast10Minutes, click Next, and Apply.

It should be something like this if there is no data being inserted on the table.

Now it comes the fun time. If everything is configured correctly, we just have to insert data on our sample table and see it flowing to the dashboard!

Known problems

It seems when a big transaction happens (like an update on 200k rows), Debezium stops pointing that the message size was bigger than the maximum size defined from Event Hubs. Maybe there is a way to break it on smaller messages, maybe it is how it works by design. If the base (CDC) table are OLTP oriented (small, single row operations), the solutions seem to run fine.

References

Enable and Disable Change Data Capture (SQL Server)

Debezium Server Architecture

Docker image with example

SQL Server connector

Azure Event hubs connection

Our journey to make Excel work well for everyone

by Contributed | May 5, 2021 | Technology

This article is contributed. See the original author and article here.

Accessibility is about making our products accessible and inclusive to everyone, including the 1 billion+ people with disabilities around the world. It is a core Excel and Microsoft priority, and an area where we continuously strive to improve. For more information about Microsoft’s commitment to accessibility, visit microsoft.com/accessibility.

Excel’s approach

Making Excel more accessible is a journey, and we will always have room for improvement as we strive to make spreadsheets work for everyone. We have three overarching objectives to guide us:

1. Work seamlessly with assistive technology. Our partners around the globe and within Microsoft create amazing technology to support people with disabilities. A few examples of assistive technology include screen readers, dictation software, magnifiers, and physical devices. Our priority is to collaborate with these partners so that everyone can use Excel in the way that works best for them, with tools that are already familiar.


2. Build inclusive and delightful experiences inspired by people with disabilities. Beyond “just working,” Excel should be efficient and delightful to use. We are always looking for opportunities to simplify your workflow, summarize your content, or suggest information. By learning from and being inspired by the experiences of people with disabilities, we can make Excel better for all.


3. Support authors to create accessible content. Making Excel as an application accessible is only half the battle; the other half is making sure workbooks created in Excel are accessible. For that, we rely heavily on you, the author…but that doesn’t mean we can’t help! We look for opportunities to automatically create accessible content on your behalf, support you while you create accessible content, let you know when something is inaccessible, and help you fix accessibility issues before sharing your workbook with others.

Feature Spotlight: Accessibility ribbon

As a part of our goal to help you create accessible content, there is a new contextual ribbon called “Accessibility” coming to Excel. The Accessibility ribbon will be available when you open the Check Accessibility pane on the Review tab. On the Accessibility ribbon you will find a collection of the most common tools you need to make your workbook accessible. The ribbon is available today to those in the Office Insiders program.

Contextual Accessibility ribbon in Excel shows when the Check Accessibility pane is open.

To learn more about the Accessibility ribbon, view our announcement on the Office Insiders blog. If you are interested in learning more about how to create accessible workbooks, check out our support article Make your Excel documents accessible to people with disabilities.

Next steps

Please let us know what you think! Your feedback and suggestions shape our approach to accessibility and inclusive design. To get in touch, contact the Disability Answer Desk or use Help > Feedback. We look forward to hearing from you!

This is the first in a series of blogs about our accessibility work in Excel. We hope you will join us over the next several months as we discuss exciting new improvements to Excel through an inclusive lens.

Subscribe to our Excel Blog and join our Excel Community to stay connected with us and other Excel fans around the world.

Announcing the Azure Sentinel: Zero Trust (TIC3.0) Workbook

by Contributed | May 5, 2021 | Technology

This article is contributed. See the original author and article here.

The Azure Sentinel: Zero Trust (TIC3.0) Workbook provides an automated visualization of Zero Trust principles cross walked to the Trusted Internet Connections framework. Compliance isn’t just an annual requirement, and organizations must monitor configurations over time like a muscle. This workbook leverages the full breadth of Microsoft security offerings across Azure, Office 365, Teams, Intune, Windows Virtual Desktop, and many more. This workbook enables Implementers, SecOps Analysts, Assessors, Security & Compliance Decision Makers, and MSSPs to gain situational awareness for cloud workloads’ security posture. The workbook features 76+ control cards aligned to the TIC 3.0 security capabilities with selectable GUI buttons for navigation. This workbook is designed to augment staffing through automation, artificial intelligence, machine learning, query/alerting generation, visualizations, tailored recommendations, and respective documentation references.

Azure Sentinel: Zero Trust (TIC3.0) Workbook

Mapping technology to Zero Trust frameworks is a challenge in the federal sector. We need to change our thinking in security assessment as the cloud evolves at the speed of innovation and growth, which often challenges our security requirements. We need a method to map Zero Trust approaches to technology while measuring change over time like a muscle.

What Are the Use Cases?

There are numerous use cases for this workbook including role alignment, mappings, visualizations, time-bound measurement, and time-saving features:

Roles

• Implementers: Build/Design


• SecOps: Alert/Automation Building


• Assessors: Audit, Compliance, Assessment


• Security & Compliance Decision Makers: Situational Awareness


• MSSP: Consultants, Managed Service

Mappings

• Framework to Requirement to Microsoft Technology

Visualization

• Hundreds of Visualizations, Recommendations, Queries

Time-Bound

• Measure Posture Over Time for Maturity

Time-Saving

• Aggregation & Analysis


• Capabilities Assessment


• Navigation


• Documentation


• Compliance Mapping


• Query/Alert Generation

Microsoft Offerings Overlay to TIC Capabilities

What is Zero Trust?

Zero Trust is a security architecture model that institutes a deny until verified approach to access resources from both inside and outside the network. This approach addresses the challenges associated with a shifting security perimeter in a cloud-centric and mobile workforce era. The core principle of Zero Trust is maintaining strict access control. This concept is critical to prevent attackers from pivoting laterally and elevating access within an environment.

At Microsoft, we define Zero Trust around universal principles.

• Verify Explicitly: Always authenticate and authorize based on all available data points, including user identity, location, device health, service or workload, data classification, and anomalies.


• Use Least Privileged Access: Limit user access with just-in-time and just-enough-access (JIT/JEA), risk-based adaptive policies, and data protection to secure both data and productivity.


• Assume Breach: Minimize blast radius for breaches and prevent lateral movement by segmenting access by the network, user, devices, and app awareness. Verify all sessions are encrypted end to end. Use analytics to get visibility, drive threat detection, and improve defenses.

These principles are technology-agnostic and aligned to six Zero Trust pillars.

Zero Trust Defined

• Identity: Whether they represent people, services, or IoT devices—define the Zero Trust control plane. When an identity attempts to access a resource, verify that identity with strong authentication, and ensure access is compliant and typical.


• Endpoints: Once an identity accesses a resource, data can flow to different endpoints—from IoT devices to smartphones, BYOD to partner-managed devices, and on-premises workloads to cloud-hosted servers. This diversity creates a massive attack surface area. Monitor and enforce device health and compliance for secure access.


• Data: Ultimately, security teams are protecting data. Where possible, data should remain safe even if it leaves the devices, apps, infrastructure, and networks the organization controls. Classify, label, and encrypt data, and restrict access based on those attributes.


• Apps: Applications and APIs provide an interface for data consumption. They may be legacy on-premises, lifted-and-shifted to cloud workloads, or modern SaaS applications. Apply controls and technologies to discover shadow IT, ensure appropriate in-app permissions, gate access based on real-time analytics, monitor for abnormal behavior, control user actions, and validate secure configuration options.


• Infrastructure: Infrastructure—whether on-premises servers, cloud-based VMs, containers, or micro-services—represents a critical threat vector. Assess for version, configuration, and JIT access to harden defense. Use telemetry to detect attacks and anomalies, and automatically block and flag risky behavior and take protective actions.


• Network: All data transits over network infrastructure. Networking controls can provide critical controls to enhance visibility and prevent attackers from moving laterally across the network. Segment networks and deploy real-time threat protection, end-to-end encryption, monitoring, and analytics.

What is Trusted Internet Connections (TIC3.0)?

Trusted Internet Connections (TIC) is a federal cybersecurity initiative to enhance network and perimeter security across the United States federal government. The TIC initiative is a collaborative effort between the Office of Management and Budget (OMB), the Department of Homeland Security (DHS), Cybersecurity and Infrastructure Security Agency (CISA), and the General Services Administration (GSA). The TIC 3.0: Volume 3 Security Capabilities Handbook provides various security controls, applications, and best practices for risk management in federal information systems.

Is Zero Trust Equivalent to TIC 3.0?

No, Zero Trust is a best practice model and TIC 3.0 is a security initiative. Zero Trust is widely defined around core principles whereas TIC 3.0 has specific capabilities and requirements. This workbook demonstrates the overlap of Zero Trust Principles with TIC 3.0 Capabilities. The Azure Sentinel Zero Trust (TIC 3.0) Workbook demonstrates best practice guidance, but Microsoft does not guarantee nor imply compliance. All TIC requirements, validations, and controls are governed by the Cybersecurity & Infrastructure Security Agency. This workbook provides visibility and situational awareness for security capabilities delivered with Microsoft technologies in predominantly cloud-based environments. Customer experience will vary by user and some panels may require additional configurations for operation. Recommendations do not imply coverage of respective controls as they are often one of several courses of action for approaching requirements which is unique to each customer. Recommendations should be considered a starting point for planning full or partial coverage of respective requirements.

Deploying the Workbook

It is recommended that you have the log sources listed above to get the full benefit of the Zero Trust (TIC3.0) Workbook, but the workbook will deploy regardless of your available log sources. Follow the steps below to enable the workbook:

Requirements: Azure Sentinel Workspace and Security Reader rights.

1) From the Azure portal, navigate to Azure Sentinel

2) Select Workbooks > Templates

3) Search Zero Trust and select Save to add to My Workbooks

Navigating the Workbook

The Legend Panel provides a helpful reference for navigating the workbook with respective colors, features, and reference indicators.

Workbook Navigation

The Guide Toggle is available in the top left of the workbook. This toggle allows you to view panels such as recommendations and guides, which will help you first access the workbook but can be hidden once you’ve grasped respective concepts.

Guide Toggle

The Resource Parameter Options provide configuration options to sort control cards by Subscription, Workspace, and Time Range. The Parameter Options are beneficial for Managed Security Service Providers (MSSP) or large enterprises that leverage Azure Lighthouse for visibility into multiple workspaces. It facilitates assessment from both the aggregate and individual workspace perspectives. Time range parameters allow options for daily, monthly, quarterly, and even custom time range visibility.

Resource Parameter Options

The Capabilities Ribbon provides a mechanism for navigating the desired security capabilities sections highlighted in the TIC3.0 framework. Selecting a capability tab will display Control Cards in the respective area. An Overview tab provides more granular detail of the overlaps between the Microsoft Zero Trust model and the TIC3.0 framework.

Capabilities Selector

This workbook leverages automation to visualize your Zero Trust security architecture. Is Zero Trust the same as TIC 3.0? No, they’re not the same, but they share numerous common themes which provide a powerful story. The workbook offers detailed crosswalks of Microsoft’s Zero Trust model with the Trusted Internet Connections (TIC3.0) framework to better understand the overlaps.

TIC 3.0 Overlay to Microsoft Offerings and Zero Trust Principles

The Azure Sentinel Zero Trust (TIC3.0) Workbook displays each control in a Capability Card. The Capability Card provides respective control details to understand requirements, view your data, adjust SIEM queries, export artifacts, onboard Microsoft controls, navigate configuration blades, access reference materials, and view correlated compliance frameworks.

Capability Card

How to Use It?

There are several use cases for the Azure Sentinel Zero Trust (TIC 3.0) Workbook depending on user roles and requirements. The graphic below shows how a SecOps analyst can leverage the workbook to review requirements, explore queries, configure alerts, and implement automation. There are also several additional use cases where this workbook will be helpful:

• Security Architect: Build/design a cloud security architecture to compliance requirements.


• Managed Security Services Provider: Leverage the workbook for Zero Trust (TIC3.0) Assessments.


• SecOps Analyst: Review activity in query, configure alerts, deploy SOAR automation.


• IT Pro: Identify performance issues, investigate issues, set alerts for remediation monitoring.


• Security Engineer: Assess security controls, review alerting thresholds, adjust configurations.


• Security Manager: Review requirements, analyze reporting, evaluate capabilities, adjust accordingly.

SecOps Analyst Use-Case

Configurations & Troubleshooting

It’s important to note that this workbook provides visibility and situational awareness for control requirements delivered with Microsoft technologies in predominantly cloud-based environments. Customer experience will vary by user, and some panels may require additional configurations and query modification for operation. It’s unlikely that all 76+ panels will populate data, but this is expected as panels without data highlight respective areas for evaluation in maturing cybersecurity capabilities. Capability Cards without data will display the custom error message below. Most issues are resolved by confirming the log source’s licensing/availability/health, ensuring the log source is connected to the Sentinel workspace, and adjusting time thresholds for larger data sets. Ultimately this workbook is customer-controlled content, so panels are configurable per customer requirements. You can edit/adjust Control Card queries as follows:

• Zero Trust (TIC3.0) Workbook > Edit > Edit Panel > Adjust Panel KQL Query > Save

Custom Error Messaging

While using Microsoft offerings for the Zero Trust (TIC3.0) Workbook is recommended, it’s not a set requirement as customers often rely on many security providers and solutions. Below is a use-case example for adjusting a Control Card to include third-party tooling. The default KQL query provides a framework for target data, and it is readily adjusted with the desired customer controls/solutions.

3rd Party Tool Use-Case

Get Started with Azure Sentinel and Learn More About Zero Trust with Microsoft

Below are additional resources for learning more about Zero Trust (TIC3.0) with Microsoft. Bookmark the Security blog to keep up with our expert coverage on security matters and follow us at @MSFTSecurity or visit our website for the latest news and cybersecurity updates.

• Get Started with Azure Sentinel


• Tutorial: Visualize and Monitor your Data with Workbooks


• Microsoft Zero Trust Model


• Zero Trust Deployment Center


• Zero Trust: 7 Adoption Strategies from Security Leaders


• Microsoft Zero Trust Video

Disclaimer

The Azure Sentinel Zero Trust (TIC 3.0) Workbook demonstrates best practice guidance, but Microsoft does not guarantee nor imply compliance. All TIC requirements, validations, and controls are governed by the  Cybersecurity & Infrastructure Security Agency. This workbook provides visibility and situational awareness for control requirements delivered with Microsoft technologies in predominantly cloud-based environments. Customer experience will vary by user, and some panels may require additional configurations and query modification for operation. Recommendations do not imply coverage of respective controls as they are often one of several courses of action for approaching requirements which is unique to each customer. Recommendations should be considered a starting point for planning full or partial coverage of respective control requirements.

7 Steps to Successful Reflect Adoption

by Contributed | May 5, 2021 | Technology

This article is contributed. See the original author and article here.

As schools begin to reopen across the world, concern about student wellbeing is at the forefront of many discussions. Educators want to help students recognize and navigate their emotions by providing regular opportunities to share and be heard.

This need inspired us to develop Reflect in Microsoft Teams for Education.

Reflect can help broaden learners’ emotional vocabulary and deepen empathy for their peers while also providing valuable feedback to educators for a healthy classroom community.

Whatever your role in the educational community (educator, school leader, or part of the wellbeing team), this blog post will show you how to support student wellbeing by encouraging reflective conversations in your school and making emotional check-ins a part of your routine.

Decades of research on the science of learning, and the important role emotions play in how we humans learn and process information, has finally caught up with what most educators have always known – students need more than academics to successfully navigate their way through this complex world they have inherited. Social Emotional Learning (SEL) is about treating a student as a whole person, emotions and all, and cultivating the skills we all need to become engaged citizens leading happy, healthy, successful lives.

CASEL (Collaborative for Academic, Social, and Emotional Learning) defines SEL as:

The process through which children and adults acquire and effectively apply the knowledge, attitudes, and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions.

Once we understand SEL, we can implement it throughout the school community. One tool that can help you is the new Reflect app in Microsoft Teams, and here are some tips on how to start.

Student journal in Reflect

What is Reflect, and why it’s used

We want to build a school community that recognizes the whole student. It is important that we support them emotionally, understand when they are going through a rough time, and ensure that the classroom environment stays positive and conducive to learning.

Reflect is an emotional check-in app that helps educators support their students and the class as a whole. Reflect helps students recognize and navigate their emotions by providing regular opportunities to share and be heard. Reflect can help broaden a student’s emotional vocabulary and deepen empathy for their peers while also providing valuable feedback to educators for a healthy classroom community.

This check-in app uses emojis and research-backed emotional granularity to support educators in adding social and emotional learning and support into their routine.

Schools are reopening, Reflect is as valuable as ever

The future is unpredictable. Preparing youth for the challenges of tomorrow in the ambiguity of today is a hurdle that educators have taken in their stride. Social-emotional skills help us better navigate complexity and uncertainty while minimizing the adverse effects of disruption and sustaining the relationships that support the students. The past year’s changes have highlighted the importance of personal connection, accelerated the integration of technology in the classroom, and amplified the role of teachers. Adjusting back to in-person learning is a disruption in itself. Students will need support in re-establishing their bonds, noticing social cues, and communicating their needs to their peers and educators. A regular SEL routine like Reflect can provide an entry point for educators to host classroom conversations and a safe space for students to voice their concerns.

Privacy and Security

Reflect follows the same privacy and security standards as Education Insights to protect students’ sensitive information.

The information collected and shown through Reflect meets more than 90 regulatory and industry standards, including GDPR and the Family Education Rights and Privacy Act (FERPA) for students and children’s security and other, similar, privacy-oriented regulations.

Students never see the names of other students, only how they responded. While they can see the distribution of responses, they cannot see the student names associated with each reflection.

Steps to successful Reflect adoption

1. Enable Reflect

You can locate the Reflect app through the Teams apps gallery or using this link: https://aka.ms/getReflect.
If you don’t find the Reflect app in the gallery or if the link is not accessible, it may be that your IT Admin has limited the app in your tenant. In such a case, please refer your IT Admin to the following document: IT Admin Guide to Reflect in Microsoft Teams.

Only when Reflect is allowed can educators see and use the app in their classes.

Guest users (students or educators) cannot use Reflect.

2. Work with educators, so they understand the importance of regular check-ins and emotional granularity

Reflect helps educators easily check how their students feel in general or about a specific topic, such as learning from home, an assignment, current events, or a change within their community.

Emotional granularity is the ability to differentiate between emotions and articulate the specific emotion experienced.  Developing language to talk about feelings is foundational to social and emotional learning. Accurately defining our emotions can help identify the source of the feeling and plan to meet our needs.

Introduce your educators to the app and show them how it can support a positive emotional climate in the classroom. Reflect provides an emotional safe space to grow student-teacher connections and help students develop their emotional vocabulary.  We worked with SEL experts to select 50 emotion words that students can identify with as they reflect.

3. Ask educators to try Reflect and share their feedback

The first step is always the hardest. Reflect provides a variety of questions educators can ask their class for some common scenarios. The questions are a closed list, developed with the help of experts, and designed to provide Insights when associated with other data. One way to customize a Reflect check-in is to publish it and then “reply” to the message with specific guidelines.

Ask the educators to share their experiences using Reflect so that you can support them and their specific needs.

Choose a check-in question in Reflect

We have created a resource page for educators that includes instructions, short videos, and a free course on getting started.

Listen to their feedback, and work with them to find what works best for their classes.

* The capacity to schedule repeat check-ins and access detailed data for Reflect in Insights will be available summer of 2021

4. Ask educators to use Reflect as part of their routine

Encouraging communication about emotions and supporting students in voicing their needs can go a long way toward preventing feelings of isolation, frustration, and disengagement. Once educators understand how Reflect supports them, suggest to them that they use it as part of their routine.

Here are some suggestions for using Reflect in the classroom.

• Implement a daily check-in to provide students with opportunities to practice evaluating and naming their emotions and ensure educators have a touchpoint with every student every day.


• Jumpstart conversations with students after a difficult day by asking What does the mood in our classroom feel like? This can be an excellent opportunity to model self-reflection and set goals together as a class.


• To support learning, assess student confidence about a concept or assignment, and plan accordingly.


• Celebrate wins with a Reflect check-in after a successful activity. Remember, mistakes can be framed as successes too!


• Identify dynamics between students by asking How do your friendships feel today? This can be paired with conversations or stories about sharing, bullying, and healthy relationships.

5. Empower the educator

The educator has complete control of when (or even if) to use Reflect. Only they should decide whether to post a new Reflect check-in, when and what type. If an educator feels it is not suitable for their classroom or they cannot support the students that way, they will not publish a new check-in, and you need to support them in that decision.

6. Don’t set goals for students

There is no right or wrong answer when dealing with emotions. All feelings are valid, and we want students to provide an honest assessment from their perspective and feel safe when describing their emotions. The goal is to develop a deeper understanding of student needs and work together to create a more positive climate and foster greater well-being in class.

We want to make the classroom a place where student emotions are valued, and everyone feels safe and heard.

7. Use Insights reports

Through Education Insights, educators can see how students respond to check-ins over time, making it easier to identify and address student needs.

The digital activity report now includes Reflect check-in data, with additional Reflect insights coming soon.

Education Insights – Reflect check-ins in the digital activity report

Educators can also track the adoption of the tool amongst their students to make sure they feel comfortable sharing their emotions. If there is low adoption, you may want to provide a framework to help establish the classroom as a safe space to share and support them in developing the emotional vocabulary needed to feel confident in sharing.

If you have Education Insights Premium (currently in preview), you can also see adoption across classes, grade level, school, etc., in your organizational view, and provide support and assistance for those who either don’t use Reflect at all or stopped using it after the initial adoption.

We’re always looking for ways to make Education Insights and Reflect better. Have questions, comments, or ideas? Let me know! Add your idea here, share your comment below, and find me on Twitter (@grelad).

Elad Graiver
Senior Program Manager, Education Insights and Reflect

A Visual Guide to #SustainabilityCity: Let's Talk Minecraft!

by Contributed | May 5, 2021 | Technology

This article is contributed. See the original author and article here.

A Visual Guide To #Sustainability City

Welcome to the second in my series of visual guides focused on the topic of Sustainability and Green Tech. The first one focused on a Visual Guide To Sustainable Software Engineering, visualizing the core philosophies and eight principles of sustainable software engineering, as defined by this fantastic Microsoft Learn Module.


Putting Philosophy Into Practice
In fact, it was that first philosophy (“Everyone has a part to play in the climate solution”) that inspired me to work with the Microsoft Green Cloud Advocacy team on the #VisualGreenTech challenge for EarthDay and co-host a special Earth Day themed episode of #HelloWorld featuring Green Tech experts from Microsoft.

The challenge itself featured 24 prompts, three of which explored Sustainability interactively using Minecraft Education Edition resources for Earth Day. The first of these prompts focused on Sustainability City – a Minecraft world where you can take green buses around a bustling city, visiting various facilities to learn about sustainable practices targeting water treatment, food production, sustainable forestry, green buildings, energy-efficient homes, and the power grid. This community-created video does a great job of navigating the world in under eight minutes.

It was there that my personal journey into putting that philosophy to practice began!

Using Minecraft To Motivate Sustainability Education

Minecraft is an amazing resource for educating K-12 students on sustainability in actionable ways. The Minecraft Education Edition provides downloadable words that students can navigate and it has detailed lesson plans with activities and discussion guides to help students go from awareness to engagement, and action.

In my case, I took advantage of our Microsoft employee access to the Minecraft Edu Edition to begin a sustainability city journey with my 12-year-old. We took every bus, talked to every character, and had interesting follow-up conversations like: where does our water come from? how can we be more sustainable at home during the pandemic? and my favorite: should we create our own vegetable garden so we know where our food comes from?

If you are a parent, I strongly advocate for doing this exploration with kids and using the visual guide below to have a conversation once you leave the world. If you are an educator with access to this edition of Minecraft, I hope you find this visual guide a good resource for classroom conversations or continued awareness of what they learned, once they have completed that lesson.

Visual Guide & Navigation

Here is the visual guide to Sustainability City. You can find a hi-res downloadable version of this visual guide here – warning: this is a large file (13MB) so make sure you have the data/bandwidth to download it. See this tweet for a behind-the-scenes time-lapse replay of how it was created.

The visual guide has six sections, each mapping to one of the regions of Sustainability City. Start from the top left and work your way clockwise to the last one. Here is what you’ll learn. Start with  Food Production to explore sustainable farming practices including water reclamation and composting. Next, travel to the Water Outflow Reclamation Facility to learn how water from sewers and drains is “cleaned” and used for irrigation or returned to source (water positive) – the removed biosolids become fodder for composting.

Then, explore sustainable practices in the construction of large buildings (make them self-sustaining in energy needs) and explore sustainable forestry practices required to support our lumber needs. Finally, we look at energy-efficient housing and explore the power grid in some detail. As we know, electricity is a proxy for carbon, and understanding the various ways we generate, transport, and use, energy is critical to sustainability education.

I hope you found the guide useful. Making visual guides takes time but is infinitely rewarding. Have comments or feedback? Do leave them below.

Storytelling for Champions – new resources available

by Contributed | May 5, 2021 | Technology

This article is contributed. See the original author and article here.

We are pleased to share a set of new resources on storytelling for Champions! 

Our overall goal for delivering these new resources is to enable you and your organization to achieve greater results from your adoption and change management programs. Stories are an amazing way to connect with people through shared experiences, inspire others to act, and are an influential way to communicate. Storytelling can play a key role in any change journey as a powerful way to showcase examples of the new behavior change in action.  

I have worked with enterprise customers at Microsoft for nearly two decades and have seen the impact of great adoption success stories when they are shared by champion business users who have realized the positive impact of change. This has helped them grow and enhance executive sponsorship, lowered resistance to change among later adopters, inspired new ideas, and reinforced the value of the change program.

The resources include:

• A new storytelling guide that includes:




• A framework for finding and growing good success stories: Empower, Capture, Amplify, Learn;


• A sample structure for capturing compelling stories: Situation, Complication, Resolution, The Point;


• And additional learning resources.




• A storytelling course on LinkedIn Learning where you can also earn a certificate for completion.

We encourage you to take a look and leverage with your teams. We also welcome your feedback here on how we can improve this and other resources on the site to keep inspiring your communities to achieve more!

​Access the content at storytelling for Champions.