by Contributed | Aug 14, 2021 | Technology
This article is contributed. See the original author and article here.
Azure Synapse Analytics Data Flows has enabled Direct Workspace DB Connector as a public preview. This new connector type in data flows enables data engineers to quickly and easily build ETL processes using Spark-based lake databases in Synapse without the need to first create linked services or datasets.
https://docs.microsoft.com/en-us/azure/data-factory/data-flow-source#workspace-db-synapse-workspaces-only
https://docs.microsoft.com/en-us/azure/data-factory/data-flow-sink#workspace-db-synapse-workspaces-only
By using Synapse Analytics for your end-to-end big data analytics projects, you can now define lake database tables using Spark Notebooks, then open the visual data flow designer graph environment and immediately access those tables and data for ETL pipeline building. This new Workspace DB process eliminates the need to create ADF-based linked services and datasets inside of your Synapse workspace studio UI because Synapse is providing the complete integrated experience for data engineers in a single pane of glass.
by Contributed | Aug 13, 2021 | Technology
This article is contributed. See the original author and article here.
Join us on Thursday, August 19th at 10 AM PDT at https://aka.ms/sfcommunityqa!
In addition to our normal Q&A in each community call, we will focus on topics related to various components of the Service Fabric platform, provide updates to the roadmap, upcoming releases, and showcase solutions developed by our customers that benefit the community.
Agenda:
- Demo on node tagging
- Demo on BYOVNet on SFMC
- Q&A
If you would like to suggest a topic or provide a talk please reach out or let us know in a call. We try to focus on topics related to various components of the Service Fabric platform, provide updates about new features and services, upcoming releases, and highlight solutions developed by our customers that benefit the community. We have posted recordings of all our past Service Fabric Community call here. You can also tell us ahead of time what topics you want to be covered or questions you want to be addressed by submitting ideas here at https://aka.ms/sfcommunityideas.
Though it’s not required, feel free to RSVP and comment “Attending” or “Maybe” if you’re thinking of joining. Please come prepared with all your questions – we look forward to seeing you there!
by Contributed | Aug 13, 2021 | Technology
This article is contributed. See the original author and article here.
Are you enjoying the summer or winter – wherever you are in the world, and want to keep up to date with the latest and greatest? We can help you cure that need :smiling_face_with_smiling_eyes:
Over the past few months, we have made big product announcements across the Microsoft Defender products and Microsoft Cloud App Security, and of course we want you to stay updated!
With the following resources you can bring yourself up to speed, and with the knowledge check at the end you can verify your learnings. Plus, you can request either a Ninja summer or winter special edition fun certificate to enrich your Ninja certs collection!
Legend:
 Product videos
|
 Webcast recordings
|
 Tech Community
|
 Docs on Microsoft
|
 Blogs on Microsoft
|
 GitHub
|
⤴ External
|
 Interactive guides
|
|
Microsoft Defender for Endpoint
Unmanaged devices
Mobile threat defense
Threat and vulnerability management
Device control
Live response
Evaluation Lab
Microsoft 365 Defender:
Threat Analytics
Advanced hunting
Integration and APIs
Webinar: Monthly threat insights: New webinar series: Monthly threat insights – Microsoft Tech Community
Defender for Office 365:
Phishing protection
Business Email Compromise
Incident investigation
Configuration
Threat Analytics
Attack Simulation Training
Defender for Identity:
General:
Portal Convergence:
Detections
Identity Security Posture Management assessments
Cloud App Security:
3rd Party Integration
Threat Protection
Conditional Access App Control
Data Loss Prevention
If you want to verify your learnings, you can participate in this knowledge check.
Once you’ve finished the knowledge check, please click here to request your certificate (you’ll see it in your inbox within a couple of days.)
Let us know how you like it!
As a reminder, the full Ninja Trainings are here:
Microsoft 365 Defender > http://aka.ms/m365dninja
Microsoft Defender for Office 365 > https://aka.ms/mdoninja
Microsoft Defender for Endpoint > http://aka.ms/mdeninja
Microsoft Defender for Identity > http://aka.ms/mdininja
Microsoft Cloud App Security > http://aka.ms/mcasninja
by Contributed | Aug 12, 2021 | Technology
This article is contributed. See the original author and article here.
By: Zoe Statman-Weil & Mark Mathis, Impact Observatory, Inc.
Global decision makers need timely, accurate maps
Land use and land cover (LULC) maps are used by decision makers in governments, civil society, industries, and finance to observe how the world is changing, and to understand and manage the impact of their actions. Historically, LULC maps are produced using expensive, semi-automated techniques requiring significant human input and thus leading to significant delays between collection of satellite images and production of maps, limiting the ability to get regular and frequent temporal updates to users. Making the detailed, accurate maps the whole world needs to understand our rapidly changing planet with timely updates requires automation. A groundbreaking artificial intelligence-powered 2020 global LULC map was produced for Esri on Microsoft Azure by Impact Observatory, a mission-driven technology company bringing AI algorithms and on-demand data to environmental monitoring and sustainability risk analysis. This map will be used to help decision makers address challenges in climate change mitigation and adaptation, biodiversity preservation, and sustainable development.
The Impact Observatory LULC machine learning (ML) model was trained on an Azure NC12s v2 virtual machine (VM) powered by NVIDIA® Tesla® P100 GPUs using over 5 billion pixels hand-labeled into one of ten classes: trees, water, built area, scrub/shrub, flooded vegetation, bare ground, cropland, grassland, snow/ice, and clouds. The model was then deployed over more than 450,000 Copernicus Sentinel-2 Level-2A 10-meter resolution, surface reflectance corrected images, each 100 km x 100km in size and totaling 500 terabytes of satellite imagery (1 terabyte = 1012 bytes) hosted on the Microsoft Planetary Computer. The processing leveraged geospatial open standards, Azure Batch, and other Azure resources to efficiently produce the final dataset at scale and at a low cost.
Geospatial Open Standards support distributed processing
The Microsoft Planetary Computer and Impact Observatory (IO) make extensive use of geospatial open standards, specifically Cloud Optimized GeoTIFF (COG) and Spatial Temporal Asset Catalog (STAC). Use of these standards enabled the team to produce the Esri 2020 Land Cover map using distributed processing at scale.
GeoTIFF is a widely used open standard for geospatial data based on the common TIFF image file format, able to support imagery with bands beyond the usual red, green, blue visible light bands, and containing additional metadata to locate the image on the surface of the Earth. A COG is a regular GeoTIFF file, aimed at being hosted on a HTTP file server, with an internal organization that enables more efficient workflows on the cloud. Not only can COGs be read from the cloud without needing to duplicate the data to a local filesystem, but a portion of the file can be read using HTTP GET Range requests allowing for targeted reading and efficient processing. Azure Blob Storage is an ideal solution for hosting COGs as it is an unstructured data storage system accessible via HTTP requests. The LULC map was produced using Sentinel-2 COGs hosted on Microsoft’s Planetary Computer in Blob Storage, and all prediction rasters produced from the model were saved as COGs to Blob Storage.
The STAC specification is a common language used to index geospatial data for easy search and discovery. IO searched the Planetary Computer’s STAC catalog to identify Sentinel-2 imagery for certain locations, times, and cloud coverage. IO applied a community supported implementation of the STAC interface to create its own STAC catalog on Azure App Services with Azure Database for PostgreSQL as the underlying data store. IO’s STAC catalog was used to index data throughout the model deployment pipeline and thus served as both a tool for checkpointing pipeline progress, as well as indexing the final product.
COGs and STAC, both easily leveraged in Azure, provide a scalable and highly flexible framework for processing geospatial data.
Azure Batch enabled Impact Observatory to map the globe at record scale & speed
Azure Batch was used by IO to efficiently deploy the model over satellite images in parallel at a large scale. IO bundled the ML model, and deployment and processing code into Docker containers, and ran Batch tasks within these containers on a Batch pool of compute nodes.
The data processing pipeline consisted of three primary tasks: 1) Deploying the model over one 100 km x 100 km Sentinel-2 COG by chipping it into hundreds of overlapping 5 km X 5 km smaller images, running those chips through the model, and finally merging the chips back together; 2) Computing a class weighted mode across all model predictions for a given Sentinel-2 image footprint; and 3) Combining the class weighted modes produced in #2 for a given Military Grid Reference System (MGRS) zone into one COG. IO relied heavily on Batch’s task dependency capabilities, which allowed, for example, for the class-weighted mode task (#2) to only be scheduled for execution when the relevant set of model deployment tasks (#1) were completed successfully.
While the model was trained on a GPU-enabled VM, the model deployment over the image chips was executed on CPU-based virtual machines, enabling resource efficient computation at scale. Due to the task dependent nature of the pipeline, all tasks needed to be run on the same pool, and thus the same VM type. RAM and network bandwidth requirements fluctuated for the tasks, but the high CPU usage ended up being the defining factor in VM choice. In the end, the data was processed on low-priority Standard Azure D4 V2 virtual machine powered by Intel® Xeon® scalable processors with seven task slots allocated per node.
It took over one million core hours to process the data for the entire LULC map. With the scaling flexibility of Batch, IO was able to process over 10% of the earth’s surface a day. The completed Esri 2020 Land Cover map is now freely available on Esri Living Atlas and the Microsoft Planetary Computer.
For additional information visit https://www.impactobservatory.com/
by Contributed | Aug 11, 2021 | Dynamics 365, Microsoft 365, Technology
This article is contributed. See the original author and article here.
With the back-to-school season just around the corner, school and university administrators, educators, parents, and students are preparing for another challenging year. The August release of Dynamics 365 education accelerator includes updated data model and sample apps that can help partner ISVs continue to build and deliver solutions for schools and universities.
In addition, the K-12 portal, an information hub built on Microsoft Power Apps, has been redesigned to make it easier for parents and students to track school activities and assignments. Plus, the release includes a data model aligned with Microsoft School Data Sync (SDS), an application offered with Microsoft 365 Education.
What’s new in the August release
- Higher education event marketing allows education institutions to leverage Dynamics 365 Marketing capabilities to reach students and families in meaningful ways.
- K-12 portal update includes an improved interface of the student portal for K-12 schools and a better experience for all users.
- School Data Sync (SDS) data model alignment makes it easier for partners to build connectors and for educational institutions to leverage the ability to flow data from SDS to Microsoft Dataverse.
Take advantage of event management capabilities in Dynamics 365 Marketing
With the August release of Dynamics 365 education accelerator, partners can leverage Dynamics 365 Marketing to bring the power of personalized communication and campaigns to education customers around the world. Partners will help customers enable targeted communication, direct customer journey, and ensure ongoing engagement with their institution.
This capability can be used to build campaigns for a variety of school events, including back-to-school orientation, family night, or school tours. Schools and universities can create events where participants can register and communicate with the institution, and schools can view attendance. Best of all, this capability is completely flexible for partners to customize.
Improved usability of the K-12 student portal
The K-12 student portal has received a design uplift to improve user experience.
School Data Sync (SDS) data model alignment transforms data gathering
School Data Sync (SDS), an offer from Microsoft 365 Education, reads rosters from most Student Information Systems (SIS) and creates classes and groups in Microsoft Teams, Intune for Education, and third-party applications.
By aligning the data model between the education accelerator and SDS, we will enable the possibility for customers already using SDS to leverage the powers ofDataverse, the Common Data Model for Education, and Power Platform.This is transformational from how data was gathered before and allows partners to build specific connectors for their customers.
Eduphoric is one such partner, and we are excited to see how customers will benefit from this work. According to Pamela Knott, Eduphoric’s director of education technology, harnessing the advantages of the Power platform help ” educators to support more students, more effectively in less time.” Knott says Eduphoric’s Learning Team solution ” aggregates attendance, behavior, course performance, wellness, and tech readiness data into one holistic student profile so that teachers, counselors, administrators and support specialists can identify struggling students and work together to get them back om track.”
What other partners are saying about Dynamics 365 education accelerator release
Microsoft’s partners who have seen a sneak peek are excited about the power and flexibility the August release provides:
- Thomas Manders, managing director for Coffee + Dunn: “With many of our clients in the education space, we understand schools’ needs to increase communications efficiency, improve the student relationships and innovate for the future of education. Serving as a robust foundation, the accelerator helps schools eliminate data silos, enable powerful insights and act on these insights to deliver exceptional student and alumni experiences. As a partner, we can help our clients achieve these goals much faster than starting organically.”
- Shekar Kadaba, chief experience officer for Frequency Foundry: “Over the last couple of releases, the Industry Accelerator team had responded to the market need for supporting and engaging students in different learning modes by adding additional functionality to support the student lifecycle. We see that envelope being been further pushed with the latest release.”
- Dr. Jennifer Beyer, vice president of product management for Anthology: “Our college and university partners have been able to leverage D365 Marketing as part of our solution to increase their intentional connections with their students whether it is driving them to complete their admissions application or keeping current students on track.”
Next steps
We encourage you to register andtest drive the education accelerator. You can accessGitHubto try out the solution, data samples, Power BI examples, and more.
If you have any questions,don’t hesitate to reach out to our team.If you’re a partner working in the higher education space and want to use the accelerator and help shape the next version of it, pleasereach out to us.
The post Dynamics 365 education accelerator helps partners deliver solutions for educators and students appeared first on Microsoft Dynamics 365 Blog.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments