SharePoint at 25: How Microsoft is putting knowledge to work in the AI era

SharePoint at 25: How Microsoft is putting knowledge to work in the AI era

This article is contributed. See the original author and article here.

Twenty-five years ago, SharePoint set out to help people share knowledge and work better together, a mission that today operates at extraordinary scale.

The post SharePoint at 25: How Microsoft is putting knowledge to work in the AI era appeared first on Microsoft 365 Blog.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Building Smarter Observability for Agentic ERP World using Dynamics 365 

Building Smarter Observability for Agentic ERP World using Dynamics 365 

This article is contributed. See the original author and article here.

As enterprise workloads become more agentic, the expectations of ERP systems—and the teams that operate them—are shifting. Batch jobs, workflow orchestration, data import/exports, and background processes are no longer “just” technical plumbing–they are critical pieces of the operational fabric. They deliver timely financial results, accurate supply chain data, and reliable business intelligence driving process optimization. 

To support this shift, observability needs to evolve beyond simple logs and reactive troubleshooting. Observability needs to provide meaningful insights into execution behavior, performance patterns, and operational context. This ensures IT teams can run ERP with confidence and reliability. 

In Dynamics 365 ERP apps, we’ve long provided integration with Azure Application Insights to help organizations collect telemetry about user activity, failures, and application behavior. Now, with the expansion of batch telemetry signals — including start/stop events, failure data, throttling conditions, thread availability, and queue behavior — administrators and IT architects can gain deeper visibility into the health of critical batch-based workloads.  

Why Observability Matters Now 

ERP observability historically focused on basic monitoring. It observed which jobs were running, whether a job failed, or whether alerts were triggered. These indicators are useful, but they lack operational context. Modern enterprise workloads are increasingly interconnected, and automation driven. Delays or failures in one workload can ripple outward, affecting downstream processes, reporting accuracy, and service delivery. 

At the same time, teams are beginning to rely on AI agents to help monitor, diagnose, and in some cases suggest remediation steps. These tools need high-quality signals to be effective. 

Batch workloads are a prime example. Batch jobs directly impact business outcomes, from overnight posting to inventory sync and settlements.
Without execution insights, teams guess root causes and waste time on manual investigation.

What Batch Telemetry Brings to the Table 

The monitoring and telemetry capabilities in Dynamics 365 ERP enable customers to send application telemetry to Azure Application Insights for analysis and alerting. The recent expansion of telemetry signals for batch workloads builds on this foundation by adding behavioral data specifically for batch execution patterns. 

These signals include: 

  • Batch start and stop events to show how long jobs take to run, not just whether they completed. 
  • Failure information that correlates with info log entries and execution context. 
  • Throttling indicators that highlight contention due to system load. 
  • Thread availability data that helps reveal when jobs are waiting because capacity is constrained. 
  • Queue depth metrics shows number of waiting tasks for all queues that are part of the Priority Based Scheduling queues.  

Emitting these signals into a customer-owned Application Insights resource means teams can apply their existing monitoring pipelines, dashboards, and alerting logic without changing how data is consumed. 

From Visibility to Insight 

Once batch telemetry data flows into Application Insights, teams can query it using Kusto Query Language (KQL) and build dashboards that correlate workload behavior with other operational metrics.  

This richer observability enables several practical outcomes: 

  • Faster investigation of execution behavior without sifting through logs. 
  • Trend analysis to detect regressions or capacity bottlenecks before they impact business cycles. 
  • More informed capacity planning based on actual observed patterns. 
  • Alignment of SLA expectations with real operational performance. 

Here are some real‑world business scenarios that show how telemetry insights are helping customers troubleshoot issues and resolve problems faster. 

A global consumer goods company frequently sees high priority jobs completing late. Batch Queue telemetry exposes queue congestion and thread exhaustion, showing when noncritical tasks bury priority workloads.

It helps surface when priority-based scheduling queues build up and delay time‑sensitive workloads, while also revealing misconfigured priorities that cause jobs to be processed out of order. It further enables teams to closely monitor queue health during cutover or high‑load events, ensuring critical workloads flow smoothly. 

Similarly, a finance team’s bank reconciliation jobs remain “Waiting” for long periods. Thread telemetry reveals thread starvation—jobs were queued, but threads were fully consumed. 

It helps explain why jobs remain stuck in a “Waiting” state by revealing when thread capacity is fully consumed by parallel workloads. It also highlights thread saturation patterns, enabling teams to right‑size AOS batch capacity for smoother, more predictable processing. 

A Foundation for Intelligent Operations 

The expanded telemetry signals are not just a diagnostic tool. They serve as a foundation for smarter operations in an era where agents play an increasing role. High-fidelity Batch telemetry enables experiences like: 

  • Automated detection of anomalies based on execution baselines. 
  • Correlation of workload performance with business-critical thresholds. 
  • Enhanced alerts that tie operational conditions to business impact. 

By making execution behavior more observable and actionable, Dynamics 365 ERP helps teams focus on outcomes, not just symptoms. 

Getting Started 

If you haven’t already configured monitoring and telemetry for your environment, the first step is to integrate your Dynamics 365 ERP instance with Azure Application Insights – refer. Monitoring and telemetry overview – Finance & Operations | Dynamics 365 | Microsoft Learn .  

Once telemetry is configured, expanded batch signals can be toggled on from within system administration and begin flowing to your Application Insights pipeline for analysis.  

Rich observability is a core requirement for running modern ERP workloads, especially as organizations adopt more automation and begin exploring agent-assisted operational tooling. By bringing deeper insight into batch execution behavior, our ERP portfolio apps in Dynamics 365 helps IT teams move from reactive troubleshooting toward proactive reliability and informed decision-making.  

For more details visit Available telemetry – Finance & Operations | Dynamics 365 | Microsoft Learn

The post Building Smarter Observability for Agentic ERP World using Dynamics 365  appeared first on Microsoft Dynamics 365 Blog.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP

Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP

This article is contributed. See the original author and article here.

Enterprise resource planning (ERP) decisions are among the most consequential investments a business can make; shaping how organizations operate, scale, and compete for years to come. Yet many ERP transformations have historically carried risk: high costs, long timelines, heavy customization, and uncertain returns.

To bring clarity to these decisions, Microsoft commissioned Forrester Consulting to conduct two independent Total Economic Impact™ (TEI) studies examining the business value of Microsoft Dynamics 365 ERP on enterprises and midmarket organizations.

From fragmentation to integration: Why ERP modernization matters

Across both 2026 studies, Forrester’s projections showed organizations from a similar place: fragmented ERP landscapes, siloed data, manual processes, and highly customized legacy systems that were difficult to upgrade or scale. These environments could limit real-time visibility, slow decision-making, and increased operational risk; particularly as organizations grew, expanded into new markets, or managed increasingly complex supply chains. These constraints didn’t just slow operations; they could limit a leader’s ability to respond to volatility, growth, and supply chain disruption with confidence.

In response, organizations turned to Microsoft Dynamics 365 ERP to consolidate finance and supply chain operations with a unified, cloud-based platform. By centralizing data and standardizing processes, organizations can improve operational efficiency and gain timely, actionable insights across the business.

Importantly, this shift reframed ERP from a back-office system of record to a platform for informed, faster decision-making that connects data, people, and processes across the enterprise.

Quantifying business value with Forrester’s TEI methodology

The strength of the TEI studies lies in their focus on quantifiable business impact. Forrester evaluated benefits, costs, flexibility, and risks over a multi‑year period, modeling a composite organization based on real customer interviews and survey responses. This approach allows leaders to evaluate ERP investments using a transparent financial framework rather than vendor claims alone.

Enterprise ERP: Financial impact at scale

In the enterprise TEI study, Forrester modeled a composite organization representing large, complex businesses using Dynamics 365 ERP. The analysis projects that over three years, the organization achieved:

  • 101% return on investment (ROI)
  • Net present value (NPV) of $12.9 million

These results were driven by a combination of operational efficiency gains, productivity improvements, and cost reductions, particularly from consolidating legacy systems and reducing infrastructure and IT operations spend.

The study highlights that value did not come from isolated features, but from standardizing processes, unifying data across finance and supply chain functions, and reducing reliance on heavily customized, on-premises ERP environments.

Key enterprise findings business leaders should note

For business decision makers evaluating ERP at scale, several findings stand out:

  • Improved operational efficiency and productivity can be enabled through streamlined workflows and better access to real-time insights
  • Reduced infrastructure and IT operations costs can be enabled by retiring multiple legacy systems and shifting to a cloud-based ERP model
  • Faster, more confident decision making enabled by unified financial and supply-chain data

Together, these benefits contributed directly to the projected positive NPV and ROI modeled in the study, reinforcing ERP modernization as a business investment, not just an IT upgrade.

Midmarket ERP: Enterprise-grade value without enterprise complexity

While enterprises face complexity at scale, midmarket organizations often face a different challenge: how to grow without adding disproportionate cost or operational overhead. Forrester’s Total Economic Impact™ study of Microsoft Dynamics 365 ERP for midmarket organizations examined how modern ERP can support expansion, improve visibility, and standardize operations without the burden of traditional enterprise‑scale implementations.

In the study, Forrester modeled a composite midmarket organization based on customer interviews and survey data. The analysis projects that the organization would achieve:

  • Payback in 16 months
  • Net present value (NPV) of $3.3 million over three years

These outcomes were driven by streamlined finance and supply chain operations, automation of manual processes, and the replacement of disconnected legacy systems with a single, cloud‑based ERP platform. By consolidating systems and standardizing processes, organizations can reduce operational friction while supporting improved visibility and control across the business.

Key midmarket findings business leaders should note

For midmarket decision-makers, the study highlights several critical outcomes:

  • Enabled faster time-to-value, with measurable financial returns realized in just over a year
  • Enabled improvements to operational efficiency and productivity through streamlined finance and supply chain processes and automation of manual tasks
  • Potentially reduced complexity and IT overhead by replacing disconnected legacy systems with a unified cloud ERP platform

These benefits contributed directly to the projected positive NPV and rapid payback modeled in the study, reinforcing ERP modernization as a financially disciplined investment for midmarket organizations focused on growth and resilience.

Why independent research matters for ERP decisions

ERP investments shape the future of an organization for years—sometimes decades. That’s why independent, third-party validation is critical. The Forrester TEI studies do not ask leaders to accept conclusions at face value; instead, they provide:

  • A transparent financial model
  • Explicit assumptions and risk adjustments
  • Clear linkage between operational improvements and economic outcomes

For executives, CFOs, COOs, and IT leaders, these studies offer a common language for aligning stakeholders and setting realistic expectations for ERP transformation.

Go deeper: Explore the full Forrester TEI studies

This summary only scratches the surface. The full Forrester Total Economic Impact™ studies include detailed benefit breakdowns, cost considerations, and financial modeling that business leaders can adapt to their own organizations.

For organizations considering ERP modernization, these studies provide a data-driven foundation to evaluate options, build a credible business case, and make informed decisions with confidence.

The post Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP appeared first on Microsoft Dynamics 365 Blog.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP

Announcing Desktop Companion App (DCA) support for Dynamics 365 Contact Center Embedded in Third-Party CRMs  

This article is contributed. See the original author and article here.

We’re excited to introduce Desktop Companion App (DCA) support for Dynamics 365 Contact Center in Embedded mode, delivering lower latency, improved reliability, and resilient voice continuity when Customer Service Reps (CSR) work inside third party (3P) CRM environments. With DCA running alongside the embedded conversation widget, contact centers can maintain active calls even if the browser refreshes or becomes unresponsive without forcing CSRs to leave their CRM workspace. 

Why it matters 

Embedded deployments let organizations use Dynamics 365 Contact Center inside a nonMicrosoft CRM through a lightweight widget, so Customer Service Representatives (CSR) don’t have to switch tools. Pairing that embedded experience with DCA provides a dedicated voice path that’s independent of the browser, helping eliminate dropped or interrupted calls caused by page reloads, page freezes, or tab navigation. The result is faster call setup, steadier audio, and consistent CSR workflows across CRMs. 

Real world outcomes from earlier DCA adopters include reduced average speed to answer and fewer connectivity issues, underscoring the operational value of a desktop resident voice companion. 

What’s new for Embedded mode 

DCA + Embedded widget: a resilient voice experience inside your CRM UI 

  • Call continuity during browser events 
    • Active calls remain connected when the CRM page refreshes or becomes unresponsive; CSRs can continue the conversation and regain full web context when the tab recovers. 
  • Lower latency, better audio consistency 
    • DCA’s desktop process helps reduce connection delays and smooths device handling, complementing the embedded browser experience 
  • Familiar, lightweight controls 
    • CSRs can mute/unmute and end calls from DCA while the embedded widget reloads; when recording or transcription is enabled in the web app, they continue uninterrupted. 
  • Built for crossCRM 
    • Works alongside the embedded experience in third party CRMs that host the HTML/JavaScript widget. 

The internal brief for Embedded mode reiterates these benefits specifically for external CRM workspaces, including continuity across inbound/outbound workflows. 

How it works 

  1. Route and render: Dynamics 365 Contact Center routes the voice interaction; the embedded widget renders in the CRM for CSR workflows 
  1. Establish desktop voice path: DCA runs as a companion process on the desktop and maintains the call even if the 3P CRM browser reloads, freezes, or loses focus 
  1. Resynchronize: When the page returns, the call state resynchronizes with the embedded widget so the CSR continues in one unified flow. 

Business value 

  • Higher reliability: Fewer dropped calls and better resiliency against browser variability and tab navigation. 
  • Lower latency: Faster connection setup and more responsive audio device handling 
  • CSR productivity: CSRs stay in their CRM UI; DCA protects the call while the page recovers minimizing context loss and reducing redial effort 
  • Operational consistency: A common voice experience across standalone and embedded deployments, aligned to voice best practices. 

Getting started 

  1. Enable Embedded experience 
    • Follow the guide to configure and surface the conversation widget inside your 3P CRM. Retrieve the widget URL from the Copilot Service admin center and complete the setup steps in your CRM. 
  1. Install and manage DCA 
    • Deploy the Desktop Companion App to CSR devices, install the browser extension(s), and (optionally) control updates via policy/registry settings. 
  1. CSR usage 
    • CSRs sign in, handle calls as usual in the embedded widget, and use DCA as needed (e.g., during a refresh). Recording and transcription continue if configured in the web app 
  1. Validate with best practices 
    • Review voice channel best practices for network, device, and telemetry guidance to ensure optimal call performance across your environment 

Learn More

Install and manage Desktop companion application for voice channel | Microsoft Learn 

The post Announcing Desktop Companion App (DCA) support for Dynamics 365 Contact Center Embedded in Third-Party CRMs   appeared first on Microsoft Dynamics 365 Blog.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Reimagining Secure Customer Interactions with Secure Consult & Transfer 

Reimagining Secure Customer Interactions with Secure Consult & Transfer 

This article is contributed. See the original author and article here.

Empowering contact centers to safely handle sensitive, high‑trust customer journeys 

Delivering secure, compliant, and seamless customer experiences is no longer optional — it’s foundational. Across industries such as financial services, healthcare, and public sector, organizations must enable customers to complete high‑trust actions (like payments or identity verification) without exposing sensitive information to agents or core contact center systems. Secure contact center customer journeys enabled through capabilities like Secure Consult & Transfer bring this capability to life by allowing agents to involve external secure endpoints in the conversation while maintaining strict privacy boundaries. 

Today, we’re excited to highlight how Secure Consult & Transfer modernizes sensitive interactions and prepares organizations for the next generation of compliant service workflows. 

How It Works 

Secure numbers are created by applying specific settings to a contact. During runtime, the platform enforces the necessary protections instantly — without requiring manual intervention. The settings are flexible, and can be applied to only consult or transfer, or both. For consult and transfer the administrator can decide to follow workstream recording & transcription settings, stop recording but continue transcription, or stop both transcription & recording. Additionally, during consult, administrators can choose to either put their representatives on hold, or follow workstream settings to have the customer placed on hold with the representative able to take them off. 

Key Benefits at a Glance 

  • Protect sensitive customer data without disrupting workflows. 
  • Enable secure payment and verification scenarios using external trusted endpoints. 
  • Automate compliance controls — recording and transcription management happens instantly and safely. 

Secure Consult & Transfer ushers in a new era of secure, compliant customer interactions — one where sensitive workflows can happen inside the call experience without adding risk, friction, or operational burden. 

If your organization handles sensitive customer actions, now is the time to explore how this capability can strengthen trust, reduce risk, and streamline your service operations. 

Accelerating Secure Contact Center Customer Journeys with DTMF Broadcast 

A modern foundation for high‑trust, compliant, real‑time voice interactions 

As organizations modernize customer engagement, the need for secure and friction‑free voice workflows has become essential—especially for processes that rely on keypad inputs, such as payment authentication, IVR navigation, or identity verification. Traditional DTMF forwarding approaches rely on slow relays, creating latency, reliability gaps, and compliance concerns. 

DTMF Broadcast introduces a new, faster way for participants in a call to share DTMF tones in real time— addressing these gaps by sending DTMF tones from one participant instantly to all non‑hold participants, far faster than traditional forwarding. 

This includes all legs of the call, meaning that representatives can more reliably send tones to external endpoints, and if representatives drop off after a transfer to an external endpoint, customers will still be able to send DTMF to that endpoint, navigating IVRs independently. 

How to Enable DTMF Broadcast 

The toggle to enable DTMF Broadcast for an organization is in the Copilot Service Admin Center under Support Experience->Workspaces->Voice Experiences. 

The post Reimagining Secure Customer Interactions with Secure Consult & Transfer  appeared first on Microsoft Dynamics 365 Blog.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

SharePoint at 25: How Microsoft is putting knowledge to work in the AI era

Microsoft Sovereign Cloud adds governance, productivity, and support for large AI models securely running even when completely disconnected

This article is contributed. See the original author and article here.

Microsoft Sovereign Cloud’s expansion of capabilities includes Azure Local disconnected operations, Microsoft 365 Local disconnected, and Microsoft Foundry addition of large model and modern infrastructure capabilities.

The post Microsoft Sovereign Cloud adds governance, productivity, and support for large AI models securely running even when completely disconnected appeared first on Microsoft 365 Blog.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP

Announcing End of Support for Dynamics 365 Project Service Automation (PSA) on the U.S. Government cloud

This article is contributed. See the original author and article here.

On March 19, 2024, we announced that support for Dynamics 365 Project Service Automation on the commercial cloud will end on March 31, 2025. As planned, end of support went into effect on that date. 

Today we are announcing the end of support for  Dynamics 365 Project Service Automation on the U.S. Government cloud (GCC) beginning March 31, 2027. Dynamics 365 Project Operations has been available in U.S. Government cloud (GCC) since December 2025 

Beginning March 31, 2027, Microsoft will no longer support PSA on GCC environments. There will not be any feature enhancements, updates, bug fixes, or other updates to this offering. Any support ticket logged for the PSA application on GCC will be closed with instructions to upgrade to Project Operations.    

We strongly encourage all PSA customers on GCC environments to start planning your upgrade process as soon as possible so you can take advantage of many new Project Operations features such as:   

  • Integration with Planner capabilities on Dataverse with many new advanced scheduling features  
  • Project Budgeting and Time-phased forecasting    
  • Date Effective price overrides   
  • Revision and Activation on Quotes     
  • Material usage recording in projects and tasks   
  • Subcontract Management   
  • Advances and Retained-based contracts   
  • Contract not-to-exceed   
  • Task and Progress based billing   
  • Multi-Customer contracts   
  • AI and Copilot based experiences.   

 For Project Service Automation customers on GCC High or DoD, we will have a future announcement regarding the availability of Dynamics 365 Project Operations. 

    Learn more about Dynamics 365 Project Operations  

    Project Operations was first released in October 2020 as a comprehensive product to manage Projects from inception to close by bringing together the strengths of Dataverse, Microsoft Dynamics 365 Finance, Microsoft Dynamics 365 Supply Chain Management, and Microsoft Planner. 

    Want to learn more about Project Operations? Check this link and navigate to our detailed documentation!   

    Want to try Project Operations? Click here and sign up for a 30-day trial!    

    The post Announcing End of Support for Dynamics 365 Project Service Automation (PSA) on the U.S. Government cloud appeared first on Microsoft Dynamics 365 Blog.

    Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

    SharePoint at 25: How Microsoft is putting knowledge to work in the AI era

    The ultimate Microsoft 365 community event returns—your front‑row seat to the future of intelligent work

    This article is contributed. See the original author and article here.

    This event is your front-row seat to everything new and next across Microsoft 365—with hundreds of opportunities to learn directly from product makers and connect with the best community in tech.

    The post The ultimate Microsoft 365 community event returns—your front‑row seat to the future of intelligent work appeared first on Microsoft 365 Blog.

    Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

    Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP

    General Availability of Quality Evaluation Agent’s conversation capabilities 

    This article is contributed. See the original author and article here.

    Quality Evaluation Agent in Dynamics 365 Customer Service and Dynamics 365 Contact Center is an AI-led evaluation framework that empowers teams to deliver consistent, scalable quality oversight and automate quality evaluations across customer interactions. 

    Beginning February 6, QEA conversation capabilities become generally available, joining case evaluation as a GA feature, as previously announced in this blog. This milestone expands QEA’s coverage and impact across customer support scenarios. 

    Looking Forward: 

    QEA continues to evolve with key upcoming enhancements across the evaluation framework. This includes multilanguage support, criteria versioning, the ability to flag critical questions, simulation capabilities, knowledge source adherence, and more. 

    Get started today by enabling QEA in your Dynamics 365 Customer Service and Dynamics 365 Contact Center environment.   

    Learn more  

    Watch a quick  video introduction.  

    For configuration steps, feature updates, and best practices, see  Manage Quality Evaluation Agent | Microsoft Learn  

    The post General Availability of Quality Evaluation Agent’s conversation capabilities  appeared first on Microsoft Dynamics 365 Blog.

    Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

    Forrester studies project more than 100% ROI for enterprises and 16-month payback for midmarket organizations using Dynamics 365 ERP

    The Key Facets of AI Evaluation in the Contact Center 

    This article is contributed. See the original author and article here.

    As AI becomes central to contact center operations–powering every customer engagement channel–evaluation is no longer a back-office technical exercise. Evaluation is a critical business capability directly impacting customer experience, operational effectiveness, and business outcome 

    But the evaluation is not one-dimensional. Organizations must think about when, how, by whom, and on what data AI systems are evaluated. This blog explores the key facets of AI evaluation and how they apply specifically to contact center environments. 

     1: Development-Stage vs. Production-Stage Evaluation 

    Definition: Evaluation at development time refers to tests and assessments performed during the AI software creation phase. Production time evaluation occurs after deployment, when the AI software is live and serving real users. 

    Implications: Development-time evaluation provides a controlled environment to identify and address issues before AI reaches customers. This helps reduce risk and prevent costly failures. However, it cannot fully capture real-world complexity. Production-time evaluation reflects actual customer behavior and operating conditions. While it offers critical insight into true performance and experience, manage it carefully to avoid customer impact when issues surface. 

    How contact centers should think about this (example): 
    • Use development-time evaluation to prevent issues. These include incorrect intent detection, poor prompt behavior, broken escalation flows, non-compliant responses, or unacceptable latency before they ever reach customers. 
    • Use production-time evaluation to detect and measure real customer impact, like drops in containment, rising transfers to human agents, customer frustration, regional or channel-specific issues, and performance degradation caused by real traffic patterns. 

     2: Manual vs. Automated Evaluation 

    Definition: Manual execution involves running evaluation tasks at human command. Automated evaluation is run at predetermined time or based on triggers.  

    Implications: Manual evaluation brings human judgment, context, and nuance that automation alone cannot capture, making it especially valuable when evaluation needs are unpredictable, environmental changes are not captured by automated triggers, or when automated runs would be too costly. Automated evaluation complements this, providing consistent, scalable coverage as AI systems evolve, reliably reevaulating systems after known changes, such as releases or configuration updates through CI/CD pipelines.  

    How contact centers should think about this: 

    • Use automation for baseline quality, regression, and continuous monitoring 
    • Use manual evaluation for exceptions, deep dives, and human judgment 
    • The best strategy combines both. Automation ensures assessment of critical changes, while human evaluators interpret results, investigate anomalies, and adapt to unexpected conditions. 

     3: Evaluations Run by the Platform vs. by Customers 

    Definition: Evaluations can be conducted internally by the developer organization (such as Microsoft) or externally by customers using the software in their own environments.  

    Implications: Developer-run and customer-run evaluations each provide distinct and necessary value. Internal evaluations establish a consistent baseline for quality, safety, and compliance. Customer-led evaluations surface real-world behaviors, operational constraints, and usage patterns that cannot be fully anticipated during development. Relying on only one limits visibility and can leave gaps in reliability or usability. 

    How contact centers should think about this: 

    • Rely on platform evaluations to establish a trusted baseline. This ensures core capabilities—such as accuracy, safety and compliance, latency, escalation behavior, and failure handling—meet enterprise standards before features are rolled out broadly. 
    • Platform providers should partner closely with customers. This enables them to run their own evaluations and deeply understand AI performance within their specific domains, workflows, and operating environments. This collaboration helps surface both expected and edge-case behaviors—across positive and negative scenarios 

     4: Synthetic Data vs. Production Traffic 

    Definition: Synthetic data refers to artificially generated datasets designed to simulate specific scenarios. Production traffic comprises actual user interactions and data generated during live operation. 

    Implications: Data Fidelity and Risk: Synthetic data enables safe, repeatable evaluation without exposing sensitive information or impacting real users. However, it may lack the complexity and unpredictability of production data. Production traffic delivers high-fidelity insights but carries risks of data leakage, performance degradation, or user impact. Relevance: Synthetic data is valuable for early-stage, edge-case, or privacy-sensitive evaluations. Production traffic is essential for verifying AI system behavior under real-world conditions. 

    How contact centers should think about this: 

    • Begin with synthetic data to evaluate safely and iterate quickly, especially when testing new scenarios, edge cases, or changes. 
    • Leverage production data to validate performance at scale, ensuring AI behaves as expected under real customer traffic and operating conditions. 
    • Treat production evaluation as a continuous monitoring and learning loop, focused on measuring impact and improving quality—rather than experimenting on live customers. 

     5: Evaluation After vs. During Execution 

    Definition: Post-execution evaluation analyzes the results after a process or test run finishes, while in-execution (real-time) evaluation monitors and assesses behavior as it unfolds.  

    Implications: post-execution evaluation enables deep analysis and long-term improvement, while in-execution evaluation allows faster detection and mitigation of issues. Using both helps contact centers balance insight with real-time protection of customer experience. 

    How contact centers should think about this: 

    • Post-conversation evaluation can provide a large amount of information about correctness, groundedness, resolution effectiveness across completed AI interactions.  
    • Real-time evaluation of empathy and sentiment enables timely intervention, such as escalating to a human agent or allowing supervisor guidance during the interaction 

    Together, these approaches form a core part of AI evaluation in the contact center, helping organizations balance deep analysis with real‑time protections.

    Final Thoughts: A Modern Evaluation Mindset 

    There is no single “right” way to evaluate AI systems. Instead, evaluation should be viewed as a multi-dimensional strategy that evolves alongside your AI systems. 

    By thoughtfully strategizing across evaluation dimensions, organizations can build AI systems that are not only intelligent, but also trustworthy, resilient, and customer-first. Evaluation is no longer optional- it is how modern organizations ensure AI delivers on its promise, every day. 

    Get more details:  

    Measuring What Matters: Redefining Excellence for AI Agents in the Contact Center 

    Evaluating AI Agents in Contact Centers: Introducing the Multi-modal Agents Score 

    The post The Key Facets of AI Evaluation in the Contact Center  appeared first on Microsoft Dynamics 365 Blog.

    Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.